Electricity consumption abnormity group identification method combining association rule mining and KMeans clustering algorithm
By combining association rule mining and KMeans clustering algorithm, the group of electricity consumption abnormalities in the power grid system is identified, and the problems of low identification accuracy and no association of electricity consumption behavior in the prior art are solved, thus achieving efficient group identification of electricity consumption abnormalities and cause analysis.
Patent Information
- Application Number
- CN202410971499.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-08
AI Technical Summary
In power grid systems, it is difficult for the prior art to effectively identify the group of abnormal electricity consumption, resulting in power losses and economic losses. The single algorithm has low recognition accuracy and fails to consider the correlation between electricity consumption behaviors between users.
Combining association rule mining and KMeans clustering algorithm, abnormal groups are identified through data preprocessing, association rule mining and KMeans clustering, feature vectors are extracted and applied to KMeans clustering, and outlier detection is performed to identify abnormal groups.
Effectively identifying the group of abnormal electricity consumption improves the identification accuracy and efficiency, reduces the difficulty and cost of model training, and can find out the possible causes of abnormal electricity consumption behavior, which enhances the explanatory nature of the results.
Smart Images

Figure CN118916771A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of power consumption anomaly detection, and in particular to a power consumption anomaly group recognition method combining association rule mining and KMeans clustering algorithm. Background Art
[0002] There are some abnormal power users in the power grid system. These abnormal power users may have problems such as power equipment failure, energy waste or power theft, which may lead to power loss and additional costs. By identifying abnormal power users, timely measures can be taken to solve the problem and reduce power loss and economic losses. The power consumption behavior of abnormal power users may cause instability in the power supply, such as excessive load, frequent start and stop or unreasonable power consumption pattern.
[0003] Since the original electricity consumption data may contain missing values, outliers, duplicate values, etc., data cleaning and preprocessing are required to ensure the accuracy and completeness of the data. Extracting effective features from the original data is crucial for abnormal group identification, but how to select and extract features related to abnormal groups remains a challenge. In addition, the existing identification of abnormal groups is generally based on data statistical analysis, which is inaccurate and inefficient, and does not consider the correlation between electricity consumption behaviors among users, resulting in identification errors. Summary of the invention
[0004] In order to solve the above problems, the purpose of the present invention is to provide a method for identifying abnormal groups of electricity consumption by combining association rule mining and KMeans clustering algorithm, which can effectively identify abnormal groups based on electricity consumption analysis.
[0005] To achieve the above object, the present invention adopts the following technical solutions:
[0006] A method for identifying abnormal power consumption groups by combining association rule mining and KMeans clustering algorithm includes the following steps:
[0007] Step 1. Obtain the user's electricity consumption data from the smart meter, sensor device or power company's database, including information such as electricity consumption, electricity consumption time, and electricity consumption pattern. Specific data also includes daily / hourly electricity consumption, electricity consumption time period, and type of electricity consumption equipment.
[0008] Step 2: Data preprocessing: cleaning, denoising and normalizing the collected data;
[0009] Step 3: Based on the association rule mining algorithm, discover the association relationship between different electricity consumption characteristics;
[0010] Step 4: Extract features from the results of association rule mining as feature vectors of electricity consumption;
[0011] Step 5: Apply the extracted feature vectors to the KMeans clustering algorithm to divide users into different groups;
[0012] Step 6: Perform anomaly detection on users in each group and use the outlier detection method in the KMeans algorithm to identify abnormal groups.
[0013] Step 7: Visualize the identified abnormal groups.
[0014] Furthermore, step 2 is specifically as follows:
[0015] Step 2.1 first handles missing values, outliers, and duplicate values to ensure the completeness and accuracy of the data.
[0016] For missing values, use interpolation to fill
[0017] For outliers, identify them through statistical analysis or visualization methods and delete them;
[0018] For duplicate values, detect and delete duplicate records to ensure that there is no duplicate information in the data;
[0019] Step 2.2: Use the sliding average method to smooth the data and obtain the data sequence after removing the noise;
[0020] Given a window size of n, the smoothed value of the ith data point is:
[0021]
[0022] Data normalization: The maximum-minimum normalization method is used to process the data to ensure that the numerical ranges between different features are consistent and to improve the comparability of the data.
[0023] Furthermore, step 3 is specifically as follows:
[0024] Step 3.1: According to the characteristics of the data set and the mining objectives, the minimum support is set to 0.1 and the minimum confidence threshold is set to 0.7 to screen frequent item sets and association rules;
[0025] Step 3.2: Mining frequent itemsets based on Apriori algorithm;
[0026] Step 3.3: Generate association rules using the frequent item set mining results and calculate the confidence of the rules;
[0027] For each frequent item set X, generate association rules Where Y is a subset of X, and the confidence of the association rule is calculated:
[0028]
[0029] in,
[0030] Step 3.4: According to the confidence of the rules, select the association rules with higher support and confidence;
[0031] Furthermore, step 4 is specifically as follows:
[0032] Extract frequently occurring features and rule combinations from frequent item sets and association rules;
[0033] Encode the extracted features and convert them into numerical form using one-hot encoding. Encode each feature as 0 or 1, indicating whether the feature appears in the feature combination;
[0034] The encoded features are combined into a feature vector, where each feature corresponds to a dimension in the vector.
[0035] Furthermore, step 5 is specifically as follows:
[0036] Step 5.1: Use the feature vector extracted from the association rule mining results as input data.
[0037] Step 5.2: Determine the optimal K value by using the Elbow Method.
[0038] Step 5.3: Randomly select K data points as initial cluster centers.
[0039] Step 5.4: For each data point, calculate its distance to all cluster centers and assign it to the cluster represented by the nearest cluster center.
[0040] Calculate the distance between the data point x and the cluster center c using the Euclidean distance:
[0041]
[0042] Among them, xi and ci are the coordinates of data point x and cluster center c in the i-th dimension respectively;
[0043] Step 5.5: Recalculate the center point Cj of each cluster, which is usually the mean of all points in the cluster;
[0044]
[0045] Among them, Sj is the set of all data points in the jth cluster;
[0046] Step 5.6: Repeat steps 5.4 and 5.5 until the cluster center no longer changes significantly or the preset number of iterations is reached;
[0047] Step 5.7: Perform feature analysis based on the user groups obtained through clustering to understand the electricity consumption characteristics of different groups.
[0048] Furthermore, step 6 is specifically as follows:
[0049] Perform outlier detection on the feature vectors of users in each group, and use the feature vectors of each group as a sub-dataset;
[0050] Determine the outlier detection metrics, including the distance from the data point to the cluster center;
[0051] Apply the KMeans algorithm to each sub-dataset to obtain the cluster center.
[0052] Calculate the distance between each data point and the center of the cluster to which it belongs, and mark the data points whose distance exceeds a certain threshold as outliers;
[0053] The number of data points marked as outliers in each group is counted to determine which group of users are identified as abnormal groups.
[0054] The present invention has the following beneficial effects:
[0055] 1. The present invention combines association rule mining and KMeans clustering algorithm to realize the identification of abnormal groups of electricity consumption, which can effectively identify abnormal groups based on electricity consumption analysis and provide a reference for further electricity consumption management and optimization;
[0056] 2. The present invention converts the results of association rule mining into feature vectors to better describe the correlation between different features in electricity consumption, and combines K-MEANS clustering and association rule mining algorithms to identify abnormal groups, solving the problem that a single algorithm may not be able to effectively identify abnormal groups. At the same time, it does not require a large amount of labeled data for training, reducing the difficulty and cost of model training. Through association rule mining, not only can abnormal groups be identified, but also the possible causes of abnormal electricity consumption behavior can be found, enhancing the interpretability of the results. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 is a flow chart of the method of the present invention;
[0058] Figure 2 A schematic diagram of the model training process provided by an embodiment of the present invention;
[0059] Figure 3 Schematic diagram of outliers in an embodiment of the present invention. DETAILED DESCRIPTION
[0060] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0061] Please refer to Figure 1 , Figure 1 The flowchart of a method for identifying abnormal groups based on power consumption analysis in an embodiment of the present invention comprises the following steps:
[0062] Step 1. Obtain the user's electricity consumption data from the smart meter, sensor device or power company's database, including information such as electricity consumption, electricity consumption time, and electricity consumption pattern. Specific data also includes daily / hourly electricity consumption, electricity consumption time period, and type of electricity consumption equipment.
[0063] Step 2: Data preprocessing: Clean, denoise and normalize the collected data to ensure data quality and comparability.
[0064] Step 2.1 first handles missing values, outliers, and duplicate values to ensure the completeness and accuracy of the data.
[0065] For missing values, use interpolation to fill
[0066] For outliers, identify them through statistical analysis or visualization methods and delete them;
[0067] For duplicate values, detect and delete duplicate records to ensure that there is no duplicate information in the data;
[0068] Step 2.2: Use the sliding average method to smooth the data and obtain the data sequence after removing the noise;
[0069] Given a window size of n, the smoothed value of the ith data point is:
[0070]
[0071] Data normalization: The maximum-minimum normalization method is used to process the data to ensure that the numerical ranges between different features are consistent and to improve the comparability of the data.
[0072] Step 3: Based on the association rule mining algorithm, discover the association relationship between different electricity consumption characteristics, such as the association rules between electricity consumption and electricity consumption time.
[0073] Step 3.1: According to the characteristics of the data set and the mining objectives, the minimum support is set to 0.1 and the minimum confidence threshold is set to 0.7 to screen frequent item sets and association rules;
[0074] Step 3.2: Mining frequent itemsets based on Apriori algorithm;
[0075] Step 3.3: Generate association rules using the frequent item set mining results and calculate the confidence of the rules;
[0076] For each frequent item set X, generate association rules Where Y is a subset of X, and the confidence of the association rule is calculated:
[0077]
[0078] in,
[0079] Step 3.4: According to the confidence of the rules, select the association rules with higher support and confidence;
[0080] Step 4: Extract features from the results of association rule mining as feature vectors of electricity consumption.
[0081] Extract frequently occurring features and rule combinations from frequent item sets and association rules, such as {A, B}, {B, C}, {A, C}, etc.;
[0082] Encode the extracted features and convert them into numerical form using one-hot encoding. Encode each feature as 0 or 1, indicating whether the feature appears in the feature combination.
[0083] Combine the encoded features into a feature vector, where each feature corresponds to a dimension in the vector;
[0084] Step 5: Apply the extracted feature vectors to the KMeans clustering algorithm to divide users into different groups.
[0085] Step 5.1: Use the feature vector extracted from the association rule mining results as input data.
[0086] Step 5.2: Determine the optimal K value by using the Elbow Method.
[0087] Step 5.3: Randomly select K data points as initial cluster centers.
[0088] Step 5.4: For each data point, calculate its distance to all cluster centers and assign it to the cluster represented by the nearest cluster center.
[0089] Calculate the distance between the data point x and the cluster center c using the Euclidean distance:
[0090]
[0091] Among them, xi and ci are the coordinates of data point x and cluster center c in the i-th dimension respectively;
[0092] Step 5.5: Recalculate the center point Cj of each cluster, which is usually the mean of all points in the cluster;
[0093]
[0094] Among them, Sj is the set of all data points in the jth cluster;
[0095] Step 5.6: Repeat steps 5.4 and 5.5 until the cluster center no longer changes significantly or the preset number of iterations is reached;
[0096] Step 5.7: Perform feature analysis based on the user groups obtained through clustering to understand the electricity consumption characteristics of different groups.
[0097] Step 6: Perform anomaly detection on users in each group and use the outlier detection method in the KMeans algorithm to identify abnormal groups.
[0098] Specifically: outlier detection is performed on the feature vectors of users in each group, and the feature vectors of each group are used as a sub-dataset.
[0099] Determine the outlier detection metrics, including the distance from the data point to the cluster center;
[0100] Apply the KMeans algorithm to each sub-dataset to obtain the cluster center.
[0101] The distance from each data point to the center of the cluster to which it belongs is calculated, and data points whose distance exceeds a certain threshold are marked as outliers.
[0102] The number of data points marked as outliers in each group is counted to determine which group of users are identified as abnormal groups.
[0103] Step 7: Visualize the identified abnormal groups and conduct in-depth analysis of abnormal power usage behaviors to find out the causes of the abnormalities and possible solutions.
[0104] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0105] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0106] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0108] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.
Claims
1. A method for identifying abnormal power consumption groups by combining association rule mining and KMeans clustering algorithm, characterized in that: The following steps are involved: Step 1. Obtain the user's electricity consumption data from the smart meter, sensor device or power company's database, including information such as electricity consumption, electricity consumption time, and electricity consumption pattern. Specific data also includes daily / hourly electricity consumption, electricity consumption time period, and type of electricity consumption equipment. Step 2: Data preprocessing: cleaning, denoising and normalizing the collected data; Step 3: Based on the association rule mining algorithm, discover the association relationship between different electricity consumption characteristics; Step 4: Extract features from the results of association rule mining as feature vectors of electricity consumption; Step 5: Apply the extracted feature vectors to the KMeans clustering algorithm to divide users into different groups; Step 6: Perform anomaly detection on users in each group and use the outlier detection method in the KMeans algorithm to identify abnormal groups. Step 7: Visualize the identified abnormal groups.
2. According to claim 1, the method for identifying abnormal power consumption groups by combining association rule mining and KMeans clustering algorithm is characterized in that: The step 2 is specifically as follows: Step 2.1 first handles missing values, outliers, and duplicate values to ensure the completeness and accuracy of the data. For missing values, use interpolation to fill For outliers, identify them through statistical analysis or visualization methods and delete them; For duplicate values, detect and delete duplicate records to ensure that there is no duplicate information in the data; Step 2.2: Use the sliding average method to smooth the data and obtain the data sequence after removing the noise; Given a window size of n, the smoothed value of the ith data point is: Data normalization: The maximum-minimum normalization method is used to process the data to ensure that the numerical ranges between different features are consistent and to improve the comparability of the data.
3. The method for identifying abnormal electricity consumption groups by combining association rule mining and KMeans clustering algorithm according to claim 1 is characterized in that: The step 3 is specifically as follows: Step 3.1: According to the characteristics of the data set and the mining objectives, the minimum support is set to 0.1 and the minimum confidence threshold is set to 0.7 to screen frequent item sets and association rules; Step 3.2: Mining frequent itemsets based on Apriori algorithm; Step 3.3: Generate association rules using the frequent item set mining results and calculate the confidence of the rules; For each frequent item set X, generate association rules Where Y is a subset of X, and the confidence of the association rule is calculated: in, Step 3.4: According to the confidence of the rules, filter out the support of the association rules with higher support and confidence.
4. The method for identifying abnormal power consumption groups by combining association rule mining and KMeans clustering algorithm according to claim 1 is characterized in that: The step 4 is specifically as follows: Extract frequently occurring features and rule combinations from frequent item sets and association rules; Encode the extracted features and convert them into numerical form using one-hot encoding. Encode each feature as 0 or 1, indicating whether the feature appears in the feature combination; The encoded features are combined into a feature vector, where each feature corresponds to a dimension in the vector.
5. The method for identifying abnormal electricity consumption groups by combining association rule mining and KMeans clustering algorithm according to claim 1 is characterized in that: The step 5 is specifically as follows: Step 5.1: Use the feature vector extracted from the association rule mining results as input data. Step 5.2: Determine the optimal K value by using the Elbow Method. Step 5.3: Randomly select K data points as initial cluster centers. Step 5.4: For each data point, calculate its distance to all cluster centers and assign it to the cluster represented by the nearest cluster center. Calculate the distance between the data point x and the cluster center c using the Euclidean distance: Among them, xi and ci are the coordinates of data point x and cluster center c in the i-th dimension respectively; Step 5.5: Recalculate the center point Cj of each cluster, which is usually the mean of all points in the cluster; Among them, Sj is the set of all data points in the jth cluster; Step 5.6: Repeat steps 5.4 and 5.5 until the cluster center no longer changes significantly or the preset number of iterations is reached; Step 5.7: Perform feature analysis based on the user groups obtained through clustering to understand the electricity consumption characteristics of different groups.
6. The method for identifying abnormal power consumption groups by combining association rule mining and KMeans clustering algorithm according to claim 1 is characterized in that: The step 6 is specifically as follows: Perform outlier detection on the feature vectors of users in each group, and use the feature vectors of each group as a sub-dataset; Determine the outlier detection metrics, including the distance from the data point to the cluster center; Apply the KMeans algorithm to each sub-dataset to obtain the cluster center. Calculate the distance between each data point and the center of the cluster to which it belongs, and mark the data points whose distance exceeds a certain threshold as outliers; The number of data points marked as outliers in each group is counted to determine which group of users are identified as abnormal groups.
Citation Information
Patent Citations
System and method of smart power grid big data mining for supporting smart city operation management
CN104881735A
Detection method for distributed local outliers
CN106503086A
Gas multi-factor coupling relation analysis and early warning model construction method
CN112926270A
Energy power supply and demand system based on block chain
CN115455072A
User power consumption data anomaly analysis method and system based on power consumption information collection
CN116976707A
Cited By
Abnormal electricity utilization detection method based on electricity utilization analysis
CN121524879A
Power load abnormity early warning method and device combined with power utilization behavior big data mining
CN121637039A
Fusion deep mode mining of electricity theft inspection rule intelligent agent construction method
CN122798445A
