Deep learning-dominated stacked machine learning and deep learning electricity larceny detection method and system

Abstract

The invention provides a deep learning-dominated stacked machine learning and deep learning electricity stealing detection method and system, and the method comprises the following steps: S1, obtaining a data set, and carrying out the cleaning of the data set; s2, carrying out normalization processing on the cleaned data; s3, dividing the obtained preprocessed data set into a training set and a test set; s4, balancing the two sample categories by adopting an SMOTE + Tomek-Links comprehensive sampling method for training a subsequent electricity larceny detection model; s5, constructing and training an electricity stealing detection model; the electricity stealing detection model comprises a deep learning feature extraction module, a support vector machine SVM, a random forest RF, a gradient boosting decision tree GBDT and an extreme gradient boosting tree XGBoost; and S6, inputting to-be-detected data into the trained electricity larceny detection model to obtain a detection result. According to the invention, the SMOTE + Tomek-Links is used for processing the unbalance degree problem of data, and the pre-classification output of machine learning is stacked through the feature extraction module of deep learning to serve as the input of XGBoost, so that the precision of electricity stealing behavior detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of abnormal data detection and neural network, and in particular to a method and system for detecting electricity theft based on stacked machine learning and deep learning dominated by deep learning. Background Art

[0002] Smart grids are designed to provide high-quality, safe, and reliable electricity while addressing environmental pollution and energy shortages. They achieve sustainable energy solutions by optimizing energy management, improving grid efficiency, and integrating renewable energy. Although the widespread deployment of smart meters has greatly improved data measurement accuracy, collection efficiency, and analysis capabilities, the field still faces many challenges such as power theft. These obstacles mainly stem from the complexity of technical architecture, inconsistent regulatory frameworks, and the continuous development of illegal technical methods. According to a survey by the Northeast Group, power theft causes economic losses of $64.7 billion each year. In addition, according to incomplete statistics, the annual loss caused by power theft in Fujian Province is about 100 million yuan.

[0003] Driven by economic and industrial needs, certain mechanisms for detecting electricity theft are usually set up. At the same time, consumers often set a pre-set limit on electricity purchases, but non-technical losses will eventually increase the economic burden on end users. In recent years, reducing non-technical losses has become one of the key drivers of the development of smart grids. With the help of advanced technologies such as big data analysis, the level of intelligence in the operation of power systems has been significantly improved, and the supervision of electricity theft has also become an important indicator. By strengthening the control of electricity theft, the public sector can effectively reduce energy consumption, optimize electricity regulation, and achieve a more reasonable allocation of power resources within a certain period of time. This not only brings economic benefits to the cost of power generation, but also reduces the occurrence of a large number of violations at the planning and resource allocation levels. Accurate and efficient identification of electricity theft problems can help narrow the gap between supply and demand, ensure the safe and efficient operation of the power management system, thereby improving the reliability of energy utilization and reducing uncertainty in power production. However, electricity theft is dynamic and complex, involving multiple energy consumption patterns and showing a nonlinear development trend over time. At the same time, electricity demand is affected by many factors, including changes in intrinsic demand, fluctuations in fuel prices, and the supply and transportation of renewable energy. The frequent fluctuations in electricity demand require smart meters to monitor these variables in real time. The huge amount of data generated poses a challenge to analysis, and the data processing of smart sensors and smart sub-meters becomes increasingly complex. Summary of the invention

[0004] The purpose of the present invention is to propose a method and system for detecting electricity theft by stacking machine learning and deep learning with deep learning as the leading factor, using SMOTE+Tomek-Links to process the imbalance problem of data, and then stacking the pre-classification output of machine learning through the feature extraction module of deep learning as the input of XGBoost, which can improve the accuracy of electricity theft detection.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] A method for detecting electricity theft by stacking machine learning and deep learning with deep learning as the main method comprises the following steps:

[0007] Step S1: Obtain a data set and clean it, including filling missing values ​​and processing outliers;

[0008] Step S2: normalize the cleaned data to obtain a preprocessed data set;

[0009] Step S3: Divide the obtained preprocessed data set into a training set and a test set;

[0010] Step S4: For the imbalance problem of the binary classification of normal users and electricity theft users, the SMOTE+Tomek-Links comprehensive sampling method is used for the training set and the test set to balance the two sample categories for the subsequent training of the electricity theft detection model;

[0011] Step S5: constructing and training an electricity theft detection model for the classification problem; the electricity theft detection model includes a deep learning feature extraction module, a support vector machine SVM, a random forest RF, a gradient boosting decision tree GBDT and an extreme gradient boosting tree XGBoost;

[0012] The deep learning feature extraction module includes multiple parallel convolution feature extraction modules PCFEM, a flattening layer and a fully connected layer. After the model input data passes through the multiple parallel convolution feature extraction modules PCFEM, the tensor is first flattened and then processed through the fully connected layer to obtain the features extracted by deep learning;

[0013] The support vector machine SVM, random forest RF, and gradient boosting decision tree GBDT respectively pre-classify the model input data;

[0014] The extreme gradient boosting tree XGBoost takes the features extracted by deep learning and the three pre-classification label results as input to obtain the final classification result;

[0015] Step S6: input the data to be tested into the trained electricity theft detection model to obtain the detection result.

[0016] Preferably, the step S1 specifically includes the following steps:

[0017] Step S11: Use Python to analyze the data, find out the points where the data is missing and count the number;

[0018] Step S12: Preset a first threshold N 1 and the second threshold N 2 , and N 1 ﹥N 2 ; When the number of empty values ​​in the user's electricity consumption record exceeds N 1 , delete the user from the data set; the number of null values ​​is greater than or equal to N 2 Replace the null value with 0; if the number of null values ​​is less than N 2 , interpolate the null values ​​with the mean value;

[0019] Step S13: Eliminate erroneous values ​​and restore data after interpolation processing by using the 3σ principle to complete data outlier processing;

[0020] Step S14: Calculate the average power consumption Avg(x) and standard deviation σ(x) of the user

[0021] Step S15: When the user's power consumption data x at a certain point i Satisfy x i >Avg(x)+2σ(x), let

[0022] f(x i )=Avg(x)+2σ(x)

[0023] Where x represents the user's electricity consumption data vector.

[0024] Preferably, the first threshold N is set 1 Set the second threshold N to 600. 2 is 7.

[0025] Preferably, the calculation of the normalization process is specifically as follows:

[0026]

[0027] In the formula, x represents the electricity consumption data vector of a single user, x i represents the electricity consumption data of the i-th sampling point, g(x i ) represents the normalized data of the i-th sampling point.

[0028] Preferably, the step S4 specifically includes the following steps:

[0029] Step S41: Use the SMOTE algorithm to analyze the samples and artificially synthesize new samples based on the minority class samples;

[0030] Step S42: Using the Tomek-Links undersampling algorithm, the distance between two sample points is calculated to establish a Tomek-link pair, and the samples belonging to the majority class in the Tomek-link pair are removed.

[0031] Preferably, the new samples are artificially synthesized based on the minority class samples; the new samples are constructed specifically according to the following formula:

[0032] x new =x+rand(0,1)×(x′-x)

[0033] Where x and x′ are the minority class samples and their nearest neighbor samples, respectively. new A new sample for the build.

[0034] Preferably, the training of the electricity theft detection model includes:

[0035] Binary cross entropy loss is used as the loss function to calculate the loss between the actual category and the predicted category for the training of the deep learning feature extraction module; the specific loss function is as follows:

[0036]

[0037] where y i is an input-output pair (x i ,y i ), N represents the number of samples, h θ (x i ) represents the model prediction category;

[0038] Set the hyperparameter range for support vector machine SVM, random forest RF, gradient boosted decision tree GBDT and extreme gradient boosted tree XGBoost, and use the Optuna library to optimize the parameters and find the optimal parameters as the final parameters of the model.

[0039] Preferably, the deep learning feature extraction module includes 6 parallel convolution feature extraction modules PCFEM.

[0040] Preferably, the deep learning feature extraction module specifically performs the following operations:

[0041] In the parallel convolution feature extraction module PCFEM, the input data first undergoes a convolution step, and then undergoes feature splicing after parallel convolution of two convolution kernels of different sizes; the spliced ​​features are extracted through convolution with a step size;

[0042] After passing through 6 parallel convolutional feature extraction modules PCFEM, the tensor is flattened by the flattening layer and then passed through the fully connected layer to obtain the result.

[0043] A system for detecting electricity theft using stacked machine learning and deep learning with deep learning as the main method, comprising a processor, a memory and a computer program stored in the memory. When the processor executes the computer program, the steps in the above-mentioned method for detecting electricity theft using stacked machine learning and deep learning with deep learning as the main method are specifically performed.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention uses SMOTE+Tomek-Links to process the imbalance problem of data, and then stacks the pre-classification output of machine learning through the feature extraction module of deep learning as the input of XGBoost. Compared with a single deep learning method or a machine learning method that uses a single model for classification detection, the present invention can effectively improve the accuracy of electricity theft detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the principle flow of a method for detecting electricity theft according to an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of the distribution of data before and after comprehensive sampling according to an embodiment of the present invention;

[0048] Figure 3 This is a diagram of the PCFEM architecture of a parallel convolution feature extraction module according to an embodiment of the present invention;

[0049] Figure 4 Schematic diagram of different prediction results of data before and after comprehensive sampling according to an embodiment of the present invention;

[0050] Figure 5 4 is a performance comparison chart under different models of an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following is combined with Figure 1-4 , the technical solution of the present invention is specifically described.

[0052] like Figure 1 As shown, the present invention proposes a method for detecting electricity theft by stacking machine learning and deep learning with deep learning as the main method, comprising the following steps:

[0053] Step S1: Obtain a data set. Since there are missing values ​​and outliers in the data set, clean the data set, including filling missing values ​​and processing outliers; improve its data quality to improve the accuracy of subsequent detection;

[0054] Step S2: normalize the cleaned data to obtain a preprocessed data set to improve the training speed of the model;

[0055] Step S3: Divide the obtained preprocessed data set into a training set and a test set;

[0056] Step S4: For the imbalance problem of the binary classification of normal users and electricity theft users, the SMOTE+Tomek-Links comprehensive sampling method is used for the training set and the test set to balance the two sample categories for the subsequent training of the electricity theft detection model;

[0057] Step S5: constructing and training an electricity theft detection model for the classification problem; the electricity theft detection model includes a deep learning feature extraction module, a support vector machine SVM, a random forest RF, a gradient boosting decision tree GBDT and an extreme gradient boosting tree XGBoost;

[0058] The deep learning feature extraction module includes multiple parallel convolution feature extraction modules PCFEM, a flattening layer and a fully connected layer. After the model input data passes through the multiple parallel convolution feature extraction modules PCFEM, the tensor is first flattened and then processed through the fully connected layer to obtain the features extracted by deep learning;

[0059] The support vector machine SVM, random forest RF, and gradient boosting decision tree GBDT respectively pre-classify the model input data and obtain pre-classification results;

[0060] The extreme gradient boosting tree XGBoost takes the features extracted by deep learning and the three pre-classification label results as input to obtain the final classification result;

[0061] Step S6: input the data to be tested into the trained electricity theft detection model to obtain the detection result.

[0062] In this embodiment, step S1 specifically includes the following steps:

[0063] Step S11: Use Python to analyze the data, find out the points where the data is missing and count the number;

[0064] Step S12: Preset a first threshold N 1 and the second threshold N 2 , and N 1 ﹥N 2 ; When the number of empty values ​​in the user's electricity consumption record exceeds N 1 , delete the user from the data set; the number of null values ​​is greater than or equal to N 2 Replace the null value with 0; if the number of null values ​​is less than N 2 , interpolate the null values ​​with the mean value;

[0065] Step S13: Eliminate erroneous values ​​and restore data after interpolation processing by using the 3σ principle to complete data outlier processing;

[0066] Step S14: Calculate the average power consumption Avg(x) and standard deviation σ(x) of the user

[0067] Step S15: When the user's power consumption data x at a certain point i Satisfy x i >Avg(x)+2σ(x), let

[0068] f(x i )=Avg(x)+2σ(x)

[0069] Where x represents the user's electricity consumption data vector.

[0070] In this embodiment, the first threshold N is set 1 Set the second threshold N to 600. 2 is 7.

[0071] In this embodiment, the calculation of the normalization process is specifically as follows:

[0072]

[0073] In the formula, x represents the electricity consumption data vector of a single user, x i represents the electricity consumption data of the i-th sampling point, g(x i ) represents the normalized data of the i-th sampling point.

[0074] In this embodiment, step S4 specifically includes the following steps:

[0075] Step S41: Use the SMOTE algorithm to analyze the samples and artificially synthesize new samples based on the minority class samples;

[0076] Step S42: Calculate the distance between two sample points to establish a Tomek-link pair using the Tomek-Links undersampling algorithm, and remove the samples belonging to the majority class in the Tomek-link pair; Figure 2 It is the distribution of data before and after comprehensive sampling.

[0077] In this embodiment, the new samples are artificially synthesized based on the minority class samples; the new samples are constructed specifically according to the following formula:

[0078] x new =x+rand(0,1)×(x′-x)

[0079] Where x and x′ are the minority class samples and their nearest neighbor samples, respectively. new A new sample for the build.

[0080] Preferably, the training of the electricity theft detection model includes:

[0081] Binary cross entropy loss is used as the loss function to calculate the loss between the actual category and the predicted category for the training of the deep learning feature extraction module; the specific loss function is as follows:

[0082]

[0083] where y i is an input-output pair (x i ,y i ), N represents the number of samples, h θ (x i ) represents the model prediction category;

[0084] Set the hyperparameter range for support vector machine SVM, random forest RF, gradient boosted decision tree GBDT and extreme gradient boosted tree XGBoost, and use the Optuna library to optimize the parameters and find the optimal parameters as the final parameters of the model.

[0085] In this embodiment, the deep learning feature extraction module includes 6 parallel convolution feature extraction modules PCFEM.

[0086] In this embodiment, the deep learning feature extraction module specifically performs the following operations:

[0087] In the parallel convolution feature extraction module PCFEM, the input data first undergoes a convolution step, and then undergoes feature splicing after parallel convolution of two convolution kernels of different sizes; the spliced ​​features are extracted through convolution with a step size; at the same time, the problem of too many parameters caused by stacking modules is avoided;

[0088] After passing through 6 parallel convolution feature extraction modules PCFEM, the tensor is flattened by the flattening layer, and then the result is obtained by the fully connected layer;

[0089] During the model training process, a series of repeated models are generated under different parameter settings, and the binary cross entropy loss metric described above is used to measure the final prediction accuracy. The smaller the binary cross entropy loss, the more accurate the feature extraction result. Figure 3 The specific architecture of the parallel convolutional feature extraction module (PCFEM).

[0090] Figure 4 and Figure 5 They are respectively different prediction results of data before and after comprehensive sampling according to an embodiment of the present invention, and performance comparison diagrams under different models.

[0091] The present invention also provides a system for detecting electricity theft by stacking machine learning and deep learning with deep learning as the main method, comprising a processor, a memory and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the above-mentioned method for detecting electricity theft by stacking machine learning and deep learning with deep learning as the main method.

[0092] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A method for detecting electricity theft by stacking machine learning and deep learning with deep learning as the main method, characterized in that: The following steps are involved: Step S1: Obtain a data set and clean it, including filling missing values ​​and processing outliers; Step S2: normalize the cleaned data to obtain a preprocessed data set; Step S3: Divide the obtained preprocessed data set into a training set and a test set; Step S4: For the imbalance problem of the binary classification of normal users and electricity theft users, the SMOTE+Tomek-Links comprehensive sampling method is used for the training set and the test set to balance the two sample categories for the subsequent training of the electricity theft detection model; Step S5: constructing and training an electricity theft detection model for the classification problem; the electricity theft detection model includes a deep learning feature extraction module, a support vector machine SVM, a random forest RF, a gradient boosting decision tree GBDT and an extreme gradient boosting tree XGBoost; The deep learning feature extraction module includes multiple parallel convolution feature extraction modules PCFEM, a flattening layer and a fully connected layer. After the model input data passes through the multiple parallel convolution feature extraction modules PCFEM, the tensor is first flattened and then processed through the fully connected layer to obtain the features extracted by deep learning; The support vector machine SVM, random forest RF, and gradient boosting decision tree GBDT respectively pre-classify the model input data; The extreme gradient boosting tree XGBoost takes the features extracted by deep learning and the three pre-classification label results as input to obtain the final classification result; Step S6: input the data to be tested into the trained electricity theft detection model to obtain the detection result.

2. According to claim 1, a method for detecting electricity theft by stacking machine learning and deep learning with deep learning as the main method, characterized in that: The step S1 specifically includes the following steps: Step S11: Use Python to analyze the data, find out the points where the data is missing and count the number; Step S12: Preset a first threshold value N1 and a second threshold value N2, and N1>N2; when the number of null values ​​in the user's electricity consumption record exceeds N1, delete the user from the data set; when the number of null values ​​is greater than or equal to N2; replace the null values ​​with 0; when the number of null values ​​is less than N2, interpolate the null values ​​with the average value; Step S13: Eliminate erroneous values ​​and restore data after interpolation processing by using the 3σ principle to complete data outlier processing; Step S14: Calculate the average power consumption Avg(x) and standard deviation σ(x) of the user Step S15: When the user's power consumption data x at a certain point i Satisfy x i >Avg(x)+2σ(x), let f(x i )=Avg(x)+2σ(x) Where x represents the user's electricity consumption data vector.

3. The method for detecting electricity theft based on stacked machine learning and deep learning with deep learning as the main method according to claim 2, characterized in that: The first threshold N1 is set to 600, and the second threshold N2 is set to 7.

4. According to claim 1, a method for detecting electricity theft by stacking machine learning and deep learning with deep learning as the leading factor, characterized in that: The calculation of the normalization process is specifically as follows: In the formula, x represents the electricity consumption data vector of a single user, x i represents the electricity consumption data of the i-th sampling point, g(x i ) represents the normalized data of the i-th sampling point.

5. According to claim 1, a method for detecting electricity theft by stacking machine learning and deep learning with deep learning as the main method, characterized in that: The step S4 specifically comprises the following steps: Step S41: Use the SMOTE algorithm to analyze the samples and artificially synthesize new samples based on the minority class samples; Step S42: Using the Tomek-Links undersampling algorithm, the distance between two sample points is calculated to establish a Tomek-link pair, and the samples belonging to the majority class in the Tomek-link pair are removed.

6. The method for detecting electricity theft based on stacked machine learning and deep learning with deep learning as the main method according to claim 5, characterized in that: According to the minority class samples, new samples are artificially synthesized; specifically, new samples are constructed according to the following formula: x new =x+rand(0,1)×(x′-x) Where x and x′ are the minority class samples and their nearest neighbor samples, respectively. new A new sample for the build.

7. The method for detecting electricity theft based on stacked machine learning and deep learning with deep learning as the main method according to claim 1, characterized in that: The training of the electricity theft detection model includes: Binary cross entropy loss is used as the loss function to calculate the loss between the actual category and the predicted category for the training of the deep learning feature extraction module; the specific loss function is as follows: where y i is an input-output pair (x i ,y i ), N represents the number of samples, h θ (x i ) represents the model prediction category; Set the hyperparameter range for support vector machine SVM, random forest RF, gradient boosted decision tree GBDT and extreme gradient boosted tree XGBoost, and use the Optuna library to optimize the parameters and find the optimal parameters as the final parameters of the model.

8. The method for detecting electricity theft based on stacked machine learning and deep learning with deep learning as the main method according to claim 1, characterized in that: The deep learning feature extraction module includes 6 parallel convolution feature extraction modules PCFEM.

9. The method for detecting electricity theft based on stacked machine learning and deep learning with deep learning as the main method according to claim 8, characterized in that: The deep learning feature extraction module specifically performs the following operations: In the parallel convolution feature extraction module PCFEM, the input data first undergoes a convolution step, and then undergoes feature splicing after parallel convolution of two convolution kernels of different sizes; the spliced ​​features are extracted through convolution with a step size; After passing through 6 parallel convolutional feature extraction modules PCFEM, the tensor is flattened by the flattening layer and then passed through the fully connected layer to obtain the result.

10. A deep learning-based electricity theft detection system that stacks machine learning and deep learning, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory. When the processor executes the computer program, the method specifically performs the steps in the method for detecting electricity theft by stacking machine learning and deep learning with deep learning as the leading factor as described in any one of claims 1 to 9.

Images