Abnormal power consumption detection method, device and medium based on semi-fixed integrated data

Abstract

The present invention discloses an abnormal power consumption detection method, device, equipment and medium based on semi-fixed integrated data. The method includes obtaining original power consumption data and classifying abnormal data values ​​therein, simulating abnormal power consumption behavior according to data features in different types of abnormal data sets to expand each abnormal data set, using a semi-fixed sample generation method to recombinant the expanded abnormal data set with a preset number of random normal power consumption data sets to obtain several sample subsets, training each sample subset and constructing an abnormal power consumption data detection model, and using the abnormal power consumption data detection model to perform abnormal power consumption detection. The present invention effectively increases the number of abnormal power consumption data samples through data expansion technology, combines semi-fixed sample combinations with diversified machine learning integrated algorithms, improves the detection accuracy and generalization ability of the model, and ensures the accuracy of the system when facing complex unbalanced power data sets.

Description

Technical Field

[0001] The present invention relates to the field of data detection technology, and in particular to an abnormal power consumption detection method, device, equipment and medium based on semi-fixed integrated data. Background Art

[0002] In the power system, real-time monitoring and analysis of power consumption data is crucial for the stable operation of the power grid. However, due to the diversity and uncertainty of power consumption patterns, power consumption data often presents a serious class imbalance problem, that is, the samples of normal power consumption patterns are far more than the samples of abnormal power consumption patterns.

[0003] Traditional methods for detecting abnormal electricity consumption data, such as statistical methods and simple machine learning models, are usually unable to effectively deal with the bias problem caused by unbalanced data sets. When faced with a large amount of normal electricity consumption data and a small amount of abnormal electricity consumption data, the model is prone to ignore abnormal patterns, resulting in missed detections.

[0004] It can be seen that how to effectively handle the problem of anomaly detection in unbalanced data in the power system has become a technical problem that technical personnel in this field need to solve urgently. Summary of the invention

[0005] The present invention provides an abnormal power consumption detection method, device and medium based on semi-fixed integrated data, so as to detect abnormal power consumption data in a power system.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides an abnormal power consumption detection method based on semi-fixed integrated data, comprising:

[0007] The original power consumption data is obtained through the power system, and the original power consumption data is divided into a training set, a validation set and a test set according to a time series.

[0008] According to the variation range of the abnormal data values ​​in the training set and the validation set, the abnormal data values ​​are classified to obtain different types of abnormal data sets.

[0009] Feature extraction is performed on the data in each of the abnormal data sets to obtain corresponding abnormal features, and abnormal power consumption behaviors are simulated according to the abnormal features, power analysis is performed on each of the abnormal power consumption behaviors, and each of the abnormal data sets is expanded based on the analysis results.

[0010] A semi-fixed sample generation method is used to reorganize each of the expanded abnormal data sets with a preset number of random normal power consumption data sets to obtain a plurality of sample subsets.

[0011] A machine learning algorithm is used to train each of the sample subsets to obtain a number of base learners corresponding to each of the sample subsets. Each of the base learners is screened according to preset performance indicators, and the screened base learners are integrated to complete the construction of the abnormal electricity consumption data detection model.

[0012] The abnormal power consumption data detection model is used to detect abnormal power consumption data in the test set data, and the abnormal power consumption data detection model is optimized according to the detection result, and abnormal power consumption detection is performed with the optimized abnormal power consumption data detection model.

[0013] Further, the feature extraction of the data in each abnormal data set to obtain corresponding abnormal features, simulating abnormal power consumption behaviors according to the abnormal features, performing power analysis on each abnormal power consumption behavior, and expanding each abnormal data set based on the analysis results includes:

[0014] The power consumption characteristics of the data in each of the abnormal data sets are extracted, wherein the power consumption characteristics include power consumption, power consumption time, power consumption frequency and power factor.

[0015] Abnormal power consumption behavior is simulated according to the power consumption characteristics of each type of abnormal data, and a number of abnormal power consumption data are generated according to the simulated abnormal power consumption behavior, and each abnormal power consumption data is collected into the corresponding abnormal data set to expand the capacity of each abnormal data set.

[0016] Furthermore, the semi-fixed sample generation method is used to reorganize each of the expanded abnormal data sets with a preset number of random normal power consumption data sets to obtain several sample subsets, including:

[0017] Each of the abnormal data sets after expansion is taken as a fixed set, and a number of groups of normal power consumption data of a preset number are randomly selected from the normal power consumption data by sampling with replacement as a random normal power consumption data set.

[0018] Each of the random normal power consumption data sets is combined with the data in the abnormal data set to obtain a plurality of sample subsets.

[0019] Further, the machine learning algorithm is used to train each of the sample subsets to obtain a plurality of base learners corresponding to each of the sample subsets, each of the base learners is screened according to a preset performance indicator, and each of the screened base learners is integrated to complete the construction of the abnormal power consumption data detection model, including:

[0020] Each of the sample subsets is trained using a machine learning algorithm, and a plurality of base learners are generated for each of the sample subsets.

[0021] According to the abnormal data detection accuracy of each base learner, the base learners with performance lower than the preset performance index are screened out, and the Boosting algorithm is used to assign weights corresponding to different abnormal data types to each base learner and integrate them to obtain an abnormal power consumption data detection model.

[0022] Furthermore, after obtaining the original power consumption data through the power system, the method further includes:

[0023] The raw electricity consumption data is preprocessed, and the preprocessing operation includes data cleaning, missing value processing and data labeling.

[0024] The missing value processing is to use the mean filling method to fill the missing values ​​in the original power consumption data To fill:

[0025]

[0026] in, is the observed value, is the number of non-missing values.

[0027] Furthermore, the classifying the abnormal data values ​​according to the variation ranges of the abnormal data values ​​in the training set and the validation set includes:

[0028] Data analysis is performed on each of the abnormal data values. According to the data characteristics of each of the abnormal data values, abnormal data values ​​of abnormal decrease in power consumption and sudden drop to below a preset level are classified as power theft types, abnormal data values ​​of sudden increase or decrease in power consumption and abnormal periodic fluctuations are classified as equipment failure types, and power consumption significantly higher than the normal level is classified as high energy consumption type.

[0029] Furthermore, the screening of each base learner according to a preset performance indicator includes:

[0030] Set the corresponding AUC threshold for each sample subset corresponding to each abnormal data type, and select the base learner with an AUC value higher than the AUC threshold according to the AUC value under the ROC curve. :

[0031]

[0032] in, is the base learner to be screened, is the number of base learners, is the AUC threshold.

[0033] Another embodiment of the present invention provides an abnormal power consumption detection device based on semi-fixed integrated data, comprising:

[0034] The data partitioning module is used to obtain original power consumption data through the power system, and divide the original power consumption data into a training set, a verification set and a test set according to a time series.

[0035] The data classification module is used to classify the abnormal data values ​​according to the variation range of the abnormal data values ​​in the training set and the validation set to obtain different types of abnormal data sets.

[0036] The data expansion module is used to extract features from the data in each of the abnormal data sets to obtain corresponding abnormal features, simulate abnormal power consumption behaviors according to the abnormal features, perform power analysis on each of the abnormal power consumption behaviors, and expand each of the abnormal data sets based on the analysis results.

[0037] The sample generation module is used to reorganize each of the abnormal data sets after expansion with a preset number of random normal power consumption data sets using a semi-fixed sample generation method to obtain a plurality of sample subsets.

[0038] The model building module is used to use a machine learning algorithm to train each of the sample subsets to obtain a number of base learners corresponding to each of the sample subsets, screen each of the base learners according to preset performance indicators, and integrate the screened base learners to complete the construction of the abnormal power consumption data detection model.

[0039] A data detection module is used to detect abnormal power consumption data in the test set data using the abnormal power consumption data detection model, optimize the abnormal power consumption data detection model according to the detection result, and perform abnormal power consumption detection with the optimized abnormal power consumption data detection model.

[0040] Another embodiment of the present invention provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the abnormal power consumption detection method based on semi-fixed integrated data as described above when executing the computer program.

[0041] Yet another embodiment of the present invention provides a computer-readable storage medium storing a computer program, wherein when the device where the computer-readable storage medium is located executes the computer program, the abnormal power consumption detection method based on semi-fixed integrated data as described above is implemented.

[0042] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0043] (1) The number of abnormal electricity consumption data samples is effectively increased through data expansion technology. The detection accuracy and generalization ability of the model are improved by combining semi-fixed sample combinations with diversified machine learning integration algorithms.

[0044] (2) The integrated learning of multiple base learners further optimizes the model performance, ensuring the high accuracy and robustness of the system when facing complex unbalanced data sets, and providing a more reliable technical solution for abnormal data detection and safe operation management of power systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of the steps of the abnormal power consumption detection method based on semi-fixed integrated data provided by an embodiment of the present invention;

[0046] Figure 2 A flowchart for constructing an abnormal power consumption data detection model provided by an embodiment of the present invention;

[0047] Figure 3 A structural block diagram of an abnormal power consumption detection device based on semi-fixed integrated data provided by an embodiment of the present invention;

[0048] Figure 4 A structural diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the feature. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0051] In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. The terms "vertical", "horizontal", "left", "right", "upper", "lower" and similar expressions used herein are only for illustrative purposes, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0052] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood by specific circumstances.

[0053] An embodiment of the present invention provides a method for detecting abnormal power consumption based on semi-fixed integrated data. For details, see Figure 1 and Figure 2 , Figure 1 FIG. 1 is a flowchart showing a method for detecting abnormal power consumption based on semi-fixed integrated data in one embodiment of the present invention. Figure 2 A flowchart of constructing an abnormal power consumption data detection model provided by an embodiment of the present invention is shown, including steps S11 to S16:

[0054] Step S11, obtaining original power consumption data through the power system, and dividing the original power consumption data into a training set, a validation set and a test set according to a time series.

[0055] After obtaining the original electricity consumption data through the power system, the original electricity consumption data is preprocessed, and the preprocessing operations include but are not limited to data cleaning, missing value processing, data type conversion, and data labeling based on domain knowledge.

[0056] In the preprocessing operation, data cleaning is used to remove noise data and duplicate records in the original electricity consumption data, missing value processing is used to interpolate or fill in the missing electricity consumption data based on statistics, and data annotation based on domain knowledge is specifically as follows: the original electricity consumption data is screened according to the data structure related to the power system (such as power grid structure, power load pattern, etc.) according to the defined rules, and the data points that meet the abnormal electricity consumption standards are marked as abnormal samples, and the remaining data points are marked as normal samples.

[0057] Among them, the processing of missing values ​​can be based on statistics, and the mean, median or mode can be used. For example, the mean filling method can be used to fill the missing values ​​in the original electricity consumption data. To fill:

[0058]

[0059] in, is the observed value, is the number of non-missing values.

[0060] The original electricity consumption data after preprocessing is divided into a training set, a validation set, and a test set. Since electricity consumption data usually has time series characteristics, that is, the data changes over time and presents certain trends and regularities, the electricity consumption data after segmentation needs to maintain continuity in time. The specific ratio of the training set, validation set, and test set data segmentation can be adjusted according to the size and characteristics of the actual data set. Specifically, the division ratio adopted in this embodiment is 60-80% / 10-20% / 10-20% (training set / validation set / test set).

[0061] Step S12: classify the abnormal data values ​​according to the variation ranges of the abnormal data values ​​in the training set and the validation set to obtain different types of abnormal data sets.

[0062] Among them, the types of abnormal data include but are not limited to: electricity theft (abnormal decrease in electricity consumption data or sudden drop to an extremely low level), equipment failure (sudden increase or decrease in electricity consumption, abnormal periodic fluctuations), high energy consumption (electricity consumption is significantly higher than normal levels) and equipment overload (electricity load exceeds capacity, load continues to decline) and other abnormal types.

[0063] Step S13, extracting features from the data in each of the abnormal data sets to obtain corresponding abnormal features, simulating abnormal power usage behaviors based on the abnormal features, performing power analysis on each of the abnormal power usage behaviors, and expanding each of the abnormal data sets based on the analysis results.

[0064] Since the number of normal electricity consumption data and abnormal electricity consumption data in the original electricity consumption data is extremely unbalanced, it is necessary to use a sample expansion strategy to expand the sample capacity of different types of abnormal data for the abnormal electricity consumption data samples in the abnormal data set after classification of the training set and the validation set, so as to provide a basis for obtaining samples with a balanced quantity in the subsequent process.

[0065] The process of expanding the abnormal data set is: extracting the power consumption characteristics of the data in each abnormal data set, the power consumption characteristics including data characteristics such as power consumption, power consumption time, power consumption frequency and power factor.

[0066] Abnormal power consumption behavior is simulated according to the power consumption characteristics of each type of abnormal data, and a number of abnormal power consumption data are generated according to the simulated abnormal power consumption behavior, and each abnormal power consumption data is collected into a corresponding abnormal data set to expand the capacity of each abnormal data set.

[0067] Specifically, when this embodiment uses the above-mentioned expansion method to expand the capacity of the abnormal data set, the processing methods for different types of abnormal data are slightly different. Among them, for abnormal power consumption data of the power theft type, based on the power consumption characteristics and patterns of the abnormal power consumption data, taking into account factors such as the time distribution of the power theft behavior and the range of power consumption changes, simulated data that conforms to the actual situation is generated. For abnormal power consumption data of the equipment failure type, the type and characteristics of the equipment failure corresponding to the abnormal data are analyzed according to the power consumption characteristics, and factors such as the time of the failure, duration, and power consumption change trend are considered to generate simulated data that conforms to the characteristics of the equipment failure. For abnormal power consumption data of the high energy consumption type, the power consumption mode and power consumption characteristics of the high energy consumption equipment of the corresponding data are analyzed, such as energy consumption peak value, fluctuation range, etc., and based on the analysis results, abnormal power consumption data of the high energy consumption type is simulated and generated.

[0068] Step S14, using a semi-fixed sample generation method to reorganize each of the expanded abnormal data sets with a preset number of random normal power consumption data sets to obtain a plurality of sample subsets.

[0069] After the expansion, a semi-fixed sample generation method is used to construct the sample subset. The specific process is as follows:

[0070] The abnormal samples after expansion in step S13 are taken as the unchanged part in the sample subset, and then normal power consumption data of the same amount as the number of data in the abnormal data set are randomly extracted from the normal power consumption data in the training set and the validation set as a random normal power consumption data set. The above behavior is repeated to obtain multiple groups of random normal power consumption data sets. In order to ensure the randomness of each group of random normal power consumption data sets and reduce data errors, in the process of extracting normal power consumption data, this embodiment adopts a sampling method with replacement.

[0071] Each abnormal data set is combined with multiple random normal power consumption data sets to obtain several sample subsets for training the base learner in the subsequent process. Each abnormal data set corresponds to multiple sample subsets.

[0072] Step S15, using a machine learning algorithm to train each of the sample subsets to obtain a number of base learners corresponding to each of the sample subsets, screening each of the base learners according to preset performance indicators, and integrating the screened base learners to complete the construction of the abnormal power consumption data detection model.

[0073] A machine learning algorithm is used to train each sample subset, and several base learners are generated for each sample subset. During the training process, the base learner continuously adjusts the learner parameters according to the data in the sample subset.

[0074] During the training process, the base learners with performance lower than the preset performance index are screened out according to the abnormal data detection accuracy of each base learner. The value is used as the standard for judging the accuracy of abnormal data detection, specifically: according to the area under the receiver operating characteristic curve (ROC) curve (Area Under Curve, ) to select the base learner :

[0075]

[0076] in, is the base learner to be screened, is the number of base learners, is the AUC threshold.

[0077] According to the differences and detection difficulties between different abnormal data, different AUC thresholds are set for the sample subsets corresponding to each abnormal data type. For abnormal data types with relatively low detection difficulty, the AUC threshold is increased, and for types with relatively high detection difficulty, the AUC threshold is appropriately lowered.

[0078] After completing the setting of the AUC value, in each round of model screening, if the AUC value of a base learner is higher than the corresponding set AUC threshold, the base learner will be selected for subsequent ensemble learning. For example, the AUC threshold corresponding to a certain type of abnormal data is 0.6, where For the current setting Lower limit, if the base learner of Higher than ( 60), then the base learner is selected for subsequent ensemble learning.

[0079] Repeat the process of extracting sample subsets and training base learners. If the number of models after screening does not meet the preset requirement, return to step S14 and continue to construct sample subsets for training base learners until the number of selected base learners reaches the preset target. (For example 00) or when the predetermined number of iterations is reached, the iteration is stopped and the existing base learners are used for integration to obtain the abnormal power consumption data detection model.

[0080] Methods for integrating base learners include but are not limited to Boosting, Stacking, and Blending.

[0081] Specifically, this embodiment takes the Boosting algorithm as an example to perform integration of base learners. The Boosting algorithm can flexibly select different types of weak learners for integration, which enables Boosting to make full use of the advantages of different learners, thereby building a more complex and complete abnormal electricity consumption data detection model.

[0082] In addition, the Stacking algorithm can also be used for integration. The Stacking algorithm can combine the prediction results of multiple basic models to train a meta-model for the final abnormal data detection. The specific steps are as follows:

[0083] (1) Use the selected base learners as the first-level model. These base learners are trained on their respective sample subsets, each of which contains the expanded abnormal samples and an equal number of normal samples.

[0084] (2) For each first-level model, use the prediction results generated on the validation set as "meta-features". Meta-features are usually prediction probabilities or leaf node IDs of decision tree models, which become input features of the second-level model.

[0085] (3) Build a second-level model, also called a meta-model, which can be a linear regression, logistic regression, or any other appropriate machine learning model. The second-level model is trained using the meta-features generated by the first-level model as input and the true labels of the validation set as output.

[0086] (4) After the second-level model training is completed, for new unknown data, all the models of the first level are first used to make predictions to generate meta-features. Then, these meta-features are input into the second-level model to obtain the final integrated abnormal power consumption data detection model.

[0087] Step S16, using the abnormal power consumption data detection model to detect abnormal power consumption data in the test set data, optimizing the abnormal power consumption data detection model according to the detection result, and performing abnormal power consumption detection with the optimized abnormal power consumption data detection model

[0088] In order to further improve the solution and make it better understood, the following examples are provided to illustrate the model detection effect:

[0089] Embodiment 1

[0090] Energy Company A is a large power supplier that provides power services to urban areas. Company A collected electricity consumption data for a month, with a total of 100,000 records, of which 99% were normal and 1% were abnormal. The preprocessing stage involved data cleaning, removing duplicate records and outliers, and filling missing electricity consumption data with the median.

[0091] The number of samples of abnormal electricity usage was increased from 1,000 to 10,000 to balance the dataset.

[0092] Subset creation and model training: 10,000 samples of normal electricity usage were randomly selected and paired with 10,000 abnormal samples to form a training subset. Company A trained 1,000 such subset models using the logistic regression algorithm.

[0093] Model screening and integration: Each model was tested on an independent validation set, with an AUC threshold of 0.9. Only when the AUC of the model exceeded 0.8 was it selected for ensemble learning. After three iterations of sample subset combination, the number of models reached the preset requirement of 300. In the end, Company A selected 300 models for Boosting ensemble learning and conducted subsequent abnormal power consumption data detection.

[0094] The AUC of the integrated model on the test set reached 0.80, indicating that the model has high accuracy and robustness, can effectively detect abnormal power consumption events, reduce false alarm and missed alarm rates, and improve the operating efficiency and safety of the power grid.

[0095] Table 1 Comparison of model performance of three pre-training set compositions

[0096]

[0097] In Table 1, without sample expansion, the F1 score of the model is 0.60 and the AUC is 0.65. This shows that the model has a high false alarm rate and missed alarm rate when detecting abnormal electricity consumption data. By expanding the abnormal samples and randomly extracting the same number of samples from normal samples for pairing, the F1 score of the model is improved to 0.7, and the AUC is also improved to 0.7. This shows the positive effect of sample expansion on improving model performance. Using the semi-fixed ensemble learning method proposed in the present invention, the F1 score of the model is further improved to 0.8, and the AUC also reaches 0.9. This proves that this method can more effectively detect abnormal electricity consumption data, reduce the false alarm rate and missed alarm rate, and improve the efficiency and safety of power grid operation.

[0098] Embodiment 2

[0099] Industrial Group B has multiple automated production lines. During the production process, it is necessary to monitor the energy consumption of various equipment in real time to ensure production efficiency and quality control. The problem encountered by Group B is that the cases of production anomalies (such as equipment failure) are far less than normal production data, causing the anomaly detection system to often miss important alarms. Energy consumption data of the production lines were collected over two months, totaling 500,000 records, of which abnormal data accounted for only 0.2%. Preprocessing includes removing invalid data points and filling missing energy consumption data with the majority. Increase the number of abnormal energy consumption samples from 1,000 to 10,000 to balance the data set.

[0100] 10,000 pieces of normal energy consumption data are randomly selected and paired with abnormal samples to form 10,000 training subsets. Group B uses the random forest algorithm to train these subsets separately. Each model is evaluated on the validation set, setting The threshold is 0.75. Finally, 100 models were selected from the 10,000 trained models. Perform Stacking integration.

[0101] The F1 score of the integrated model can reach 0.8. It can reach 0.85. The integrated model detects abnormal energy consumption events on the production line in a timely and accurate manner, reduces downtime caused by equipment failure, and improves production efficiency and product quality.

[0102] The abnormal power consumption detection method based on semi-fixed integrated data of the present invention solves the problem of abnormal detection under specific environments. The number of abnormal samples is effectively increased through data expansion technology. The detection accuracy and generalization ability of the model are significantly improved by combining semi-fixed sample combinations with diversified machine learning integrated algorithms. The integrated learning strategy further optimizes the model performance, ensuring the high accuracy and robustness of the system when facing complex unbalanced data sets. It effectively solves the limitations of traditional detection methods in abnormal pattern recognition and provides a more reliable technical solution for the safe operation and intelligent management of power systems.

[0103] The embodiment of the present invention further provides an abnormal power consumption detection device based on semi-fixed integrated data, which is used to execute the abnormal power consumption detection method based on semi-fixed integrated data as described above. Figure 3 This is a structural block diagram of an abnormal power consumption detection device based on semi-fixed integrated data according to an embodiment of the present invention, and the device includes:

[0104] The data partitioning module 21 is used to obtain original power consumption data through the power system, and to partition the original power consumption data into a training set, a validation set and a test set according to a time series.

[0105] The data classification module 22 is used to classify the abnormal data values ​​according to the variation range of the abnormal data values ​​in the training set and the validation set to obtain different types of abnormal data sets.

[0106] The data expansion module 23 is used to extract features from the data in each of the abnormal data sets to obtain corresponding abnormal features, simulate abnormal power consumption behaviors according to the abnormal features, perform power analysis on each of the abnormal power consumption behaviors, and expand each of the abnormal data sets based on the analysis results.

[0107] The sample generation module 24 is used to use a semi-fixed sample generation method to reorganize each of the abnormal data sets after expansion with a preset number of random normal power consumption data sets to obtain a plurality of sample subsets.

[0108] The model building module 25 is used to use a machine learning algorithm to train each of the sample subsets to obtain a number of base learners corresponding to each of the sample subsets, screen each of the base learners according to preset performance indicators, and integrate the screened base learners to complete the construction of the abnormal power consumption data detection model.

[0109] The data detection module 26 is used to detect abnormal power consumption data in the test set data using the abnormal power consumption data detection model, optimize the abnormal power consumption data detection model according to the detection result, and perform abnormal power consumption detection with the optimized abnormal power consumption data detection model.

[0110] The technical features and technical effects of the device proposed in the embodiment of the present invention are the same as those of the method proposed in the embodiment of the present invention, and are not described in detail here. Each module in the above device can be implemented in whole or in part by software, hardware, and a combination thereof. Each of the above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0111] An embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the abnormal power consumption detection method based on semi-fixed integrated data as described above.

[0112] The embodiment of the present invention further provides a computer device, Figure 4 A structural block diagram of a preferred embodiment of a computer device provided by the present invention, wherein the computer device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the abnormal power consumption detection method based on semi-fixed integrated data as described above when executing the computer program.

[0113] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0114] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor. The processor is the control center of the computer device, and various parts of the computer device are connected using various interfaces and lines.

[0115] The memory mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, and a flash card (Flash Card), etc., or the memory can also be other volatile solid-state storage devices.

[0116] It should be noted that the above-mentioned computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 4 The structural block diagram is merely an example of a computer device and does not constitute a limitation of the computer device, and may include more or less components than shown in the figure, or combine certain components, or different components.

[0117] In summary, the abnormal power consumption detection method, device, equipment and medium based on semi-fixed integrated data provided by the embodiments of the present invention have the following beneficial effects compared with the prior art:

[0118] (1) The number of abnormal electricity consumption data samples is effectively increased through data expansion technology. The detection accuracy and generalization ability of the model are improved by combining semi-fixed sample combinations with diversified machine learning integration algorithms.

[0119] (2) The integrated learning of multiple base learners further optimizes the model performance, ensuring the high accuracy and robustness of the system when facing complex unbalanced data sets, and providing a more reliable technical solution for abnormal data detection and safe operation management of power systems.

[0120] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting abnormal power consumption based on semi-fixed integrated data, characterized in that: include: Acquire raw power consumption data through the power system, and divide the raw power consumption data into a training set, a validation set, and a test set according to a time series; Classifying the abnormal data values ​​according to the variation ranges of the abnormal data values ​​in the training set and the validation set to obtain different types of abnormal data sets; Feature extraction is performed on the data in each abnormal data set to obtain corresponding abnormal features, and abnormal power consumption behavior is simulated according to the abnormal features, power analysis is performed on each abnormal power consumption behavior, and each abnormal data set is expanded based on the analysis results; the expansion process specifically includes: extracting power consumption features of the data in each abnormal data set, the power consumption features include power consumption, power consumption time, power consumption frequency and power factor; simulating abnormal power consumption behavior according to the power consumption features of each type of abnormal data, and generating a number of abnormal power consumption data according to the simulated abnormal power consumption behavior, and collecting each abnormal power consumption data into the corresponding abnormal data set to expand the capacity of each abnormal data set; the analysis result is the power consumption mode and power consumption feature of the abnormal power consumption data; Using a semi-fixed sample generation method, each of the abnormal data sets after expansion is recombined with a preset number of random normal power consumption data sets to obtain a plurality of sample subsets; A machine learning algorithm is used to train each of the sample subsets to obtain a number of base learners corresponding to each of the sample subsets, each of the base learners is screened according to a preset performance indicator, and each of the screened base learners is integrated to complete the construction of the abnormal power consumption data detection model; the construction process specifically includes: using a machine learning algorithm to train each of the sample subsets to generate a number of base learners for each of the sample subsets; based on the abnormal data detection accuracy of each of the base learners, the base learners with performance lower than the preset performance indicator are screened out, and the Boosting algorithm is used to assign weights corresponding to different abnormal data types to each of the base learners and integrate them to obtain the abnormal power consumption data detection model; The abnormal power consumption data detection model is used to detect abnormal power consumption data in the test set data, and the abnormal power consumption data detection model is optimized according to the detection result, and abnormal power consumption detection is performed with the optimized abnormal power consumption data detection model.

2. The abnormal power consumption detection method based on semi-fixed integrated data according to claim 1, characterized in that: The semi-fixed sample generation method is used to reorganize each of the expanded abnormal data sets with a preset number of random normal power consumption data sets to obtain several sample subsets, including: Taking each of the abnormal data sets after the expansion as a fixed set, randomly selecting a preset number of normal power consumption data sets from the normal power consumption data by sampling with replacement as a random normal power consumption data set; Each of the random normal power consumption data sets is combined with the data in the abnormal data set to obtain a plurality of sample subsets.

3. The abnormal power consumption detection method based on semi-fixed integrated data according to claim 1, characterized in that: After obtaining the original power consumption data through the power system, the method further includes: Performing a preprocessing operation on the raw power consumption data, wherein the preprocessing operation includes data cleaning, missing value processing and data labeling; The missing value processing is to use the mean filling method to fill the missing values ​​in the original power consumption data To fill: in, is the observed value, is the number of non-missing values.

4. The abnormal power consumption detection method based on semi-fixed integrated data according to claim 1, characterized in that: The classifying the abnormal data values ​​according to the variation ranges of the abnormal data values ​​in the training set and the validation set comprises: Data analysis is performed on each of the abnormal data values. According to the data characteristics of each of the abnormal data values, abnormal data values ​​of abnormal decrease in power consumption and sudden drop to below a preset level are classified as power theft types, abnormal data values ​​of sudden increase or decrease in power consumption and abnormal periodic fluctuations are classified as equipment failure types, and power consumption significantly higher than the normal level is classified as high energy consumption type.

5. The abnormal power consumption detection method based on semi-fixed integrated data according to claim 1, characterized in that: The screening of each base learner according to a preset performance indicator includes: Set the corresponding AUC threshold for each sample subset corresponding to each abnormal data type, and select the base learner with an AUC value higher than the AUC threshold according to the AUC value under the ROC curve. : in, is the base learner to be screened, is the number of base learners, is the AUC threshold.

6. An abnormal power consumption detection device based on semi-fixed integrated data, characterized in that: include: A data partitioning module is used to obtain original power consumption data through the power system, and divide the original power consumption data into a training set, a verification set and a test set according to a time series; A data classification module, used to classify the abnormal data values ​​according to the variation range of the abnormal data values ​​in the training set and the validation set to obtain different types of abnormal data sets; A data expansion module is used to extract features from the data in each abnormal data set to obtain corresponding abnormal features, simulate abnormal power consumption behaviors according to the abnormal features, perform power analysis on each abnormal power consumption behavior, and expand each abnormal data set based on the analysis results; the expansion process specifically includes: extracting power consumption features of the data in each abnormal data set, the power consumption features including power consumption, power consumption time, power consumption frequency and power factor; simulating abnormal power consumption behaviors according to the power consumption features of each type of abnormal data, and generating a number of abnormal power consumption data according to the simulated abnormal power consumption behaviors, and collecting each abnormal power consumption data into the corresponding abnormal data set to expand each abnormal data set; the analysis results are the power consumption mode and power consumption features of the abnormal power consumption data; A sample generation module, used to reorganize each of the abnormal data sets after expansion with a preset number of random normal power consumption data sets using a semi-fixed sample generation method to obtain a plurality of sample subsets; A model construction module is used to train each of the sample subsets using a machine learning algorithm to obtain a number of base learners corresponding to each of the sample subsets, screen each of the base learners according to a preset performance indicator, and integrate the screened base learners to complete the construction of the abnormal power consumption data detection model; the construction process specifically includes: using a machine learning algorithm to train each of the sample subsets to generate a number of base learners for each of the sample subsets; based on the abnormal data detection accuracy of each of the base learners, the base learners with performance lower than the preset performance indicator are screened out, and the Boosting algorithm is used to assign weights corresponding to different abnormal data types to each of the base learners and integrate them to obtain the abnormal power consumption data detection model; A data detection module is used to detect abnormal power consumption data in the test set data using the abnormal power consumption data detection model, optimize the abnormal power consumption data detection model according to the detection result, and perform abnormal power consumption detection with the optimized abnormal power consumption data detection model.

7. A computer device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the abnormal power consumption detection method based on semi-fixed integrated data as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the device where the computer-readable storage medium is located executes the computer program, the abnormal power consumption detection method based on semi-fixed integrated data as described in any one of claims 1 to 5 is implemented.

Images