Method and system for detecting power abnormal data, equipment and medium

Abstract

The invention relates to the technical field of power data detection, and discloses a method and system for detecting power abnormal data, equipment and a medium. The method comprises the following steps: S1, constructing a power data acquisition model based on short-term power load characteristic parameters, and acquiring power data by using the power data acquisition model; s2, preprocessing the power data to generate a data set; s3, acquiring power abnormal data in the data set based on an isolated forest algorithm; and S4, detecting power abnormal data by using a genetic algorithm. The system comprises an acquisition unit, a preprocessing unit, an abnormal data acquisition unit and a detection unit. According to the method, the isolated forest algorithm and the genetic algorithm are combined, so that the stable operation of the power system is ensured, and the accuracy and reliability of detecting the abnormal power data are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power data detection, and in particular to a method, system, device and medium for detecting abnormal power data. The method uses a combination of an isolation forest algorithm and a genetic algorithm to ensure the accuracy of detecting abnormal power data. Background Art

[0002] In today's digital power system, abnormal data detection is one of the key tasks to ensure system safety and reliability. However, the complexity and diversity of power systems bring many challenges, which also makes abnormal data detection a technology that needs to be continuously improved.

[0003] Traditional methods for detecting abnormal data usually rely on statistical techniques and rules. However, the complexity and diversity of current power system data make it difficult for traditional methods to deal with abnormal data. Power system data usually has multiple dimensions and complex structures, and comes from different types of equipment. This diversity makes it difficult for traditional methods to capture abnormal patterns of all equipment, and may require manual adjustment and optimization of rules, making the design and implementation of traditional anomaly detection algorithms difficult.

[0004] Anomaly detection methods based on machine learning have shown great potential in power systems. However, the data generated by new power systems are usually large in size and high in dimension, covering a large number of parameters and variables. Traditional machine learning algorithms may face the problem of dimensionality disaster and have difficulty processing large-scale high-dimensional data. In addition, there is a serious imbalance between abnormal samples and normal samples in the power system, which makes the anomaly detection method based on machine learning biased, resulting in the model being overly adapted to normal samples and insufficient in identifying abnormal samples.

[0005] With the improvement of computing power, deep learning is also playing a significant role in the field of anomaly detection, but it also has some defects. Deep learning models can automatically learn complex feature representations from data and have better generalization and robustness, but they require a large amount of data to train complex models, and the complexity and black-box nature of the models make it difficult to interpret and understand the models. In addition, abnormal data in power systems are usually rare events, but deep learning models may require more abnormal samples for training to improve the ability to detect abnormal data, which leads to the problem of insufficient labels in large amounts of data.

[0006] Therefore, the present invention provides a method, system, device, and medium for detecting abnormal power data, which uses a method combining an isolation forest algorithm and a genetic algorithm to ensure the accuracy of detecting abnormal power data. Summary of the invention

[0007] The present invention provides a method, system, device and medium for detecting abnormal power data, and uses a method combining an isolation forest algorithm and a genetic algorithm to ensure the accuracy of detecting abnormal power data.

[0008] The present invention is achieved through the following technical solutions.

[0009] In a first aspect, the present invention provides a method for detecting abnormal power data.

[0010] The method comprises the following steps:

[0011] Step S1, constructing a power data acquisition model based on short-term power load characteristic parameters, and using the power data acquisition model to collect power data;

[0012] Step S2, preprocessing the power data to generate a data set;

[0013] Step S3, obtaining power abnormality data in the data set based on the isolation forest algorithm;

[0014] Step S4: using a genetic algorithm to detect the abnormal power data.

[0015] In order to better implement the present invention, further, step S1 includes:

[0016] The power data acquisition model is obtained according to the load point prediction residual quantile, the post-meter system residential user detection value of the power generation device, the user compliance probability prediction parameters and the load characteristic parameters.

[0017] In order to better implement the present invention, further, step S2 includes:

[0018] Firstly, the power data is subjected to power data fusion processing, and then the missing data processing is performed;

[0019] The preprocessed power data is integrated to generate a data set.

[0020] In order to better implement the present invention, further, step S3 includes:

[0021] Dividing all the power data in the data set into multiple isolated trees through an isolation forest;

[0022] The abnormality score of each power data feature is output, and the power abnormality data in the data set is obtained according to the abnormality score.

[0023] In order to better implement the present invention, further, the method of outputting the abnormal score of each power data feature includes:

[0024] Get a power data sample y in the power data set O', and the isolation forest anomaly score of each power data sample y is expressed as: Among them, d(η) is a quantized value; η is Euler's constant, k(y) is the path length of y, and F[k(y)] is the average path length in the isolated tree.

[0025] In order to better implement the present invention, further, step S4 includes:

[0026] A transmission density index of the power abnormality data is obtained according to a genetic algorithm, the transmission density index is set to a fixed value, a correlation detection threshold of the power abnormality data is set according to the transmission density index, and the power abnormality data is detected according to the correlation detection threshold.

[0027] In order to better implement the present invention, further, the expression of the transmission density index of the power abnormality data is:

[0028]

[0029] in, represents the initial execution frequency of the detection instruction, w represents the transmission definition coefficient of power anomaly data in the genetic network, and u w represents the data information detection scalar under w conditions, u max represents the maximum value of the data information detection scalar, and η represents the inductance coefficient of the network host for transmitting power abnormal data;

[0030] The expression of the correlation detection threshold of the power abnormality data is:

[0031]

[0032] In the formula, R represents the established detection vector; represents the intervention vector of the genetic application network for transmitting power anomaly data; Represents the detection vector of the genetic application network for the transmission of abnormal power data.

[0033] In a second aspect, the present invention provides a system for detecting abnormal power data, including a collection unit, a preprocessing unit, an abnormal data acquisition unit and a detection unit, wherein:

[0034] A collection unit, used to construct a power data collection model based on short-term power load characteristic parameters, and use the power data collection model to collect power data;

[0035] A preprocessing unit, used for preprocessing the power data to generate a data set;

[0036] An abnormal data acquisition unit, used for acquiring abnormal power data in the data set based on an isolation forest algorithm;

[0037] The detection unit is used to detect the abnormal power data using a genetic algorithm.

[0038] In a third aspect, the present invention provides an electronic device, which includes a processor and a memory; the processor includes the system for detecting abnormal power data described in the second aspect.

[0039] In a fourth aspect, the present invention provides a computer-readable storage medium, which includes instructions; when the instructions are executed on the electronic device described in the third aspect, the electronic device executes the method described in the first aspect.

[0040] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0041] (1) The present invention provides a method for detecting abnormal power data. Through the dual combination of isolation forest algorithm and genetic algorithm, it can not only quickly and accurately identify abnormal data in the power system, but also adapt to various complex and changeable power environments, ensure the stable operation of the power system, and significantly improve the accuracy and reliability of detecting abnormal power data.

[0042] (2) The present invention provides a method for detecting abnormal power data, which can ensure the quality and integrity of power data and provide a solid foundation for subsequent data analysis and decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention is further described in conjunction with the following drawings and embodiments, and all the concepts and innovations of the present invention should be regarded as the disclosed contents and the protection scope of the present invention.

[0044] Figure 1 A flowchart of a method for detecting abnormal power data and a method in a system, device, and medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be understood that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and therefore should not be regarded as limiting the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in this field without making creative work are within the scope of protection of the present invention.

[0046] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "disposed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0047] Embodiment 1:

[0048] A method for detecting abnormal power data in this embodiment is as follows: Figure 1 As shown, the bus transmission control protocol is used to collect short-term load power data parameters, and the time window interval for short-term load power data collection is obtained. The power data within the window time interval is arranged and reorganized to obtain a short-term load power data parameter column. The short-term load power data parameter column is expressed as in, Represents dynamic load characteristic parameters.

[0049] The power data acquisition model is obtained according to the load point prediction residual quantile, the post-meter system residential user detection value of the power generation device, the user compliance probability prediction parameter and the load characteristic parameter, and is expressed as:

[0050]

[0051] Where P is the load point prediction residual quantile, f sw is the detection value of residential users in the power generation device’s post-meter system, μ0 is the user’s compliance probability prediction parameter, Represents dynamic load characteristic parameters.

[0052] Use the power data collection model to collect power data.

[0053] Embodiment 2:

[0054] This embodiment is further optimized on the basis of Embodiment 1, and preprocesses the power data. The preprocessing method includes first fusing the collected short-term power data and then processing the missing data.

[0055] Missing data processing refers to identifying and processing missing values ​​in a data set, which may be caused by equipment failure, data transmission errors, or other reasons. These missing values ​​can be filled by interpolation, predictive models, or using the average value of historical data to ensure data integrity.

[0056] Through these two steps, we can ensure the quality and integrity of power data and provide a solid foundation for subsequent data analysis and decision-making.

[0057] The other parts of this embodiment are the same as those of Embodiment 1, and thus will not be described in detail.

[0058] Embodiment 3:

[0059] This embodiment further optimizes the above embodiment 1 or 2, and generates a preprocessed power data set O', which includes m scheduling data samples y and ε features. Next, the data in the data set O' is divided into multiple isolated trees using the isolation forest technology.

[0060] The steps to construct an isolation tree are as follows: first, randomly select a feature Q from O'; second, select an outlier P from feature Q; then, use feature Q to perform binary tree partitioning on the power system automation dispatching data. If any attribute record value of feature Q is less than Q, it is assigned to the left child node, otherwise it is assigned to the right child node; finally, recursively construct the left child node and the right child node until each power data feature shows isolation or the tree reaches the maximum height, at which time the isolation tree construction is completed until the isolation trees of all data features in the data set O' are established, that is, the isolation forest is established.

[0061] After the isolation tree and isolation forest are established, the anomaly score of each power data feature can be output. Take a data sample y in the power data set O', and the anomaly score of each y in the isolation forest is Among them, d(η) is a quantitative value; η and k(y) are the Euler constant and the path length of y respectively, which refers to the value of the edge from the root node to the isolated point of the power data feature isolation forest, and F[k(y)] is the average path length in the isolation tree. The closer i(y) is to 1, the more significant the abnormality of this data feature Q is.

[0062] The power anomaly data is obtained by calculating the isolation forest anomaly score of the data sample y in the power data set O'.

[0063] The other parts of this embodiment are the same as any of the above embodiments 1-2, so they are not repeated here.

[0064] Embodiment 4:

[0065] This embodiment is further optimized on the basis of any one of the above embodiments 1-3, and two unequal genetic feature vectors of power abnormal data are set as l1 and l2, and the expression of the abnormal cluster point of the genetic network is defined as:

[0066] Where φ represents the parameter difference of power anomaly data in the genetic network It represents the detection correction coefficient of abnormal power data transmitted by the network, and g represents the real-time arrangement coefficient of abnormal power data transmitted by the genetic network.

[0067] In the genetic algorithm, the definition expression of the abnormal cluster point is directly related to the real-time transmission density level of the abnormal data information parameter. The transmission density expression of power abnormal data is: in, represents the initial execution frequency of the detection instruction, w represents the transmission definition coefficient of power anomaly data in the genetic network, and u w represents the data information detection scalar under w conditions, u max It represents the maximum value of the data information detection scalar, and η represents the inductance of the network host to the transmission of abnormal power data.

[0068] Under the action of genetic algorithm, the value of data transmission density index is always in the physical interval of (0, 1]. In a complex network environment, since the distribution law of the data to be transmitted is always unique, the data transmission density index can be regarded as a fixed value for application when calculating the correlation detection threshold. Let x and m represent the correlation index of two randomly selected network transmission power abnormal data, and the absolute values ​​of index x and index m are not completely equal. It represents the calibration parameters of abnormal data detection of network transmission power based on genetic algorithm.

[0069] The correlation detection threshold of the genetic application network for transmission power abnormal data is defined as: In the formula, R represents the established detection vector; represents the intervention vector of the genetic application network for transmitting power anomaly data; Represents the detection vector of the genetic application network for the transmission of abnormal power data. In the genetic algorithm, the correlation detection threshold controls the relative error between the abnormal power data and the standard power data within the established numerical standard, thereby ensuring the accuracy of the network host in detecting the abnormal power data. The accuracy of detecting abnormal power data is improved through the genetic algorithm.

[0070] The other parts of this embodiment are the same as any one of the above embodiments 1-3, so they are not repeated here.

[0071] Embodiment 5:

[0072] This embodiment further optimizes any one of the above embodiments 1-4. The isolation forest algorithm can effectively identify isolated points in the data, which often represent abnormal situations. The genetic algorithm optimizes the parameter settings in the detection process by simulating natural selection and genetic mechanisms, further improving the accuracy and efficiency of detection.

[0073] The other parts of this embodiment are the same as any one of the above embodiments 1-4, so they are not repeated here.

[0074] Embodiment 6:

[0075] This embodiment also provides a system for detecting abnormal power data that matches the method, including a collection unit, a preprocessing unit, an abnormal data acquisition unit, and a detection unit.

[0076] Embodiment 7:

[0077] This embodiment also provides an electronic device, which includes a processor and a memory; the processor includes the system for detecting abnormal power data described above.

[0078] Embodiment 8:

[0079] This embodiment also provides a computer-readable storage medium, which includes instructions; when the instructions are executed on the electronic device described in the above embodiment, the electronic device executes the method described in the above embodiment. Optionally, the computer-readable storage medium may be a memory.

[0080] The processor involved in the embodiments of the present application may be a chip. For example, it may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0081] The memory involved in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0082] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0083] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0085] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0086] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one device or distributed on multiple devices. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0087] In addition, each functional module in each embodiment of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.

[0088] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may contain one or more servers, data centers and other data storage devices that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0089] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for detecting abnormal power data, characterized in that: The following steps are involved: Step S1, constructing a power data acquisition model based on short-term power load characteristic parameters, and using the power data acquisition model to collect power data; Step S2, preprocessing the power data to generate a data set; Step S3, obtaining power abnormality data in the data set based on the isolation forest algorithm; Step S4: using a genetic algorithm to detect the abnormal power data.

2. A method for detecting abnormal power data according to claim 1, characterized in that: The step S1 comprises: obtaining a power data acquisition model according to load point prediction residual quantiles, post-meter system residential user detection values ​​of power generation equipment, user compliance probability prediction parameters and load characteristic parameters.

3. The method for detecting abnormal power data according to claim 1, characterized in that: The step S2 includes: firstly performing power data fusion processing on the power data, and then performing missing data processing; The preprocessed power data is then integrated to generate a data set.

4. The method for detecting abnormal power data according to claim 1, characterized in that: The step S3 comprises: dividing all the power data in the data set into a plurality of isolated trees through an isolation forest; The abnormality score of each power data feature is output, and the power abnormality data in the data set is obtained according to the abnormality score.

5. A method for detecting abnormal power data according to claim 4, characterized in that: The method for outputting the anomaly score of each power data feature includes: obtaining a power data sample y in the power data set O', and the isolation forest anomaly score of each power data sample y is expressed as: Where d(η) is a quantized value; η is Euler's constant, k(y) is the path length of y, and F[k(y)] is the average path length in the isolated tree.

6. The method for detecting abnormal power data according to claim 1, characterized in that: The step S4 includes: obtaining a transmission density index of the power abnormality data according to a genetic algorithm, setting the transmission density index to a fixed value, setting a correlation detection threshold of the power abnormality data according to the transmission density index, and detecting the power abnormality data according to the correlation detection threshold.

7. A method for detecting abnormal power data according to claim 6, characterized in that: The expression of the transmission density index of the power abnormal data is: in, represents the initial execution frequency of the detection instruction, w represents the transmission definition coefficient of power anomaly data in the genetic network, and u w represents the data information detection scalar under w conditions, u max represents the maximum value of the data information detection scalar, and η represents the inductance coefficient of the network host for transmitting power abnormal data; The expression of the correlation detection threshold of the power abnormality data is: In the formula, R represents the established detection vector; represents the intervention vector of the genetic application network for transmitting power anomaly data; Represents the detection vector of the genetic application network for the transmission of abnormal power data.

8. A system for detecting abnormal power data, characterized in that: It includes a collection unit, a preprocessing unit, an abnormal data acquisition unit and a detection unit, wherein: A collection unit, used to construct a power data collection model based on short-term power load characteristic parameters, and use the power data collection model to collect power data; A preprocessing unit, used for preprocessing the power data to generate a data set; An abnormal data acquisition unit, used for acquiring abnormal power data in the data set based on an isolation forest algorithm; The detection unit is used to detect the abnormal power data using a genetic algorithm.

9. An electronic device, characterized in that: It comprises a processor and a memory; the processor comprises the system for detecting abnormal power data as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes instructions; when the instructions are executed on the electronic device as claimed in claim 9, the electronic device is caused to execute the method as claimed in any one of claims 1 to 7.

Images