Distribution network operation abnormal data detection method and system based on AP clustering and DNN algorithm

Abstract

The invention discloses a fuzzy evolutionary algorithm-based distribution network operation abnormal data detection method and system based on AP clustering and a DNN algorithm for dynamic iteration of a power grid planning theme, and relates to the technical field of power system data processing. Comprising the steps of obtaining power distribution network operation data, and performing time sequence segmentation; carrying out clustering analysis on historical data of the operation data of the power distribution network, and extracting characteristic curves of various abnormal behaviors; constructing a deep neural network model to carry out abnormal data correction; calculating a slope through a time sequence to construct a characteristic curve; comparing, analyzing and judging whether the power distribution network operation data is abnormal or not according to the similarity; and a deep neural network model is adopted to predict and correct abnormal data. According to the method, the operation data of the power distribution network are automatically classified and identified through the AP clustering algorithm, the number of clusters does not need to be specified in advance, subjectivity caused by manual intervention is remarkably reduced, abnormal data correction is carried out through the DNN algorithm, and the accuracy of data correction is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power system data processing, and in particular to a method and system for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm. Background Art

[0002] Time series is a common data type in life, and the analysis of time series is a hot research topic in the field of data mining. It consists of a series of observations obtained from the observed object in chronological order. Time series are ubiquitous and widely used in various industries, such as securities trading data, meteorological data, industrial power distribution data, EEG and ECG data, etc. Distribution network operation automation terminal measurement data is also a time series. The purpose of time series anomaly detection is to find abnormal subsequences in a given time series or time series database.

[0003] In time series data mining, there are several research directions:

[0004] 1. Query by content, that is, return the most similar data set based on the distribution network operation process.

[0005] 2. Cluster analysis, clustering similar time series data, including full-sequence cluster analysis and subsequence cluster analysis.

[0006] 3. Classify the time series, that is, given an unlabeled time series, give it a predefined class label.

[0007] 4. Time series forecasting, predicting the next one or more values ​​of a time series by accurately modeling the dependencies between variables.

[0008] In the process of data transmission, the amount of data uploaded every day is very huge, which is inevitable. It may lead to data loss, errors and other anomalies, which need to be paid attention to. In response to this problem, some scholars have conducted research on data anomaly detection, mainly focusing on the range of data adjacent to the anomaly point, the difference between the anomaly point and the adjacent data, and the significance of the difference between the anomaly point and the adjacent data.

[0009] There are many methods for data anomaly detection, including 3σ rule and Z-score based on statistics, KNN and DBSCAN based on distance, LOF and COF based on density, AP and K-means based on clustering, and linear regression and neural network based on models. The most commonly used clustering method is K-means clustering based on distance, that is, the closer the distance between two human targets, the higher the possibility that they belong to the same category.

[0010] With the development of the power industry, electricity has become one of the most important energy sources in my country and is related to the lifeline of the national economy. The power system is becoming more and more mature, but abnormal distribution network operation data is also emerging in an endless stream, bringing safety risks to the power industry and even the power supply system. A large number of distribution network operation automatic terminals are connected to the distribution network operation master station, and the distribution network monitoring capability has been rapidly improved, but the quality of the measurement data of the distribution network operation automation terminal still needs to be improved, especially in the power industry. The generation of abnormal data may lead to energy loss, equipment failure and other problems.

[0011] In the actual calculation process of distribution network operation data, the actual accurate calculation is often difficult to achieve, and the original data will inevitably produce data loss and data confusion. In order to improve the accuracy of the data, a detection and correction method for abnormal distribution network operation data is proposed. Aiming at the problem of abnormal data collection of distribution network operation, the abnormal distribution network operation data is first extracted from the historical data, and the AP clustering algorithm is used to classify the abnormal data, and the abnormal behavior characteristics of various historical abnormal distribution network operation curves are extracted. Then, the actual data is compared with the abnormal behavior data characteristics. And according to the similarity theory, it is judged whether the data is abnormal. Finally, the DNN algorithm is used to replace the abnormal data. In any power system, line loss is inevitable, and this method can also reduce the energy loss caused by line loss. Summary of the invention

[0012] In view of the problems existing in the prior art, the present invention is proposed.

[0013] Therefore, the problem to be solved by the present invention is how to provide a detection method and system that can automatically identify abnormal data of distribution network operation and accurately correct the abnormal data, so as to solve the problems in the prior art that abnormal data detection relies on manual settings, the correction accuracy is not high and overfitting is prone to occur.

[0014] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0015] In the first aspect, an embodiment of the present invention provides a distribution network operation abnormal data detection method based on AP clustering and DNN algorithm, which includes obtaining distribution network operation data and performing time series segmentation; performing cluster analysis on historical data of distribution network operation data, and extracting various abnormal behavior characteristic curves; constructing a deep neural network model to correct abnormal data; constructing a characteristic curve by calculating the slope of the time series; judging whether the distribution network operation data is abnormal based on similarity comparison analysis; and using a deep neural network model to predict and correct abnormal data.

[0016] As a preferred solution of the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm of the present invention, the time series segmentation includes:

[0017] The distribution network operation data is segmented by the peak and valley values ​​of the time series curve;

[0018] Segment the time series curve according to its maximum and minimum values ​​to avoid the loss of feature points caused by equidistant segmentation;

[0019] The time series is segmented by a bottom-up search method so that each segment forms a feature value group;

[0020] The slope of each segment is calculated to construct a characteristic curve, which reflects the changing trend of distribution network operation.

[0021] As a preferred solution of the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm of the present invention, the cluster analysis includes:

[0022] Initialize the attractiveness information of each point in the similarity matrix for historical data;

[0023] Update the attribution information and attribution degree of the similarity matrix and calculate the attractiveness of the candidate cluster center points;

[0024] Add damping coefficient for iterative calculation to prevent numerical oscillation;

[0025] Iteration stops when the set number of iterations is reached or the attraction and belongingness no longer change.

[0026] As a preferred solution of the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm of the present invention, the deep neural network model includes:

[0027] The input layer receives the preprocessing results of the distribution network operation data and performs feature mapping;

[0028] Multiple hidden layers perform nonlinear feature extraction through activation functions;

[0029] Set a Dropout layer between hidden layers to prevent overfitting;

[0030] The output layer predicts and corrects abnormal data based on the learned features.

[0031] As a preferred solution of the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm of the present invention, the preprocessing result is obtained by the following steps:

[0032] Clean the distribution network operation data to remove obvious noise, outliers and missing values;

[0033] Extract characteristic curves or other characteristic information from distribution network operation data;

[0034] Normalize the feature information so that its value range is between 0 and 1;

[0035] Convert the normalized data into a format that can be processed by the neural network.

[0036] As a preferred solution of the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm of the present invention, the construction process of the deep neural network model includes:

[0037] The number of nodes in the input layer matches the dimension of the input data; the hidden layer maps the input to a nonlinear output through the ReLU activation function; a Dropout layer is added between the hidden layers to prevent overfitting;

[0038] The settings of the output layer node Dropout layer include:

[0039] The feedforward neural network is simplified to prevent overfitting; some neurons are randomly discarded during the training process; the new attraction and belonging values ​​are linear combinations of the current iteration value and the previous iteration value:

[0040] r t+1 (i,k)=λr t (i,k)+(1-λ)r t+1 (i,k)

[0041] a t+1 (i,k)=λa t (i,k)+(1-λ)a t+1 (i,k)

[0042] Among them, λ is the damping coefficient, which ranges from 0.5 to 1.

[0043] As a preferred solution of the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm of the present invention, the prediction and correction process of the abnormal data includes:

[0044] Determine whether the data is abnormal by comparing the similarity of the characteristic curves; use the similarity function to compare the difference between the actual data and the historical characteristic curves; a similarity value close to 0 indicates similarity, and close to 1 indicates dissimilarity; determine whether the time series curves are similar within the allowable range by using a preset threshold;

[0045] The deep neural network model training process includes:

[0046] The preprocessing results are divided into training set and validation set; the mean square error MSE is used as the loss function:

[0047]

[0048] Among them, y i is the true value, is the predicted value of DNN. The smaller the MSE, the smaller the deviation between the predicted value and the true value, that is, the smaller the error of the regression prediction model.

[0049] In a second aspect, an embodiment of the present invention provides a distribution network operation abnormal data detection system based on AP clustering and DNN algorithm, which includes an abnormal data detection module, a feature analysis module, and a data correction module;

[0050] The abnormal data detection module is used to obtain the distribution network operation data, segment the distribution network operation data into time series, and extract the abnormal behavior characteristic curve through the AP clustering algorithm;

[0051] The feature analysis module is used to calculate the slope of the time series to construct a feature curve, and to determine whether the distribution network operation data is abnormal based on similarity comparison and analysis;

[0052] The data correction module is used to build a deep neural network model, and predict and correct abnormal data through the deep neural network model. A Dropout layer is set between the hidden layers of the deep neural network model.

[0053] In a third aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm as in the first aspect of the present invention are implemented.

[0054] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, the steps of the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm as in the first aspect of the present invention are implemented.

[0055] The beneficial effects of the present invention are as follows: the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm provided by the present invention automatically classifies and identifies the distribution network operation data through the AP clustering algorithm, without pre-specifying the number of clusters, significantly reducing the subjectivity caused by manual intervention, and improving the automation level and adaptability of abnormal data detection; by adopting the DNN algorithm to correct abnormal data, the algorithm can accurately predict reasonable alternative values ​​by learning historical data patterns, thereby improving the accuracy of data correction; by adding a Dropout layer to the DNN model to randomly "close" some neurons, the overfitting problem of the model during the training process is effectively prevented, the generalization ability of the model is enhanced, and the ability to process unknown data is improved; the present invention can not only be directly applied to the line loss data detection and correction of the power system to help power companies better manage energy losses, but can also be extended to data anomaly detection and correction in other large-scale monitoring systems such as intelligent transportation and industrial Internet of Things, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 The data anomaly detection and correction flow chart of the distribution network operation abnormal data detection method and system based on AP clustering and DNN algorithm, which combines AP clustering algorithm and DNN neural network;

[0058] Figure 2 It is a flow chart of the abnormal data detection method of distribution network operation and the abnormal location solution of the system based on AP clustering and DNN algorithm;

[0059] Figure 3 It is a distribution network operation abnormal data detection method based on AP clustering and DNN algorithm and a DNN model abnormal data correction flow chart of the system;

[0060] Figure 4 A schematic diagram of adding a Dropout layer to the neural network of the distribution network operation abnormal data detection method and system based on AP clustering and DNN algorithm. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0063] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0064] Example 1

[0065] Reference Figure 1 to Figure 4 , which is the first embodiment of the present invention, and provides a distribution network operation abnormal data detection method based on AP clustering and DNN algorithm, comprising:

[0066] S1, obtain the distribution network operation data and perform time series segmentation;

[0067] In the embodiment of the present application, time series segmentation includes:

[0068] The distribution network operation data is segmented by the peak and valley values ​​of the time series curve;

[0069] Segment the time series curve according to its maximum and minimum values ​​to avoid the loss of feature points caused by equidistant segmentation;

[0070] The time series is segmented by a bottom-up search method so that each segment forms a feature value group;

[0071] The slope of each segment is calculated to construct a characteristic curve, which reflects the changing trend of distribution network operation.

[0072] In an optional embodiment, the distribution network operation division sequence adopts a bottom-up search method to segment the entire time series by the maximum and minimum values ​​of the time series curve. Each segment of the divided time series will form a set of feature values, avoiding the problem of feature point loss that may be caused by equidistant segmentation. Assume that the time series is T = {t1, t2, ..., t n}, then according to the maximum and minimum values ​​in the time series, it is divided into several segments T1, T2, ..., T m .

[0073] S2, cluster analysis is performed on the historical data of the distribution network operation data to extract various abnormal behavior characteristic curves;

[0074] In the embodiment of the present application, cluster analysis includes:

[0075] Initialize the attractiveness information of each point in the similarity matrix for historical data;

[0076] Update the attribution information and attribution degree of the similarity matrix and calculate the attractiveness of the candidate cluster center points;

[0077] Add damping coefficient for iterative calculation to prevent numerical oscillation;

[0078] Iteration stops when the set number of iterations is reached or the attraction and belongingness no longer change.

[0079] It should be noted that to process abnormal data, we must first determine the interval where the abnormal data is located. Generally speaking, the location where the abnormal data of the distribution network operation occurs is called an abnormal point, and the probability of an abnormal point occurring is relatively small. The clustering method is used to identify abnormal data. First, cluster analysis is performed on the historical distribution network operation abnormal data to establish an abnormal behavior characteristic curve, and then the curve with a high degree of similarity to the actual operation curve is used as abnormal data for comparative analysis.

[0080] It should be noted that the most important step in the whole process is to extract and compare the characteristic curves to determine the abnormal line loss data, which is the premise of all line loss predictions. Generally speaking, there will be obvious differences between the abnormal power data of the distribution network operation and the normal data, so the characteristic curve can be used to distinguish the two. Figure 2 As shown. The classification of abnormal data first requires the extraction of historical fault data. Since there are many types of faults and the characteristic curves vary greatly, features need to be extracted after classification. The AP (Affinity Propagation) clustering process is as follows:

[0081] Determine the degree of attribution. For a given degree of attribution, update the attraction information of each point in the similarity matrix. At the beginning of the algorithm, all attribution degrees are initialized to zero; r(i,k) is the degree of attraction, the degree of attraction from point i to the candidate cluster center point k, which reflects that after considering other potential cluster centers k′, point k is suitable as the cluster center of point i. S represents the similarity between the point and the point, indicating the degree to which the point is suitable as the cluster center of the point. s(i,k) represents the similarity between point i and point k, indicating the degree of suitability of point i and the cluster center of point k. The larger the value, the more suitable it is. s(k,k) is called the reference degree, which is the point in the similarity matrix with the same horizontal axis coordinate as the vertical axis coordinate, indicating the possibility of the point being the cluster center. Generally speaking, the larger the reference degree, the greater the number of cluster centers in the end. This formula shows the influence of any candidate cluster center on other candidate cluster centers, allowing all candidate cluster centers to participate in the ownership of the disputed points;

[0082] In an optional embodiment, the attribution information is updated, a(i,k) is called the attribution degree, and the information is sent from the candidate cluster center point k to point i. After considering the support of other points i' for point i to choose point k as the cluster center, point k is finally selected. a(i,k) is equal to its own attraction plus the positive attraction obtained from other points. The update formula is:

[0083]

[0084] Among them, r(k,k) is the attraction of point k itself as a cluster center, and r(i′,k) is the positive attraction obtained from other points. The meaning of the degree of belonging a(i,k) is whether point k can become a good cluster center.

[0085] Only positive attraction is added here, because only positive attraction supports the point as a cluster center. If r(k,k) is negative, it means that point k is more suitable to belong to other cluster centers. For the self-attribution a(k,k), the update method is:

[0086] a(k,k)=∑max{0,r(i′,k)}

[0087] In an optional embodiment, iterative update

[0088] Summarize attraction information and belonging information, decide and judge whether to stop iteration, and add a damping coefficient λ to prevent numerical oscillation when updating attraction and belonging. When actually updating, the new attraction and belonging values ​​are a linear combination of the current iteration value and the previous iteration value. The damping coefficient λ is set between 0.5 and 1. The formula is:

[0089] r t+1 (i,k)=λr t (i,k)+(1-λ)r t+1 (i,k)

[0090] a t+1 (i,k)=λa t (i,k)+(1-λ)a t+1 (i,k)

[0091] Among them, λ is the damping coefficient, which ranges from 0.5 to 1.

[0092] After several iterations, the attraction and belongingness no longer change or the set number of iterations is reached, and the iteration ends. After AP clustering, the data is divided into several different clusters, each cluster represents a class of abnormal data with similar characteristics.

[0093] S3, build a deep neural network model to correct abnormal data;

[0094] In an embodiment of the present application, the deep neural network model includes:

[0095] The input layer receives the preprocessing results of the distribution network operation data and performs feature mapping;

[0096] Multiple hidden layers perform nonlinear feature extraction through activation functions;

[0097] Set a Dropout layer between hidden layers to prevent overfitting;

[0098] The output layer predicts and corrects abnormal data based on the learned features.

[0099] In one embodiment, the preprocessing result is obtained by the following steps:

[0100] Clean the distribution network operation data to remove obvious noise, outliers and missing values;

[0101] Extract characteristic curves or other characteristic information from distribution network operation data;

[0102] Normalize the feature information so that its value range is between 0 and 1;

[0103] Convert the normalized data into a format that can be processed by the neural network.

[0104] In an optional embodiment, building a DNN model requires designing a network structure, including an input layer, a hidden layer, and an output layer. The input layer is used to receive preprocessed data, and the number of nodes in the input layer should be equal to the dimension of the input data. The hidden layer is usually composed of multiple layers of neurons. The number of neurons in each layer can be adjusted according to the complexity of the problem and the amount of data. Each hidden layer maps the input to a nonlinear output through an activation function (such as ReLU). In order to avoid overfitting, a Dropout layer can be added between the hidden layers. During the training process, the Dropout layer will randomly discard some neurons to enhance the generalization ability of the model. This method adds the abnormal data of the DNN algorithm to the Dropout layer for regression prediction research, and designs a DNN model network structure that is more complex than the single hidden layer BP neural network model. Figure 4 This is the fully connected neural network structure after adding the Dropout layer.

[0105] The number of nodes in the output layer depends on the format of the data to be predicted. If a single value is predicted, the output layer has only one node; if the output is multi-dimensional, the number of nodes in the output layer matches the dimension of the output. Common activation functions include ReLU (Rectified Linear Unit) and Sigmoid. ReLU is often used in hidden layers because it can alleviate the gradient vanishing problem.

[0106] S4, construct characteristic curve by calculating slope of time series;

[0107] In an optional embodiment, the data change of each time series is represented by a slope, and the slope can be used to reflect the change trend of the distribution network operation. The formula of the slope is:

[0108]

[0109] The characteristic curve is constructed by the slope of the distribution network operation data changing over time. Each segment in the time series corresponds to a slope value, and the entire characteristic curve is a combination of these slope values.

[0110] S5, judging whether the distribution network operation data is abnormal based on similarity comparison and analysis;

[0111] In an optional embodiment, after constructing the characteristic curve, a similarity function is used to compare the difference between the actual data and the historical characteristic curve. By calculating the similarity parameter, it is determined whether the current data is similar to the historical abnormal behavior. The formula is:

[0112]

[0113] In an optional embodiment, the size of the similarity can be used to determine whether the data is abnormal. The similarity function represents the similarity of the curve trends after the time series is segmented. The closer to 0, the more similar, and the closer to 1, the less similar. A suitable threshold ε can be given to indicate that the two time series curves are similar within the allowable range. In addition, pay attention to making the number of segments of the two curves the same. When comparing two curves, it is necessary to increase the number of segments of the curve with fewer segments so that the number of segments of the two curves is the same. By comparing the similarity, the location of the abnormal data of the distribution network operation can be accurately located.

[0114] S6, uses a deep neural network model to predict and correct abnormal data;

[0115] In an optional embodiment, after AP clustering, the distribution network operation data is classified according to features and abnormal data is identified. The next step is to use DNN (Deep Neural Network) to predict and correct the abnormal data of the distribution network operation. The main goal of DNN is to fill or correct abnormal data by learning from historical data. Considering that the amount of data for each type of abnormal data is small, overfitting problems are prone to occur when training in a complex feedforward neural network. This method simplifies the feedforward neural network by adding a Dropout layer, so that some neurons stop working under a certain probability, thereby enhancing the generalization ability of the model and optimizing the performance of the neural network. The flow chart is as follows Figure 3 shown.

[0116] In an optional embodiment, before using DNN for data correction, the data must first be preprocessed. This includes cleaning the data, extracting features, and normalizing and converting it into a format that DNN can process. Data cleaning refers to removing obvious noise, outliers, and missing values. For missing values, zero padding can be used. Feature extraction means that time series or other forms of data can extract characteristic curves or other feature information as input to the DNN model. Normalization is to make the numerical range of the data consistent. The data is usually normalized or standardized so that the values ​​of all data points are in the same range (usually 0 to 1). The normalization formula is:

[0117]

[0118] In an optional embodiment, the model is trained after the model is built. The goal of the training is to learn how to correct abnormal or missing values ​​in the data through the input and output relationship of historical data. The regression problem uses the mean square error (MSE) of the loss function to measure the error between the predicted value and the true value. The data set is usually divided into a training set and a validation set. The training set is used to update the model parameters, and the validation set is used to evaluate the performance of the model on unseen data. The formula for MSE is:

[0119]

[0120] Among them, y i is the true value, is the predicted value of DNN. The smaller the MSE, the smaller the deviation between the predicted value and the true value, that is, the smaller the error of the regression prediction model.

[0121] In an optional embodiment, after the model training is completed, it can be used for data correction. First, detect the abnormal part in the current data, and judge whether the data is abnormal by comparing it with historical data or by the similarity of characteristic curves. Once the abnormal data is detected, these abnormal data are used as inputs of the DNN model. The DNN model will predict reasonable replacement values ​​based on the historical learning patterns, and these predicted values ​​will be used to replace the abnormal data or fill in the missing data.

[0122] Furthermore, the present embodiment also provides a distribution network operation abnormal data detection system based on AP clustering and DNN algorithm, including an abnormal data detection module, a feature analysis module, and a data correction module;

[0123] The abnormal data detection module is used to obtain the distribution network operation data, segment the distribution network operation data into time series, and extract the abnormal behavior characteristic curve through the AP clustering algorithm;

[0124] The feature analysis module is used to calculate the slope of the time series to construct a feature curve, and to determine whether the distribution network operation data is abnormal based on similarity comparison and analysis;

[0125] The data correction module is used to build a deep neural network model, and predict and correct abnormal data through the deep neural network model. A Dropout layer is set between the hidden layers of the deep neural network model.

[0126] This embodiment also provides a computer device, which is suitable for the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm proposed in the above embodiment.

[0127] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0128] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm as proposed in the above embodiment is implemented.

[0129] In summary, (1) Automated abnormal data detection

[0130] Traditional anomaly detection and data correction methods usually rely on manually set rules and thresholds, which are at risk of human intervention and lack good adaptability. This method uses the AP clustering algorithm, which can automatically identify abnormal patterns in data without pre-specifying the number of clusters, reducing the subjectivity of manual settings.

[0131] (2) High correction accuracy

[0132] DNN algorithms show higher accuracy and generalization ability when correcting abnormal data, especially when data is missing or abnormal. DNN can accurately predict reasonable alternative values ​​by learning the patterns of historical data, thereby effectively correcting the data.

[0133] (3) Prevent overfitting and improve model generalization ability

[0134] The Dropout layer of the DNN model is used to prevent overfitting of the neural network during training and enhance the generalization ability of the model. Dropout can randomly "turn off" some neurons, thereby preventing the model from overfitting on specific data and improving the performance of the model when processing unknown data.

[0135] (4) Wide application of technology

[0136] It can be directly applied to the detection and correction of line loss data in power systems. Line loss in power systems is a key efficiency indicator. By accurately detecting and correcting outliers in line loss data, power companies can better manage their energy losses and thus improve overall operational efficiency. Smart grids rely on a large number of sensors and data transmission to monitor and control the power distribution process. The data generated by sensors is susceptible to faults, communication errors, and environmental interference, resulting in data anomalies. This method can be used in smart grids to detect and correct data anomalies in real time, ensuring the reliability and stability of system operation.

[0137] Other large-scale monitoring systems (such as smart transportation, industrial Internet of Things, etc.) also face the problem of data anomaly detection and correction. This can be extended to these fields to automatically detect and correct anomalies in data.

[0138] Example 2

[0139] This is the second embodiment of the present invention. This embodiment provides a distribution network operation abnormal data detection method based on AP clustering and DNN algorithm. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0140] In order to verify the effectiveness of the distribution network operation abnormal data detection method proposed in this invention, the operation data of a certain urban distribution network within one month was selected for experiment. The experimental data includes key parameters such as voltage, current, active power and reactive power, and the sampling frequency is 5 minutes / time. The experiment was carried out on a workstation configured with an Intel i7-12700 processor and 32GB memory, and the algorithm was implemented using Python 3.8.

[0141] First, the raw data was preprocessed. Obvious outliers were removed through data cleaning, such as data points with a voltage of 0 or exceeding 1.2 times the rated value. Missing values ​​were filled with the average value of the adjacent points. To improve data quality, the sliding window method (window size is 12 points, i.e. 1 hour) was used to smooth the data.

[0142] In terms of time series segmentation, an improved bottom-up search algorithm is used. The slope change threshold is set to 0.15. When the slope change of two adjacent segments exceeds the threshold, it is determined as a segmentation point. Compared with the traditional equidistant segmentation method, this method can better retain data feature points. Through experiments, it is found that for voltage data, a segment is formed every 4 hours on average; for power data, due to more frequent changes, a segment is formed every 2 hours on average.

[0143] In AP cluster analysis, negative Euclidean distance is used as the similarity metric when initializing the similarity matrix. The damping coefficient λ is set to 0.7, and the maximum number of iterations is 200. The preference value p is selected as 1.5 times the median of all similarities, so that an appropriate number of clusters can be obtained.

[0144] The DNN model uses a 5-layer structure: input layer (128 nodes), three hidden layers (256, 128, and 64 nodes respectively), and output layer (1 node). The hidden layer uses the ReLU activation function, and the output layer uses the linear activation function. A Dropout layer is added between the hidden layers, and the dropout rate is set to 0.3. The Adam optimizer is used, the learning rate is set to 0.001, and the batch size is 64. The data is divided into a training set and a validation set in a ratio of 7:3.

[0145] In order to compare the effects of the method of the present invention with the prior art, the following four methods are used for testing:

[0146] Method A: The method of the present invention (AP clustering + DNN)

[0147] Method B: Traditional 3σ rule

[0148] Method C: K-means clustering + BP neural network

[0149] Method D: Isolation Forest Algorithm

[0150]

[0151] The following conclusions can be drawn from the analysis of the experimental results:

[0152] In terms of detection accuracy: the anomaly detection rate of the method of the present invention (method A) reached 95.8%, which is significantly higher than other methods. In particular, it increased by 13.3 percentage points compared with the traditional 3σ rule (method B). This is mainly due to the fact that the AP clustering algorithm can adaptively determine the number of clusters and better identify different types of abnormal patterns. At the same time, the false alarm rate is only 2.3%, which is much lower than other methods, indicating that the method of the present invention has higher reliability.

[0153] In terms of detection efficiency: the average detection time of the method of the present invention is 45ms, which is 40-75ms faster than other methods. This is because the improved time series segmentation algorithm is adopted to reduce unnecessary calculations. In addition, by introducing the Dropout layer in the DNN model, the network structure is simplified and the operation speed is improved.

[0154] In terms of generalization ability: By analyzing the missed detection rate data, it can be seen that the missed detection rate of the method of the present invention is only 4.2%, while the missed detection rates of other methods are all above 10%. This shows that the method has stronger generalization ability and can identify different types of abnormal data. This is mainly due to the deep structure design of the DNN model and the application of the Dropout mechanism, which effectively prevents the overfitting problem.

[0155] In terms of comprehensive performance: the accuracy of the method of the present invention reaches 97.2%, which is 7 percentage points higher than the second best method (method C). This result fully demonstrates the superiority of combining AP clustering with DNN algorithm in the present invention. AP clustering algorithm can better extract data features and provide high-quality input for subsequent DNN models, thereby improving the overall detection performance.

[0156] These experimental data fully prove that the method of the present invention has significant advantages in detecting abnormal data in distribution networks, which not only improves the accuracy and reliability of detection, but also has higher computational efficiency and stronger generalization ability. In particular, when processing large-scale real-time data, the method shows obvious advantages.

[0157] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A distribution network operation abnormal data detection method based on AP clustering and DNN algorithm, characterized by: Including, obtaining distribution network operation data and performing time series segmentation; Performing cluster analysis on the historical data of the distribution network operation data to extract various abnormal behavior characteristic curves; Build a deep neural network model to correct abnormal data; Constructing a characteristic curve by calculating the slope of the time series; Determine whether the distribution network operation data is abnormal based on similarity comparison analysis; The deep neural network model is used to predict and correct abnormal data.

2. The method for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm according to claim 1, characterized in that: The time series segmentation includes: Segmenting the distribution network operation data according to the peak and valley values ​​of the time series curve; Segmenting the time series curve according to the maximum and minimum values ​​to avoid feature point loss caused by equidistant segmentation; Segmenting the time series by a bottom-up search method so that each segment forms a feature value group; The characteristic curve is constructed by calculating the slope of each segment to reflect the changing trend of the distribution network operation.

3. The method for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm according to claim 2, characterized in that: The cluster analysis includes: Initializing the attractiveness information of each point in the similarity matrix for the historical data; Updating the attribution information and attribution degree of the similarity matrix, and calculating the attractiveness of the candidate cluster center points; Add damping coefficient for iterative calculation to prevent numerical oscillation; The iteration is stopped when the set number of iterations is reached or the attraction and the degree of belonging no longer change.

4. The method for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm according to claim 3, characterized in that: The deep neural network model includes: The input layer receives the preprocessing results of the distribution network operation data and performs feature mapping; Multiple hidden layers perform nonlinear feature extraction through activation functions; A Dropout layer is provided between the hidden layers to prevent overfitting; The output layer predicts and corrects the abnormal data based on the learned features.

5. The method for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm according to claim 4, characterized in that: The preprocessing result is obtained by the following steps: Cleaning the distribution network operation data to remove obvious noise, outliers and missing values; Extracting characteristic curves or other characteristic information from the distribution network operation data; Normalizing the feature information so that its value range is within the range of 0 to 1; The normalized data is converted into a format processable by a neural network.

6. The method for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm according to claim 5, characterized in that: The construction process of the deep neural network model includes: The number of nodes in the input layer matches the dimension of the input data; the hidden layer maps the input to a nonlinear output through a ReLU activation function; and the Dropout layer is added between the hidden layers to prevent overfitting; The setting of the Dropout layer of the output layer node includes: The feedforward neural network is simplified to prevent overfitting; some neurons are randomly discarded during the training process; the new attraction and belonging values ​​are linear combinations of the current iteration value and the previous iteration value: r t+1 (i,k)=λr t (i,k)+(1-λ)r t+1 (i,k) a t+1 (i,k)=λa t (i,k)+(1-λ)a t+1 (i,k) Among them, λ is the damping coefficient, which ranges from 0.5 to 1.

7. The method for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm according to claim 6, characterized in that: The prediction and correction process of the abnormal data includes: Determine whether the data is abnormal by comparing the similarity of the characteristic curves; use the similarity function to compare the difference between the actual data and the historical characteristic curves; a similarity value close to 0 indicates similarity, and close to 1 indicates dissimilarity; determine whether the time series curves are similar within the allowable range by using a preset threshold; The deep neural network model training process includes: The preprocessing results are divided into a training set and a validation set; the mean square error MSE is used as the loss function: Among them, y i is the true value, is the predicted value of DNN. The smaller the MSE, the smaller the deviation between the predicted value and the true value, that is, the smaller the error of the regression prediction model.

8. A distribution network operation abnormal data detection system based on AP clustering and DNN algorithm, based on the distribution network operation abnormal data detection method based on AP clustering and DNN algorithm according to any one of claims 1 to 7, characterized in that: It also includes, an abnormal data detection module, a feature analysis module, and a data correction module; The abnormal data detection module is used to obtain the distribution network operation data, perform time series segmentation on the distribution network operation data, and extract abnormal behavior characteristic curves through AP clustering algorithm; The characteristic analysis module is used to calculate the slope of the time series to construct a characteristic curve, and determine whether the distribution network operation data is abnormal based on similarity comparison analysis; The data correction module is used to construct a deep neural network model, and predict and correct abnormal data through the deep neural network model. A Dropout layer is set between the hidden layers of the deep neural network model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting abnormal data of distribution network operation based on AP clustering and DNN algorithm described in any one of claims 1 to 7 are implemented.

Images