Distributed photovoltaic abnormal data detection method based on periodic optimal path comparison
Through the method of periodic optimal path comparison, combined with dynamic time regularization and spectral clustering, anomaly detection is performed using local outlier factor and isolated forest model, which solves the problem of insufficient abnormal recognition capability in distributed photovoltaic power generation data, and achieves high accuracy rate and high efficiency abnormal detection.
Patent Information
- Application Number
- CN202411085581.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
It is difficult for the prior art to accurately identify and process abnormal situations in distributed photovoltaic power generation data, especially when a high proportion of distributed power is connected to the power system, traditional algorithms cannot effectively process the increase in data volume and timing characteristics, resulting in insufficient recognition capabilities.
The method based on period optimal path comparison is adopted to calculate the similarity measurement distance between photovoltaic output power and radiation data through dynamic time regularization algorithm (DTW), and the feature vector is extracted in combination with the spectral clustering algorithm, and the local outlier factor algorithm (LOF) is used. ) and the isolated forest model perform anomaly detection, and finally perform optimal votes through the decision matrix and the advantage and disadvantage solution distance method.
Effectively capture most abnormal samples, improve the recognition ability of photovoltaic abnormal data, improve the recognition accuracy and efficiency, and is suitable for data processing of high proportion distributed power supplies.
Smart Images

Figure CN119026030A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of photovoltaic technology, and in particular to a distributed photovoltaic abnormal data detection method based on period optimal path comparison. Background Art
[0002] In recent years, the scale and application of photovoltaic and wind power resources have gradually expanded, technology iterations have been continuously updated, solar cells have been industrialized, the costs of each link in the industrial chain have steadily decreased[i], and photovoltaic related technologies have developed rapidly. However, a large number of controllable loads and distributed power sources have inherent volatility. After they are connected to the power grid, the change of the traditional distribution network's single-ended power supply radial topology structure has brought more uncertainty to the operation of the distribution network, and it is also vulnerable to network attacks and affects the safe operation of the distribution network's information-physical system. Therefore, it is of great significance to carry out research on abnormal data detection methods for distributed photovoltaic industrial control networks under new power systems.
[0003] At present, there have been a lot of research on photovoltaic data anomaly detection. Yang Mao, Meng Lingjian and others used probability statistics, parameter distribution and other methods to identify and analyze photovoltaic power anomalies. Shi Min, Yin Rui and others used sliding standard deviation to clean abnormal data based on the distribution characteristics of photovoltaic operation abnormal data. Zuo Songlin, Chen Wei and others modeled photovoltaic power generation based on environmental factors and grid-connected data and used extended Kalman filtering to detect deviation values. Liu Yan, Li Wenwen and others established a Gaussian mixture model and estimated parameters to detect high-proportion abnormal data of photovoltaic power generation output. Su Yonghe, Zuo Ying and others proposed joint learning of multi-station branch power, which extracted similarity and difference features by constructing a multi-scale convolutional neural network for anomaly detection. Ye Lin, Cui Baodan and others proposed a combined recognition method for high-proportion abnormal data based on the typical characteristics of continuous abnormal data and discrete abnormal data of photovoltaic power, and achieved effective recognition. Zhao Bochao, Ma Jiajun and others decomposed photovoltaic power generation data into multiple modal components to obtain its periodic and non-periodic features, and used dynamic thresholds of predicted values and measured values to identify anomalies based on the construction of the VMD-XGBoost-BILSTM combined model. Zhou Jiaqi, Bili and others converted photovoltaic inverter time series data into two-dimensional images, used convolutional long short-term memory networks to reset the network structure, proposed a photovoltaic inverter multivariate time series image anomaly detection model based on GAN, and encoded the multivariate time series into a two-dimensional image by constructing a feature matrix, achieving high-accuracy detection of inverter abnormal data.
[0004] The above literature has made an in-depth discussion on anomaly detection research, but data anomaly detection for high-proportion distributed power sources still needs to be studied: 1) The high-proportion of new energy is booming, and its inherent randomness and volatility make it impossible for algorithms for old power grids to accurately identify and handle all abnormal situations; 2) Most of the existing literature on power data anomaly detection only focuses on distribution networks, and when a high proportion of distributed power sources are connected, the amount of data processed by the power system increases dramatically. Therefore, it is crucial to accurately and efficiently process power system anomaly detection data under distributed power source access; 3) Distributed photovoltaic power generation data is typical time series data, stored in the form of equally spaced time series. Its time series characteristics must be taken into account when detecting anomalies. Summary of the invention
[0005] In order to solve the above technical problems, the present invention provides a distributed photovoltaic abnormal data detection method based on period optimal path comparison.
[0006] To achieve the above object, the present invention is implemented according to the following technical solutions:
[0007] The distributed photovoltaic abnormal data detection method based on period optimal path comparison includes the following steps:
[0008] S1. Divide the collected photovoltaic output power data and radiance data into periodic time series of power data and radiance data, calculate the similarity of each point in the periodic time series of power data and radiance data based on the dynamic time warping algorithm DTW, obtain the similarity metric distance between the two sequences and form a cumulative distance matrix; and use the spectral clustering algorithm SCA to extract feature vectors and combine the elbow rule graph to achieve optimal clustering;
[0009] S2. For clusters with different numbers of groups, the local outlier factor algorithm LOF is used to calculate the anomaly score of each data point in the group, and it is used as the third feature of the data group; the isolation forest iForest model is built by constructing multiple iTree tree models, and the anomaly score value is used as the judgment method for data anomalies. Then, based on the decision matrix and the superior and inferior solution distance method TOPSIS, the detection results of the model are optimally voted to obtain the optimal output.
[0010] Furthermore, in step S1, the specific process of calculating the similarity of each point in the periodic time series of the power data and the radiance data based on the dynamic time warping algorithm DTW, obtaining the similarity metric distance between the two sequences and forming a cumulative distance matrix is as follows:
[0011] S101, define the periodic time series X of power data = [i 1 ,i 2 ,…,i m ] and the periodic time series of radiometric data Y = [j1 ,j 2 ,…,j n ]:
[0012]
[0013] Where X, Y are the periodic time series of power data and radiance data, d(i k ,j k ) is the calculated distance of each point in the periodic time series of power data and radiometry data, and p represents the p-norm;
[0014] S102, using formula (1) to calculate the best matching path between sequences, and minimize the sum of distances of all matching point pairs on the path;
[0015] S103. Construct a cumulative distance matrix D using formula (2) to calculate the optimal distance Dist, which is the similarity measurement distance between the two sequences:
[0016]
[0017] Where D(i,j) is the cumulative distance and d(i,j) is the current element distance.
[0018] Furthermore, in step S1, the specific process of using the spectral clustering algorithm SCA to extract feature vectors and combining the elbow rule graph to achieve optimal clustering is:
[0019] S104, record the collected data as X=[x r x p ], x r is the radiometric data, x p is the photovoltaic output power data; the missing value of X is searched, and the collected data are data points with equal time intervals. The Newton interpolation polynomial is used for interpolation, and the corresponding expression is:
[0020]
[0021] S105, after constructing the Newton interpolation polynomial, interpolate the specified data. After completing the data filling, the photovoltaic output data needs to be normalized, and z-score normalization is adopted, and the formula is as follows:
[0022]
[0023] Where, X norm is the normalized data, X is the original data, and X mean is the mean of the original data, X std is the standard deviation of the original data set;
[0024] S106, X norm Each line is the primary judgment object, querying the empty value interval of photovoltaic output data and intercepting the real value interval, and X norm Each set of data, i.e., each column of real-valued intervals in the data, is regarded as a new real-valued time series sequence;
[0025] S107, define the intercepted real-valued sequence p i ,q i , i = 1, 2, .. k, calculate x for each day r ,x p The correlation coefficient is used as the linear weight;
[0026] S108, using the Euclidean distance to construct the distance matrix D between the sequences and solve the regularized path w, then solve the optimal distance Dist of the two sequences, perform adaptive weighted average calculation, and finally find an optimal regularized path w that minimizes the cumulative distance, that is:
[0027] Dist min =∑d′(i,j) (5)
[0028] S109: After calculating the cumulative distance between each sequence and other sequences, the cumulative distance is converted into a similarity matrix D s , clustering based on similarity matrix;
[0029] S110, for the similarity matrix D s , introduce the inverse distance matrix and perform spectral clustering based on the Gaussian kernel function, where the Gaussian kernel function is constructed as:
[0030]
[0031] S111, construct the Laplace matrix and perform vector decomposition, arrange the eigenvalues in descending order, and extract the first k eigenvectors with the smallest eigenvalues according to the number of samples included. i , i = 1, 2, ... k, and take the real part to form the eigenvector matrix;
[0032] S112. Calculate the error square sum function according to the traditional k-means algorithm, and continuously update the cluster center. Combine the elbow rule method to determine the optimal number of clusters. Determine the data cluster D after K clusters according to the optimal solution and actual needs. i ,i=1,2,3,…k.
[0033] Furthermore, the step S2 specifically includes:
[0034] S21, set the initial number of isolated trees to 100, the subsampling size to 256, build an isolation forest framework and then build an isolated tree model;
[0035] S22, using the local outlier factor algorithm LOF to calculate the LOF anomaly score in each cluster, and marking the index of the point with a higher anomaly score, and using the LOF anomaly score as the third feature of the data set;
[0036] S23, randomly select features from the data set and randomly select cutting points to divide the data points into two left and right subtrees, and continue to recursively build trees until each data point becomes a leaf node, construct multiple trees, and form an isolation forest;
[0037] S24. For a given data point, calculate its path length in the isolation tree and obtain the anomaly score of the data point:
[0038] s(x,n)=2 -E(h(x)) / c(n) (7)
[0039] In the formula, h(x) is the path length of the sample in the tree, and E(h(x)) is the length of a single data point x. i The average of the depths reached in all trees, c(n) is the corrected value:
[0040] c(n)=2H(n-1)-(2(n-1) / n)(8)
[0041] Where H(i) is the harmonic number, estimated by Euler's constant, and n is the number of points in the construction tree;
[0042] S25. Based on the decision matrix and the superior and inferior solution distance method, the detection results of the model are optimally voted to obtain the optimal output:
[0043] Obtain the anomaly score matrix and solve the positive and negative ideal solutions of the final anomaly score according to formula (9);
[0044]
[0045] Then determine the optimal solution and the worst solution according to formula (10);
[0046]
[0047] The final anomaly score value of the anomaly detection model is calculated based on the optimal solution and the worst solution calculated by formula (11).
[0048]
[0049] Furthermore, the local outlier factor algorithm LOF is used to calculate the LOF anomaly score in each cluster. The specific process is as follows:
[0050] Let cluster D i, i=1,2,3,…k, where the calculation point is P, point Q is the k nearest neighbor object of point P, and the maximum distance of the k nearest neighbors is expressed as k_dist. Find its k distance neighborhood, then calculate the reachable distance of point P with respect to point O, and the local reachable density of the data point, and calculate the local anomaly factor based on formula (8):
[0051]
[0052] Where Lrd(P) is the local reachable density of data point P, Lrd(O) is the local reachable density of data point O, and N k_dist (P) is the set of objects in the k-neighborhood of point P.
[0053] Preferably, the optimal number of clusters is 4.
[0054] Compared with the prior art, the present invention measures the similarity of different periodic sequence groups based on the dynamic time warping algorithm, obtains its optimal path and constructs a similarity matrix, and then uses the spectral clustering algorithm to extract feature vectors and combines the elbow rule graph to achieve optimal clustering. Then, the LOF anomaly score is introduced for embedded feature weighting based on the density characteristics of the data in the cluster, and further, the detection results are optimally voted based on the isolation forest framework to obtain the optimal output; the method of the present invention can effectively capture most abnormal samples, and the extraction of abnormal sample accuracy data can effectively improve the recognition ability of photovoltaic abnormal data. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 This is a framework diagram of the distributed photovoltaic abnormal data detection method based on periodic optimal path comparison.
[0056] Figure 2 This is an example diagram of the DTW regularized path.
[0057] Figure 3 The clustering result diagram for comparing the optimal path of the cycle: a is the elbow criterion; b is the clustering cluster data diagram.
[0058] Figure 4 Figure 2 is an example diagram of power distribution for each type of fault.
[0059] Figure 5 An example diagram of the confusion matrix for different cluster detection results.
[0060] Figure 6 This is an example of a confusion matrix for unweighted data detection results.
[0061] Figure 7 This is an example of a confusion matrix for a single original model detection result.
[0062] Figure 8 Detection results of different anomaly detection models. DETAILED DESCRIPTION
[0063] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the invention.
[0064] Example
[0065] In view of the problem that the amount of data from the high proportion of distributed power generation connected to the distribution network has increased dramatically and the traditional anomaly detection and recognition methods cannot accurately and efficiently process the photovoltaic anomaly data with time series characteristics, such as Figure 1 As shown, this embodiment proposes a distributed photovoltaic abnormal data detection method based on period optimal path comparison, including the following steps:
[0066] S1. Divide the collected photovoltaic output power data and radiance data into periodic time series of power data and radiance data, calculate the similarity of each point in the periodic time series of power data and radiance data based on the dynamic time warping algorithm DTW, obtain the similarity metric distance between the two sequences and form a cumulative distance matrix; and use the spectral clustering algorithm SCA to extract feature vectors and combine the elbow rule graph to achieve optimal clustering;
[0067] DTW is a method for comparing the similarity of two time series. It defines the distance measurement function between the two time series and uses the dynamic programming method to calculate the sequence X = [i 1 ,i 2 ,…,i m ] and the sequence Y=[j 1 ,j 2 ,…,j n The best matching path between [ii] , as shown in formula (1), and minimize the sum of the distances of all matching point pairs on the path, and construct a cumulative distance matrix D using formula (2) to calculate the optimal distance Dist, which is the similarity measurement distance between the two sequences.
[0068]
[0069] In the formula, X, Y are time series, d(i k ,j k ) is the calculated distance of each point in the time series, and p represents the p-norm.
[0070]
[0071] Where D(i,j) is the cumulative distance and d(i,j) is the current element distance.
[0072] Generally speaking, the main determinant of photovoltaic power generation output is irradiance, and the output power of a photovoltaic power station varies with the change of irradiance. Referring to relevant literature and actual data verification, it can be seen that photovoltaic output power and irradiance have an approximate linear relationship, so the invention selects photovoltaic output power and irradiance as parameter data.
[0073] The collected data is recorded as X = [x r x p ], x r is the radiometric data, x p is the photovoltaic output power data. The new power system is characterized by a high proportion of power electronic equipment, which is susceptible to electromagnetic interference and may cause sensor data loss. Therefore, the missing value of X is first searched. The collected data is equal time interval data points, and Newton interpolation polynomial is used for interpolation. The corresponding expression is:
[0074]
[0075] After constructing the Newton interpolation polynomial, the specified data can be interpolated. After completing the data filling, the photovoltaic output data needs to be normalized, and z-score normalization is used. The formula is as follows:
[0076]
[0077] Where, X norm is the normalized data, X is the original data, and X mean is the mean of the original data, X std is the standard deviation of the original data set.
[0078] X norm Each line is the primary judgment object, querying the empty value interval of photovoltaic output data and intercepting the real value interval, and X norm Each group of data, i.e., each column of real-valued intervals in the data, is regarded as a new real-valued time series. Although the data point of photovoltaic output in one day is determined to be 96, since the real values of the daily irradiance and photovoltaic power output are not determined at certain times, the obtained real-valued sequences may not be equal. As a relatively stable variable, irradiance has a high correlation with photovoltaic output power, and photovoltaic output power is easily affected by external factors. In summary, the weighted average DTW algorithm is used to solve the optimal path of the periodic sequence. The specific steps are as follows:
[0079] 1) Define the intercepted real-valued sequence p i ,q i , i=1,2,...k, calculate x for each day r ,x p The correlation coefficients were used as linear weights.
[0080] 2) Use Euclidean distance to construct the distance matrix D between sequences and solve the regular path w, then solve the cumulative distance Dist of the two sequences, perform adaptive weighted average calculation, and finally find an optimal regular path w. The regular path is as follows Figure 2 As shown, the cumulative distance is minimized, that is:
[0081] Dist min =∑d′(i,j) (5)
[0082] 3) After calculating the cumulative distance between each sequence and other sequences, the cumulative distance is converted into a similarity matrix D s , clustering based on similarity matrix.
[0083] 4) For the similarity matrix D s , introduce the inverse distance matrix and perform spectral clustering based on Gaussian kernel function [iii] . Among them, the Gaussian kernel function is constructed as:
[0084]
[0085] 5) Construct the Laplace matrix and perform vector decomposition, sort the eigenvalues in descending order, and extract the first k eigenvectors with the smallest eigenvalues according to the number of samples included. i ,i=1,2,...k, and take the real part to form the eigenvector matrix.
[0086] 6) According to the traditional k-means algorithm, the error sum of squares function is calculated, and the cluster centers are continuously updated. The optimal number of clusters is determined by combining the elbow rule method. The optimal number of clusters is as follows: Figure 3 As shown in a, K clustered data clusters D are determined based on the optimal solution and actual needs. i ,i=1,2,3,...k.
[0087] According to the diagram and actual factors, the number of clusters is selected as 4. After spectral clustering, different categories correspond to photovoltaic output data under different meteorological factors. The clustering results of the divided data clusters are as follows: Figure 3 As shown in b.
[0088] S2. For clusters with different numbers of groups, the local outlier factor algorithm LOF is used to calculate the anomaly score of each data point in the group, and it is used as the third feature of the data group, and each feature is weighted by the objective weighting method CRITIC; by constructing multiple iTree tree models to build an isolation forest iForest model, the anomaly score value is used as the method to distinguish data anomalies, and then the optimal vote is performed on the detection results of the model based on the decision matrix and the superior and inferior solution distance method TOPSIS to obtain the optimal output;
[0089] The final anomaly detection calculation is completed based on the isolation forest algorithm. Since the abnormal data of photovoltaic output is usually highly positively correlated with the output power, while the correlation between irradiance and abnormal data is poor, it can be seen that its learning samples are insufficient. The LOF anomaly score is introduced as its third data feature to enhance the anomaly detection performance of the isolation forest anomaly detection algorithm.
[0090] The specific steps are as follows:
[0091] Set the initial number of isolated trees to 100 and the subsampling size to 256 to build an isolated forest framework and then build an isolated tree model iForest. iForest is an unsupervised learning algorithm suitable for anomaly detection. It does not need to calculate distance, density and other indicators. It is a model that describes anomalies based on the degree of alienation.
[0092] The principle of iForest is: given a data set, select a random subsample to construct a binary tree iTree; construct the branch of the tree by randomly selecting dimensions, that is, obtain the maximum value x of the dimension data max With the minimum value x min , randomly select a point x v , according to other data x on this dimension i With data point x v The left subtree and the right subtree are determined by comparing the size of . The entire branching process is recursively executed on the data set until the predetermined maximum tree depth is reached or a single data is isolated. By constructing a binary tree iTree of different subsamples, iForest is trained in combination, and then the anomaly score is assigned based on the average depth of the branch. The data points to be tested are traversed through all iTrees, and the anomaly score is calculated based on formula (7):
[0093] s(x,n)=2 -E(h(x)) / c(n) (7)
[0094] In the formula, h(x) is the path length of the sample in the tree, and E(h(x)) is the length of a single data point x. i The average of the depths reached in all trees, c(n) is the corrected value:
[0095] c(n)=2H(n-1)-(2(n-1) / n) (8)
[0096] Where H(i) is the harmonic number, estimated by the Euler constant, and n is the number of points in the construction tree.
[0097] The LOF algorithm is a density-based outlier detection method. The basic idea is to evaluate the degree of outlier of each data point by comparing the density of neighboring points around the data point.
[0098] Let cluster D i, i=1,2,3,...k, the calculation point is P, point Q is the k nearest neighbor object of point P, the maximum distance of k nearest neighbors is expressed as k_dist, find its k distance neighborhood, then calculate the reachable distance of point P with respect to point O, and the local reachable density of the data point, and calculate the local anomaly factor based on formula (9):
[0099]
[0100] Where Lrd(P) is the local reachable density of data point P, Lrd(O) is the local reachable density of data point O, and N k_dist (P) is the set of objects in the k-neighborhood of point P.
[0101] Calculate the LOF anomaly score within each cluster and mark the index of the points with higher anomaly scores. Use the LOF anomaly score as the third feature of the dataset.
[0102] 1) Randomly select features from the data set and randomly select cutting points to divide the data points into two left and right subtrees. Continue to recursively build trees until each data point becomes a leaf node, build multiple trees, and form an isolation forest.
[0103] 2) For a given data point, calculate its path length in the isolation tree and get the anomaly score of the data point.
[0104] Since the main algorithm is an isolation forest, in order to reduce the amount of calculation, the similarity calculation between the test set data and the data clusters of the established model is removed, the test set data is used to traverse the anomaly detection model, and the optimal vote is performed on the detection results of the model based on the decision matrix and the superior and inferior solution distance method to obtain the optimal output.
[0105] Application Examples
[0106] Data source and settings
[0107] Matlab is used to establish an anomaly detection program and a Simulink photovoltaic grid-connected model with a power output of 100kw. The real meteorological data from August to November 2023 in Liaoning Province is used to simulate the collected data of 96 data points in a cycle sample of 1 day and 24 hours. In view of the fact that the LCL filtering module has been added to the general photovoltaic model, Gaussian noise with a mean of 0 and a variance of 0.02 is added. A total of 100 cycles, i.e. 9600 samples, are selected. The index of each cycle is extracted from the sample data and randomly divided. The first 80% of the randomly divided cycle index data is used as the model training set, and the remaining 20% of the samples are used as the model test set to reduce the risk of overfitting. Based on Simulink, a photovoltaic grid-connected model is built, and a total of 100 days*96 points of photovoltaic power generation data are simulated. Among them, during the simulation process, faults and external influencing factors will be added to the photovoltaic model at fixed time points to simulate abnormal data. The specific settings are shown in Table 1 below. The following basic fault modules are used to form multiple types of abnormal data outputs. The specific output examples are as follows Figure 4 shown.
[0108] Table 1 Fault distribution settings for different data types
[0109]
[0110]
[0111] The test set has 20 cycles and 1920 data points. The training set data is used to divide the clusters and build the anomaly detection model. The test set data is used to traverse the anomaly detection model and vote for the best result output. The effectiveness and superiority of the anomaly detection model are verified based on the evaluation indicators. The confusion matrix of the training set data traversing the detection results of 4 different cluster models is as follows: Figure 5 shown.
[0112] It can be seen that in the anomaly detection based on weighted data, the optimal output of the confusion matrix true positive result can reach 0.96, and the true negative example can reach 0.83. The proposed method has a low false alarm rate and false negative rate. Furthermore, different detection thresholds are set to 0.7, 0.75, 0.8, 0.83, 0.87, 0.9, and 0.95, and the detection indicators such as precision and recall are calculated. The results are shown in Table 2.
[0113] Table 2 Detection indicators of anomaly detection models with different thresholds
[0114]
[0115] As shown in Table 2, within a certain threshold range, the recall rate is a stable value, while the precision rate gradually increases as the threshold increases. When the threshold reaches 0.83, although the precision rate continues to increase, the recall rate drops rapidly. In order to ensure the best anomaly detection effect and ensure the stability and consistency of the results, the model is run multiple times and cross-validated to complete the model performance evaluation, and the results are voted on optimally. The results are shown in Table 3.
[0116] Table 3. Detection indicators of multiple anomaly detection models
[0117]
[0118]
[0119] In order to verify the correctness of the value of the weight parameter and the influence of the added LOF feature on the calculation results, the data without setting the weight and the LOF feature embedding were selected for comparative experiments. Figure 6 The experimental results are shown in Table 4.
[0120] Further, verify the detection effect of a single original model, the confusion matrix is as follows Figure 7 The experimental results are shown in Table 4.
[0121] Table 4 Detection indicators of anomaly detection models under different settings
[0122]
[0123] By comparison, the original single model has low precision and recall, with the precision stable at around 0.88 and the recall rate only at 0.6. The model has weak ability to identify anomalies and cannot capture most abnormal samples. Furthermore, the model embeds LOF features, and its precision has been slightly reduced, but the recall rate has been enhanced. The precision of each data cluster is stable at 0.91, and the recall rate is stable at 0.81, indicating that the model can successfully capture most abnormal samples in the anomaly recognition at the breadth level, and the possibility of false positives and false negatives is reduced.
[0124] In order to further verify the superiority of the algorithm of the invention, an anomaly detection model based on the triple deviation algorithm, an anomaly detection model based on the LOF algorithm, and an anomaly detection model based on the SVM algorithm were built. Anomaly detection tests were performed on different models, and the anomaly detection indicators of samples under different models were calculated. The anomaly detection of the present invention can be simplified to analyze the effect of binary anomaly detection, and an evaluation index of the anomaly detection effect is given based on this. For binary anomaly detection, the confusion matrix can be used to test its results, and then the four indicators of precision (P), recall (R), F1 Score and G Value are selected to evaluate the test performance of the model. The calculation formulas for each indicator are:
[0125]
[0126] Where TN is a sample with normal labels and detected as normal, FP is a sample with normal labels but detected as abnormal, FN is a sample with abnormal labels but detected as normal, and TP is a sample with abnormal labels and detected as abnormal.
[0127] The experimental results are shown in Table 5 and Figure 8 As shown, it can be seen that the triple deviation method has a high precision but an extremely low recall rate, the LOF algorithm has a high recall rate but insufficient precision, the SVM algorithm has an excellent precision but a low recall rate. In summary, the inventive model has higher detection accuracy. The experimental results further demonstrate the superiority of the method proposed in the present invention.
[0128] Table 5
[0129]
[0130] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. All technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.
Claims
1. A distributed photovoltaic abnormal data detection method based on period optimal path comparison, characterized in that: The following steps are involved: S1. Divide the collected photovoltaic output power data and radiance data into periodic time series of power data and radiance data, calculate the similarity of each point in the periodic time series of power data and radiance data based on the dynamic time warping algorithm DTW, obtain the similarity metric distance between the two sequences and form a cumulative distance matrix; and use the spectral clustering algorithm SCA to extract feature vectors and combine the elbow rule graph to achieve optimal clustering; S2. For clusters with different numbers of groups, the local outlier factor algorithm LOF is used to calculate the anomaly score of each data point in the group, and it is used as the third feature of the data group; the isolation forest iForest model is built by constructing multiple iTree tree models, and the anomaly score value is used as the judgment method for data anomalies. Then, based on the decision matrix and the superior and inferior solution distance method TOPSIS, the detection results of the model are optimally voted to obtain the optimal output.
2. The distributed photovoltaic abnormal data detection method based on period optimal path comparison according to claim 1 is characterized in that: In step S1, the specific process of calculating the similarity of each point in the periodic time series of the power data and the radiance data based on the dynamic time warping algorithm DTW, obtaining the similarity metric distance between the two sequences and forming a cumulative distance matrix is as follows: S101, define the periodic time series of power data X=[i1,i2,...,i m ] and the periodic time series Y=[j1,j2,...,j n ]: Where X, Y are the periodic time series of power data and radiance data, d(i k ,j k ) is the calculated distance of each point in the periodic time series of power data and radiometry data, and p represents the p-norm; S102, using formula (1) to calculate the best matching path between sequences, and minimize the sum of distances of all matching point pairs on the path; S103. Construct a cumulative distance matrix D using formula (2) to calculate the optimal distance Dist, which is the similarity measurement distance between the two sequences: Where D(i,j) is the cumulative distance and d(i,j) is the current element distance.
3. The distributed photovoltaic abnormal data detection method based on period optimal path comparison according to claim 2 is characterized in that: In step S1, the specific process of using the spectral clustering algorithm SCA to extract feature vectors and combining the elbow rule graph to achieve optimal clustering is: S104, record the collected data as X=[x r x p ], x r is the radiometric data, x p is the photovoltaic output power data; the missing value of X is searched, and the collected data are data points with equal time intervals. The Newton interpolation polynomial is used for interpolation, and the corresponding expression is: S105, after constructing the Newton interpolation polynomial, interpolate the specified data. After completing the data filling, the photovoltaic output data needs to be normalized, and z-score normalization is adopted, and the formula is as follows: Where, X norm is the normalized data, X is the original data, and X mean is the mean of the original data, X std is the standard deviation of the original data set; S106, X norm Each line is the primary judgment object, querying the empty value interval of photovoltaic output data and intercepting the real value interval, and X norm Each set of data, i.e., each column of real-valued intervals in the data, is regarded as a new real-valued time series sequence; S107, define the intercepted real-valued sequence p i ,q i , i = 1, 2, ... k, calculate x for each day r ,x p The correlation coefficient is used as the linear weight; S108, using the Euclidean distance to construct the distance matrix D between the sequences and solve the regularized path w, then solve the optimal distance Dist of the two sequences, perform adaptive weighted average calculation, and finally find an optimal regularized path w that minimizes the cumulative distance, that is: Dist min =∑d′(i,j) (5) S109: After calculating the cumulative distance between each sequence and other sequences, the cumulative distance is converted into a similarity matrix D s , clustering based on similarity matrix; S110, for the similarity matrix D s , introduce the inverse distance matrix and perform spectral clustering based on the Gaussian kernel function, where the Gaussian kernel function is constructed as: S111, construct the Laplace matrix and perform vector decomposition, arrange the eigenvalues in descending order, and extract the first k eigenvectors with the smallest eigenvalues according to the number of samples included. i , i = 1, 2, ... k, and take the real part to form the eigenvector matrix; S112. Calculate the error square sum function according to the traditional k-means algorithm, and continuously update the cluster center. Combine the elbow rule method to determine the optimal number of clusters. Determine the data cluster D after K clusters according to the optimal solution and actual needs. i ,i=1,2,3,…k.
4. The distributed photovoltaic abnormal data detection method based on period optimal path comparison according to claim 1 is characterized in that: The step S2 specifically includes: S21, set the initial number of isolated trees to 100, the subsampling size to 256, build an isolation forest framework and then build an isolated tree model; S22, using the local outlier factor algorithm LOF to calculate the LOF anomaly score in each cluster, and marking the index of the point with a higher anomaly score, and using the LOF anomaly score as the third feature of the data set; S23, randomly select features from the data set and randomly select cutting points to divide the data points into two left and right subtrees, and continue to recursively build trees until each data point becomes a leaf node, construct multiple trees, and form an isolation forest; S24. For a given data point, calculate its path length in the isolation tree and obtain the anomaly score of the data point: s(x,n)=2 -E(h(x)) / c(n) (7) In the formula, h(x) is the path length of the sample in the tree, and E(h(x)) is the length of a single data point x. i The average of the depths reached in all trees, c(n) is the corrected value: c(n)=2H(n-1)-(2(n-1) / n) (8) Where H(i) is the harmonic number, estimated by Euler's constant, and n is the number of points in the construction tree; S25. Based on the decision matrix and the superior and inferior solution distance method, the detection results of the model are optimally voted to obtain the optimal output: Obtain the anomaly score matrix and solve the positive and negative ideal solutions of the final anomaly score according to formula (9); Then determine the optimal solution and the worst solution according to formula (10); The final anomaly score value of the anomaly detection model is calculated based on the optimal solution and the worst solution calculated by formula (11).
5. The distributed photovoltaic abnormal data detection method based on period optimal path comparison according to claim 4 is characterized in that: The specific process of using the local outlier factor algorithm LOF to calculate the LOF anomaly score in each cluster is as follows: Let cluster D i , i=1,2,3,…k, the calculation point is P, point Q is the k nearest neighbor object of point P, the maximum distance of the k nearest neighbor is expressed as k_dist, find its k distance neighborhood, then calculate the reachable distance of point P with respect to point O, and the local reachable density of the data point, and calculate the local anomaly factor based on formula (12): Where Lrd(P) is the local reachable density of data point P, Lrd(O) is the local reachable density of data point O, and N k_dist (P) is the set of objects in the k-neighborhood of point P.
6. The distributed photovoltaic abnormal data detection method based on period optimal path comparison according to claim 3 is characterized by: The optimal number of clusters is 4.
Citation Information
Patent Citations
Abnormal point proportion optimization method and device based on spectral clustering and computer equipment
CN109871886A
Photovoltaic output typical scene extraction method combined with spectral clustering algorithm
CN114266480A
Fluid pipeline flow detection method based on adaptive fuzzy spectral clustering algorithm
CN117454211A
Distributed photovoltaic abnormal data score decision-making method based on sequential sequence clustering
CN118277935A
Intrusion detection system and method based on intelligent network
CN118413406A
Cited By
Low-voltage distributed photovoltaic overvoltage adjustment method and device
CN121618511A
Low voltage distributed photovoltaic overvoltage adjustment method and device
CN121618511B
