Distributed photovoltaic abnormal data score decision method based on time series clustering

Abstract

The present invention belongs to the field of photovoltaic technology, and specifically discloses a distributed photovoltaic abnormal data score decision method based on time series clustering. The similarity of different characteristic periodic sequence groups is measured based on a dynamic time warping algorithm to obtain an optimal path similarity matrix, and then clustering is implemented based on k-means and elbow rule graphs to differentiate data and enhance similarity features. A third abnormal score feature is introduced and weighted, and the abnormal score matrix is ​​further obtained based on an isolation forest framework to make the best decision to obtain the optimal output, and the computational complexity is reduced on the basis of considering the difference to complete the abnormal detection; the method of the present invention can effectively capture most abnormal samples, and can effectively improve the precision and recall rate of photovoltaic abnormal data.

Description

Technical Field

[0001] The present invention relates to the field of photovoltaic technology, and in particular to a distributed photovoltaic abnormal data score decision method based on time series clustering. Background Art

[0002] In recent years, the scale and application of photovoltaic and wind power resources have gradually expanded, technology iterations have been continuously updated, solar cells have been industrialized, the costs of each link in the industrial chain have steadily decreased, and photovoltaic related technologies have developed rapidly. However, the volatility of a large number of controllable loads and distributed power sources, the change of the single-ended power supply radial topology of the traditional distribution network after their access to the power grid, and the management risks brought about by the sharp increase in data collection and processing volume have brought more uncertainty problems to the operation of the distribution network, and are also vulnerable to network attacks, causing unsafe operation of the information-physical system of the distribution network. Therefore, it is of great significance to carry out research on abnormal data detection methods of distributed photovoltaic industrial control networks under new power systems.

[0003] In the actual operation of photovoltaic power generation, the probability of various faults is highly random, resulting in unstable photovoltaic output data and easily causing large deviations in the training model. At present, the detection methods for photovoltaic abnormal data in the industry are mainly based on the following two concepts: similarity and difference. Similarity mainly considers the feature similarity measurement of different time series data points, such as similar days or similar time periods. It establishes a benchmark for normal working conditions by analyzing photovoltaic historical data under normal working mode to capture similar patterns and features to identify abnormal conditions. Liu Yan et al. established a Gaussian mixture model and estimated parameters to detect high-proportion abnormal data of photovoltaic power generation output. Zhao Bochao et al. decomposed photovoltaic power generation data into multiple modal components to obtain its periodic and non-periodic characteristics, and identified abnormalities based on the dynamic threshold of predicted values ​​and measured values ​​based on the construction of the VMD-XGBoost-BILSTM combined model. Ye Lin et al. proposed a combined recognition method for high-proportion abnormal data based on the typical characteristics of continuous abnormal data and discrete abnormal data of photovoltaic power, and achieved effective recognition. Although some of the above literatures have identified anomalies based on similar days or similar time periods, the concept of similar day clustering is rarely used in anomaly detection. Applied algorithms such as k-means clustering and grey correlation clustering are sensitive to initial parameters and cannot process time series feature data. Considering that photovoltaic output data is time series data, Wei Lianbin et al. introduced dynamic time warping (DTW) into similar day calculation to achieve effective short-term power prediction. Zhe Chen considered using time series clustering and early random classification algorithms to create new features and mine hidden deep information from historical load characteristics, which can effectively improve the prediction performance of data-driven models. Clustering can better capture the changing patterns of time correlation and help extract hidden laws and patterns in the data.

[0004] Difference is usually associated with similarity work, but it focuses more on the difference characteristics of abnormal data. Shi Min et al. used the sliding standard deviation to clean abnormal data based on the distribution characteristics of abnormal data of photovoltaic operation. Su Yonghe et al. proposed the joint learning of multi-station branch power, which extracts similarity and difference features for anomaly detection by constructing a multi-scale convolutional neural network. Zhou Jiaqi and Bili encode the multivariate time series of photovoltaic inverters into two-dimensional images by constructing feature matrices, and proposed a photovoltaic inverter multivariate time series image anomaly detection model based on GAN to achieve high-accuracy detection of abnormal data of inverters. If the amount of data is large, clustering to form subsets first helps to enhance the similarity characteristics of the data, but the subsequent anomaly detection will increase the calculation amount of the test set, and the abnormal samples are usually relatively small. Clustering may lead to further reduction of abnormal samples. Selecting a suitable algorithm can avoid these drawbacks. Ding H proposed an improved local outlier factor (LOF) and applied it to fault detection of photovoltaic arrays. Chabchoub Y introduced the concept of voting based on the isolation forest (iForest) and designed MVIForest, which converted the calculation of anomaly scores into a collective decision considering most trees. Abdel-Basset M combined the objective weighting method (Criteria Importance Through Intercriteria Correlation, CRITIC) and the Technique for Order Preference by Similarity to Ideal Solution, TOPSIS) based on uncertainty to improve the accuracy of decision maker-based evaluation.

[0005] The above literature has made an in-depth discussion on anomaly detection research, but there is still room for improvement in the data anomaly detection and identification methods for high-proportion distributed power sources: 1) With the vigorous development of a high proportion of new energy, the detection algorithm of the old power grid may be misled by the randomness and volatility of the new energy itself, resulting in false alarms and missed alarms; 2) Most of the existing literature on power data anomaly detection only focuses on distribution networks. The amount of data processed by the power system has increased dramatically under the access of distributed power sources, and there are complex correlations between high-dimensional data. The difficulty of data analysis and processing has increased, and the aggregation of multi-source data has made it difficult to deeply mine valuable information; 3) Distributed photovoltaic power generation data is a typical time series data, and its time series characteristics must be taken into account, and the two types of detection concepts of similarity and difference must be combined to realize anomaly judgment to avoid bias. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a distributed photovoltaic abnormal data score decision method based on time series clustering.

[0007] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0008] The distributed photovoltaic abnormal data score decision method based on time series clustering includes the following steps:

[0009] S1. Data preprocessing: Based on the dynamic time warping (DTW) algorithm, the similarity of each periodic sequence group of radiance and power characteristics is measured, the cumulative distance is obtained and the similarity matrix is ​​constructed, and then the optimal clustering is achieved based on k-means and elbow rule graph;

[0010] S2. Use the LOF algorithm to calculate the anomaly score of each data point in the cluster as the third feature and weight it, and then use it as the data input of the isolation forest to obtain the anomaly score matrix. Then use the superior and inferior solution distance method based on statistical information to make the best decision on the anomaly score matrix of the model and obtain the optimal output.

[0011] Furthermore, in step S1, the specific process of data preprocessing is:

[0012] S101, the collected data is X=[x r x p ], x r is the radiometric data, x p is the photovoltaic output power data; find the missing value of X and use Newton interpolation polynomial to interpolate, the corresponding expression is:

[0013]

[0014] Where: x is the data point; n is the number of data points; f[] is the difference quotient function value; x0, x1, x2…x k Respectively represent given data points; N(x) represents interpolation values;

[0015] S102, performing normalization processing, using Z-Score normalization, the formula is as follows:

[0016]

[0017] Where: X norm is the normalized data; X is the original data; X mean is the mean of the original data; X std is the standard deviation of the original data set;

[0018] S103, intercept x r 、x pThe real-valued sequence of each period in is calculated using the Pearson correlation coefficient. r , x p The correlation coefficient of each day is used as the linear weight ω i :

[0019]

[0020] Where: x rd is the radiometric data set; x pd is the power data set; cov() is the covariance of the data set; is the standard deviation of the radiometric data set; is the standard deviation of the power data set.

[0021] Furthermore, in step S1, the similarity of each periodic sequence group with different features is measured based on the dynamic time warping DTW algorithm, the cumulative distance is obtained and a similarity matrix is ​​formed, and then the specific process of achieving optimal clustering based on k-means and elbow rule graph is as follows:

[0022] S104, twist and stretch the real-valued sequence and solve the path so that the cumulative distance Dist is minimized, that is,

[0023]

[0024] In formula (4), i, j are the data points of the time series; d(i, j) is the distance of the current element; min is the minimum value of each item; Dist(i, j) is the cumulative distance;

[0025] S105, the optimal distance calculated based on formula (4) can be expressed as [Dist r Dist p ], introduce ω i Perform adaptive weighted average calculation to solve the final optimal distance Dist s :

[0026] Dist s =ω i ×Dist r +(1-ω i )×Dist p (5)

[0027] Where: i is the correlation coefficient; Dist r The optimal distance calculated for the radiometric sequence; Dist p The optimal distance calculated for the power series;

[0028] S106, traverse and calculate the optimal distances of the time series data between all cycles, and obtain multiple groups of optimal distances Dist sConvert to a similarity matrix D s ;

[0029] S107. Calculate the similarity matrix D according to the traditional k-means algorithm s The sum of squared errors SSE is:

[0030] SSE=∑d(D s (i s ,j s ),c k )(6)

[0031] In formula (6): i s ,j s is the index of each data point in the matrix; k is the number of clusters; D s (i s ,j s ) is the matrix sample data point, c k is the cluster center, d() is the matrix sample data point D s (i s ,j s ) and cluster center c k The distance between them is selected as the Euclidean distance in this paper;

[0032] S108, continuously updating the sum of squared errors SSE under different numbers of clusters to determine k optimal numbers of clusters;

[0033] S109: Determine k clustered data clusters D according to the optimal solution and actual needs i ,i=1,2,3,…k.

[0034] Furthermore, step S2 specifically includes:

[0035] S21, introduces LOF anomaly score as its third data feature, compares the density of neighboring points around each data point in the data cluster to evaluate the degree of outlier of the point;

[0036] S22. Let cluster D i , i=1,2,3,…k, the calculation point is P, point O is the k nearest neighbor object of point P, and the local anomaly factor is calculated:

[0037]

[0038] Where: Lrd(P) is the local reachable density of data point P; Lrd(O) is the local reachable density of data point O; k_dist is the maximum distance of the k nearest neighbors of data point P; N k_dist (P) is the set of objects in the k-neighborhood of point P;

[0039] S23, after calculating the LOF feature, update the original feature to X = [xr x p x LOF ], weight the three identified features, select the CRITIC weights for objective weighting, and use the weighted features as the input of the isolation forest model:

[0040]

[0041] Where: C is the calculated indicator weight; n is the number of correlation coefficients calculated between each feature; σ i is the characteristic standard deviation; ω ij is the correlation coefficient between each feature;

[0042] S24, set the initial number of isolated trees to 100, the sub-sampling size to 256, build an isolation forest framework, build binary trees iTree with different sub-samples, combine and train iForest, and then assign anomaly scores based on the average depth of the branches, and traverse all iTrees for the data points to be tested;

[0043] S25. For a given data point, calculate its path length in the isolation tree and obtain the anomaly score of the data point:

[0044] s(x,n)=2 -E(h(x)) / c(n) (9)

[0045] In the formula, h(x) is the path length of the sample in the tree, and E(h(x)) is the length of a single data point x. i The average of the depths reached in all trees, c(n) is the corrected value:

[0046] c(n)=2H(n-1)-(2(n-1) / n)(10)

[0047] Where H() is the harmonic number, estimated by Euler's constant, and n is the number of points in the construction tree;

[0048] Furthermore, the specific process of selecting the statistical information of the anomaly score matrix is ​​as follows:

[0049] Use the test set data to traverse the anomaly detection model and obtain the anomaly score matrix s l =[s l1 s l2 s l3 s l4 ], use the anomaly detection model to obtain the anomaly score matrix and calculate the median and maximum value of the anomaly score matrix, and update the anomaly score matrix s l =[s l1 s l2 s l3 s l4 ] is s m =[smid s max ], and the final matrix anomaly score calculation is completed based on formula (11):

[0050] s ml =(a×s mid +b×s max ) / (a+b+α)(11)

[0051] In the formula, a and b are constants, α is a very small number close to 0, and s mid Select the matrix for the median of the anomaly score matrix; s max Select the matrix for the maximum value of the anomaly score matrix; s ml is the calculated matrix anomaly score.

[0052] Furthermore, in formula (11), a=3, b=2; α=0.01.

[0053] Furthermore, the optimal decision process of the model's anomaly score matrix is ​​made by using the good and bad solution distance method based on matrix statistical information selection:

[0054] Find the optimal solution for the matrix anomaly score sl + and the worst solution sl - :

[0055]

[0056] In the formula, sl + is the optimal solution for the matrix anomaly score; sl - is the worst solution for the matrix anomaly score; s ml is the calculated matrix anomaly score; max() is the maximum value, and min() is the maximum value;

[0057] Finally, solve the final decision anomaly score value of the anomaly detection model:

[0058]

[0059] Furthermore, the optimal number of clusters is 4.

[0060] Beneficial effects of the present invention:

[0061] (1) By using the dynamic time warping (DTW) algorithm, the similarity of sequence groups with different characteristics is flexibly and accurately calculated, overcoming the traditional fixed step size or time alignment limitations and improving the robustness of time series data matching. The optimal path similarity matrix obtained in this way can fully reflect the internal structure and pattern changes between different sequences, and then use k-means clustering combined with the elbow method to select the optimal number of clusters, achieving effective differentiation and similarity enhancement of complex photovoltaic data, and improving the accuracy and stability of clustering analysis.

[0062] (2) The third anomaly score feature for photovoltaic systems is innovatively introduced and reasonably weighted to enhance the ability to identify potential abnormal signals. By constructing an isolation forest framework, the data set is deeply mined to obtain a more refined anomaly score matrix, allowing the model to discover hidden anomalies with lower computational complexity. This method not only improves the speed of anomaly detection, but also captures various abnormal events more comprehensively and deeply.

[0063] (3) Using the obtained anomaly score matrix, an optimal decision-making mechanism is proposed. By intelligently sorting and screening the anomaly scores, the most representative anomaly samples are selected to ensure that false positives and false negatives are reduced while ensuring detection accuracy. This stage effectively integrates multiple indicators and achieves optimized screening of anomaly detection results, which helps operation and maintenance personnel quickly locate the source of the fault and improve the reliability of equipment operation.

[0064] (4) The anomaly detection method proposed in the present invention can not only significantly improve the precision and recall of abnormal data, but also maintain a high comprehensive performance index (F-score) under complex backgrounds, and can more accurately and timely identify various abnormal conditions in the photovoltaic system, effectively ensuring the stable operation of the photovoltaic power generation system and the efficient utilization of energy output. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a framework diagram of the distributed photovoltaic abnormal data score decision method based on time series clustering.

[0066] Figure 2 It is a scatter plot of distributed photovoltaic radiation-power.

[0067] Figure 3 Figure 2 is an example diagram of power distribution for each type of fault.

[0068] Figure 4 is the clustering elbow criterion diagram.

[0069] Figure 5 It is the LOF anomaly score feature map of data cluster 1.

[0070] Figure 6 It is the LOF anomaly score feature map of data cluster 2.

[0071] Figure 7 It is the LOF anomaly score feature map of data cluster 3.

[0072] Figure 8 It is the LOF anomaly score feature map of data cluster 4.

[0073] Fig. 9 An example of a confusion matrix output for an anomaly detection model.

[0074] Fig.10 Detection results of different anomaly detection models. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the invention.

[0076] Example

[0077] In view of the problem that the amount of data from the high proportion of distributed power generation connected to the distribution network has increased dramatically and the traditional anomaly detection and recognition methods cannot accurately and efficiently process the photovoltaic anomaly data with time series characteristics, such as Figure 1 As shown, this embodiment proposes a distributed photovoltaic abnormal data score decision method based on time series clustering, including the following steps:

[0078] S1. Data preprocessing: Based on the dynamic time warping (DTW) algorithm, the similarity of each periodic sequence group with different features is measured, the cumulative distance is obtained and the similarity matrix is ​​constructed. Then, the optimal clustering is achieved based on the k-means and elbow rule graph.

[0079] Generally speaking, the main determinant of photovoltaic power generation output is irradiance, and the output power of photovoltaic power stations varies with the change of irradiance. According to relevant literature and actual data verification, photovoltaic output power and irradiance have an approximate linear relationship, so this paper selects photovoltaic output power and irradiance as parameter data.

[0080] The collected data is X=[x r x p ], x r is the radiometric data, x pis the photovoltaic output power data. The new power system is characterized by a high proportion of power electronic equipment, which is susceptible to electromagnetic interference and may cause sensor data loss. Therefore, the missing value of X is first searched. The collected data is equal time interval data points, and Newton interpolation polynomial is used for interpolation. The corresponding expression is:

[0081]

[0082] Where: x is the data point; n is the number of data points; f[] is the difference quotient function value; x0, x1, x2…x k Respectively represent given data points; N(x) represents interpolation values;

[0083] After constructing the Newton interpolation polynomial, the specified data can be interpolated. After completing the data filling, the photovoltaic output data needs to be normalized, and Z-Score normalization is used. The formula is as follows:

[0084]

[0085] Where: X norm is the normalized data; X is the original data; X mean is the mean of the original data; X std is the standard deviation of the original data set.

[0086] X norm Each line is the primary judgment object, querying the empty value interval of photovoltaic output data and intercepting the real value interval, and X norm Each group of data, i.e., each column of real-valued intervals in the data, is regarded as a new real-valued time series. Although the data point of photovoltaic output in one day is determined to be 96, since the real values ​​of the daily irradiance and photovoltaic power output are not determined at certain times, the obtained real-valued sequences may not be equal. As a relatively stable variable, irradiance has a high correlation with photovoltaic output power, and photovoltaic output power is easily affected by external factors. In summary, the weighted average DTW algorithm is used to solve the optimal path of the periodic sequence.

[0087] Intercept x r , x p The real-valued sequence of each period in is calculated using the Pearson correlation coefficient. r , x p The correlation coefficient of each day is used as the linear weight ω i :

[0088]

[0089] Where: x rd is the radiometric data set; x pd is the power data set; cov() is the covariance of the data set; is the standard deviation of the radiometric data set; is the standard deviation of the power data set.

[0090] For the established real-valued sequences, the Euclidean distance is used to construct the distance matrix D between the sequences and solve the regular path, and then the cumulative distance between the two sequences is solved, and finally an optimal regular path is found.

[0091] The real-valued sequence is twisted and stretched and the path is solved so that the cumulative distance Dist is minimized, that is,

[0092]

[0093] Where: i, j are the data points of the time series; d(i, j) is the distance of the current element; min is the minimum value of each item; Dist(i, j) is the cumulative distance.

[0094] The optimal distance calculated based on formula (4) can be expressed as [Dist r Dist p ], introduce ω i Perform adaptive weighted average calculation to solve the final optimal distance Dist s :

[0095] Dist s =ω i ×Dist r +(1-ω i )×Dist p (5)

[0096] Where: i is the correlation coefficient; Dist r The optimal distance calculated for the radiometric sequence; Dist p The optimal distance calculated for the power series.

[0097] Traverse and calculate the optimal distance of the time series data between all cycles, and obtain multiple sets of optimal distances Dist s Convert to a similarity matrix D s .

[0098] Calculate the similarity matrix D according to the traditional k-means algorithm s The sum of squared errors (SSE, Sum of SquaredErrors):

[0099] SSE=∑d(D s (i s ,j s ),c k )(6)

[0100] In formula (6): i s,j s is the index of each data point in the matrix; k is the number of clusters; D s (i s ,j s ) is the matrix sample data point, c k is the cluster center, d() is the matrix sample data point D s (i s ,j s ) and cluster center c k The distance between them is selected as the Euclidean distance in this paper;

[0101] Constantly update the error sum of squares SSE under different cluster numbers to determine the k optimal cluster numbers. The optimal cluster number is as follows: Figure 4 As shown in Figure 2, it can be seen that the SSE decrease rate slows down significantly at the inflection point, so the number of clusters is selected as 4. The clustering results of different data clusters are D i , i=1,2,3,…k corresponds to the photovoltaic output data under different meteorological factors.

[0102] S2. Use the LOF algorithm to calculate the anomaly score of each data point in the cluster as the third feature and weight it, and then use it as the data input of the isolation forest to obtain the anomaly score matrix. Then use the superior and inferior solution distance method based on statistical information to make the best decision on the anomaly score matrix of the model and obtain the optimal output.

[0103] In anomaly detection, clustering data to build a model helps to discover data patterns and structures. If the amount of data is huge, clustering to form subsets first can help to overcome the challenges brought by large-scale data sets to a certain extent. However, it will also increase the amount of calculation for anomaly detection in the test set and further reduce the number of abnormal samples. Selecting a suitable algorithm can effectively avoid these drawbacks. iForest is an unsupervised learning algorithm suitable for anomaly detection. It does not need to calculate distance, density and other indicators. It is a model that describes anomalies based on the degree of alienation. Therefore, the isolation forest algorithm is selected as the main algorithm to build an anomaly detection framework.

[0104] Since the abnormal data of photovoltaic output usually has a high positive correlation with the output power, while the correlation between irradiance and abnormal data is poor, it can be seen that there is a lack of learning samples. The photovoltaic output data of multiple cycles has been superimposed into the clustered data clusters. The data presents density-type characteristics, so the LOF anomaly score is introduced as its third data feature. By comparing the density of neighboring points around each data point in the data cluster, the degree of outlier of the point is evaluated to enhance the anomaly detection performance. The example figure of LOF anomaly score feature of different clusters is as follows: Figure 5-8 shown.

[0105] Let cluster D i, i = 1, 2, 3, ... k, where the calculation point is P, point O is the k nearest neighbor of point P, and the local anomaly factor is calculated based on formula (7):

[0106]

[0107] Where: Lrd(P) is the local reachable density of data point P; Lrd(O) is the local reachable density of data point O; k_dist is the maximum distance of the k nearest neighbors of data point P; N k_dist (P) is the set of objects in the k-neighborhood of point P;

[0108] After calculating the LOF feature, update the original feature to X = [x r x p x LOF ], weight the three identified features, consider the volatility of photovoltaic output data and the correlation between them, select CRITIC weights for objective weighting, calculate the contrast intensity and conflict indicators and standardize them to obtain data information, and use the weighted features as the input of the isolation forest model:

[0109]

[0110] Where: C is the calculated indicator weight; n is the number of correlation coefficients calculated between each feature; σ i is the characteristic standard deviation; ω ij is the correlation coefficient between each feature.

[0111] Set the initial number of isolated trees to 100 and the sub-sampling size to 256 to build an isolation forest framework. By building binary trees iTree with different sub-samples, iForest is trained in combination, and then the anomaly score is assigned based on the average depth of the branch. The data points to be tested are traversed through all iTrees to calculate the anomaly score:

[0112] s(x,n)=2 -E(h(x))c(n) (9)

[0113] Where: h(x) is the path length of the sample in the tree; E(h(x)) is the average depth of a single data point x in all trees; c(n) is the correction value:

[0114] c(n)=2H(n-1)-(2(n-1) / n)(10)

[0115] Where: H() is the harmonic number, estimated by Euler's constant; n is the number of points in the construction tree;

[0116] Since the main algorithm is an isolation forest, in order to reduce the amount of calculation, the similarity calculation between the test set data and the data clusters of the established model is removed, and the test set data is used to traverse the anomaly detection model to obtain the anomaly score matrix s l =[s l1 s l2 s l3 s l4 ], and select the superior and inferior solution distance method based on matrix statistical information selection to make the best decision on the anomaly score matrix of the model and obtain the optimal output.

[0117] Use the anomaly detection model to obtain the anomaly score matrix and calculate the median and maximum value of the anomaly score matrix, and update the anomaly score matrix s l =[s l1 s l2 s l3 s l4 ] is s m =[s mid s max ], complete the final matrix anomaly score calculation:

[0118] s ml =(a×s mid +b×s max ) / (a+b+α) (11)

[0119] In the formula, a and b are constants. In this paper, a=3 and b=2 are taken. α is a very small number close to 0. In this paper, α=0.01 is taken. s mid Select the matrix for the median of the anomaly score matrix; s max Select the matrix for the maximum value of the anomaly score matrix; s ml is the calculated matrix anomaly score.

[0120] Find the optimal solution for the matrix anomaly score sl + and the worst solution sl - :

[0121]

[0122] In the formula, sl + is the optimal solution for the matrix anomaly score; sl - is the worst solution for the matrix anomaly score; s ml is the calculated matrix anomaly score; max() is the maximum value, and min() is the maximum value.

[0123] Finally, solve the final decision anomaly score value of the anomaly detection model:

[0124]

[0125] Application Examples

[0126] Data source and settings:

[0127] The temperature and radiation model was built using real meteorological data from August to November 2023 in a certain place in Liaoning, and multiple period samples were simulated as 96 data points collected in one day. In view of the fact that the LCL filter module has been added to the general photovoltaic model, zero-mean Gaussian noise data was added to the original collected data, and 100 periods of 9600 sample data were selected. The distributed photovoltaic radiation-power scatter plot is shown in the figure. Figure 2 This paper builds a photovoltaic grid-connected model based on Simulink, sets 96 data points per day, and simulates 100 days of photovoltaic power generation data. In the simulation process, faults and external factors will be added to the photovoltaic model at a fixed time point to simulate abnormal data. The specific settings are shown in Table 1 below. The power distribution examples of each type of fault are shown in the figure below. Figure 3 As shown in the figure, photovoltaic anomalies are divided into two categories: sudden discrete anomalies and persistent continuous anomalies. Each category can contain multiple small category anomalies, that is, a small category anomaly may be repaired in a short time and is regarded as a discrete anomaly; if this anomaly is not repaired, it will continue to exist and is regarded as a continuous anomaly.

[0128] Table 1 Fault distribution settings for different data types

[0129]

[0130] Perform cross validation on all sample data in increasing time windows and take the average for comprehensive performance evaluation. Use the training set data to divide clusters and build anomaly detection models. Use the test set data to traverse the anomaly detection model and vote for the best result output. The confusion matrix of the test set data detection results is as follows: Fig. 9 shown.

[0131] It can be seen that in the detection of distributed photovoltaic abnormal data considering the similarity of time series cycle paths and score decisions, the optimal output of the confusion matrix true positive result can reach 0.92, and the true negative example can also reach 0.92. The proposed method has a low false alarm rate and false negative rate. The results of the three model runs are shown in Table 2. The precision and recall rate can reach 0.92, and the recall rate, F1Score and other indicators have been significantly improved, indicating that it has superior ability to identify anomalies and can capture most of the abnormal samples, achieving both the recall rate and the precision rate.

[0132] Table 2 Detection indicators of anomaly detection models with different thresholds

[0133] index 1 2 3 P 0.9221 0.9261 0.8962 R 0.9286 0.9000 0.9231 F1 0.9253 0.9129 0.9095 G 0.9253 0.9130 0.9096

[0134] In order to verify the influence of each module of the model on the calculation results, the anomaly detection models with different modules are selected for comparative experiments, as shown in Table 3. In Table 3, "+" indicates that the model has the module set; "-" indicates that the model has not set the module; "k-means+" and "grey correlation degree+" indicate that the module algorithm is replaced by the current algorithm.

[0135] Table 3 Anomaly detection models under different module settings

[0136] Model Setting up time series clustering Setting LOF Features Setting CRITIC weights Model 1 - + + Model 2 k-means+ + + Model 3 Grey relational degree+ + + Model 4 + - - Model 5 + + - Model 6 + - +

[0137] Considering that the threshold selected in Table 2 is the optimal threshold, in order to intuitively represent the changes in detection results under different module settings of the model, the thresholds of subsequent models are set to 0.9, 0.85, and 0.8 for detection.

[0138] In order to verify the impact of time series clustering on the calculation results, model 1 is tested and the experimental results are shown in Table 4.

[0139] Table 4 Detection indicators of anomaly detection model 1

[0140]

[0141] It can be seen that the evaluation indicators in Model 1 are all low. The precision rate is stable at around 0.9, but the maximum recall rate only reaches 0.45, which cannot capture most abnormal samples and the model has no practical value.

[0142] Furthermore, Model 2 and Model 3 were selected for comparative experiments, and the experimental results are shown in Table 5.

[0143] Table 5 Detection indicators of anomaly detection models under different clustering algorithm settings

[0144]

[0145] It can be seen that different clustering methods have a great impact on the detection performance of abnormal data when obtaining similarity conditions. Although the precision of Model 2 and Model 3 is excellent, the highest recall rate is only about 0.46, indicating that a large number of normal samples are predicted as abnormal samples. This is because the number of abnormal samples is relatively small compared to normal samples, and clustering leads to a further reduction in abnormal samples. Therefore, a large number of normal samples are learned during the model training process, and the ability to capture abnormal samples is weak, resulting in an extremely low recall rate of only 0.46. Time series clustering comparison can effectively capture the similarity and shape changes of time series data of different periods, which helps to gain insight into the patterns of data with differences in subsequent anomaly detection, and can resist the interference of noise and outliers to a certain extent. Compared with Table 2, the effectiveness and practicality of time series clustering comparison can be verified.

[0146] In order to verify the correctness of the value of the weight parameter and the influence of the LOF feature on the calculation results, model 4 was selected for experiment. The experimental results are shown in Table 6.

[0147] Table 6 Detection indicators of anomaly detection model 4

[0148]

[0149] Compared with Table 4, the precision of Model 4 has decreased slightly, but the recall rate has increased by nearly 50 percentage points, and the F1score and G index have also stabilized at 0.8, indicating that it can effectively capture most abnormal samples and improve the anomaly recognition and detection capabilities, which once again verifies the practicality of time series clustering comparison.

[0150] Although the detection index of Model 4 has been greatly improved, given the special structure and unbalanced nature of photovoltaic data, it is necessary to combine other anomaly detection algorithms to improve accuracy and enhance robustness. Furthermore, the detection effects of Model 5 and Model 6 are verified, and the experimental results are shown in Table 7.

[0151] Table 7 Anomaly detection model 5 and model 6 detection indicators

[0152]

[0153] It can be seen that the precision of Model 5 is stable at 0.9, and the recall rate can be as high as 0.8. Compared with the output of Model 4 in Table 6, its precision has been slightly improved, but the recall rate has been reduced. This is because the LOF algorithm calculates the outlier factor based on the local neighborhood. Continuous anomalies may cause the LOF algorithm to be unable to capture the local density changes of the anomaly points, resulting in insufficient recognition of abnormal samples; while the detection indicators in Model 6 change under different thresholds, which can form a certain complementary relationship with Model 5.

[0154] To further verify the superiority of the algorithm in this paper, anomaly detection models based on triple deviation algorithm, LOF algorithm, and SVM algorithm were built. Anomaly detection tests were performed on different models, and the anomaly detection indicators of samples under different models were calculated. The experimental results are shown in Figure 2. Fig.10 As shown in the figure, the triple deviation method has a high precision but a very low recall rate, the LOF algorithm has a high recall rate but insufficient precision, the SVM algorithm has an excellent precision but a low recall rate. In general, the model proposed in this paper has higher detection accuracy and better recognition effect on abnormal samples. The experimental results further demonstrate the superiority of the method proposed in this paper.

[0155] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. All technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. A distributed photovoltaic abnormal data score decision method based on time series clustering, characterized in that: The following steps are involved: S1. Data preprocessing: Based on the dynamic time warping (DTW) algorithm, the similarity of each periodic sequence group of radiance and power characteristics is measured, the cumulative distance is obtained and the similarity matrix is ​​constructed, and then the optimal clustering is achieved based on k-means and elbow rule graph; S2. Use the LOF algorithm to calculate the anomaly score of each data point in the cluster as the third feature and weight it, and then use it as the data input of the isolation forest to obtain the anomaly score matrix. Then use the superior and inferior solution distance method based on statistical information to make the best decision on the anomaly score matrix of the model and obtain the optimal output. In step S1, the specific process of data preprocessing is: S101, the collected data is X=[x r x p ], x r is the radiometric data, x p is the photovoltaic output power data; find the missing value of X and use Newton interpolation polynomial to interpolate, the corresponding expression is: Where: x is the data point; n is the number of data points; f[] is the difference quotient function value; x0, x1, x2…x k Respectively represent given data points; N(x) represents interpolation values; S102, performing normalization processing, using Z-Score normalization, the formula is as follows: Where: X norm is the normalized data; X is the original data; X mean is the mean of the original data; X std is the standard deviation of the original data set; S103, intercept x r 、x p The real-valued sequence of each period in is calculated using the Pearson correlation coefficient. r , x p The correlation coefficient of each day is used as the linear weight ωi: Where: x rd is the radiometric data set; x pd is the power data set; cov() is the covariance of the data set; is the standard deviation of the radiometric data set; is the standard deviation of the power data set; In step S1, the similarity of each periodic sequence group with different features is measured based on the dynamic time warping DTW algorithm, the cumulative distance is obtained and the similarity matrix is ​​formed, and then the specific process of achieving optimal clustering based on k-means and elbow rule diagram is as follows: S104, twist and stretch the real-valued sequence and solve the path so that the cumulative distance Dist is minimized, that is, In formula (4), i, j are the data points of the time series; d(i, j) is the distance of the current element; min is the minimum value of each item; Dist(i, j) is the cumulative distance; S105, the optimal distance calculated based on formula (4) can be expressed as [Dist r Dist p ], introduce ω i Perform adaptive weighted average calculation to solve the final optimal distance Dist s : Dist s =ω i ×Distance r +(1-ω i )×Dist p (5) Where: i is the correlation coefficient; Dist r The optimal distance calculated for the radiometric sequence; Dist p The optimal distance calculated for the power series; S106, traverse and calculate the optimal distances of the time series data between all cycles, and obtain multiple groups of optimal distances Dist s Convert to a similarity matrix D s ; S107. Calculate the similarity matrix D according to the traditional k-means algorithm s The sum of squared errors SSE is: SSE=∑d(D s (i s ,j s ),c k )(6) In formula (6): i s ,j s is the index of each data point in the matrix; k is the number of clusters; D s (i s ,j s ) is the matrix sample data point, c k is the cluster center, d() is the matrix sample data point D s (i s ,j s ) and cluster center c k The distance between them is selected as the Euclidean distance in this paper; S108, continuously updating the sum of squared errors SSE under different numbers of clusters to determine k optimal numbers of clusters; S109: Determine k clustered data clusters D according to the optimal solution and actual needs i ,i=1,2,3,…k.

2. The distributed photovoltaic abnormal data score decision method based on time series clustering according to claim 1 is characterized in that: Step S2 specifically includes: S21, introduces LOF anomaly score as its third data feature, compares the density of neighboring points around each data point in the data cluster to evaluate the degree of outlier of the point; S22. Let cluster D i , i=1,2,3,…k, the calculation point is P, point O is the k nearest neighbor object of point P, and the local anomaly factor is calculated: Where: Lrd(P) is the local reachable density of data point P; Lrd(O) is the local reachable density of data point O; k_dist is the maximum distance of the k nearest neighbors of data point P; N k_dist (P) is the set of objects in the k-neighborhood of point P; S23, after calculating the LOF feature, update the original feature to X = [x r x p x LOF ], weight the three identified features, select the CRITIC weights for objective weighting, and use the weighted features as the input of the isolation forest model: Where: C is the calculated indicator weight; n is the number of correlation coefficients calculated between each feature; σ i is the characteristic standard deviation; ω ij is the correlation coefficient between each feature; S24, set the initial number of isolated trees to 100, the sub-sampling size to 256, build an isolation forest framework, build binary trees iTree with different sub-samples, combine and train iForest, and then assign anomaly scores based on the average depth of the branches, and traverse all iTrees for the data points to be tested; S25. For a given data point, calculate its path length in the isolation tree and obtain the anomaly score of the data point: s(x,n)=2 -E(h(x)) / c(n) (9) In the formula, h(x) is the path length of the sample in the tree, and E(h(x)) is the length of a single data point x. i The average of the depths reached in all trees, c(n) is the corrected value: c(n)=2H(n-1)-(2(n-1) / n)(10) Where H() is the harmonic number, estimated by Euler's constant, and n is the number of points in the construction tree.

3. The distributed photovoltaic abnormal data score decision method based on time series clustering according to claim 2 is characterized in that: The specific process of selecting the statistics of the anomaly score matrix is: Use the test set data to traverse the anomaly detection model and obtain the anomaly score matrix s l =[s l1 s l2 s l3 s l4 ], use the anomaly detection model to obtain the anomaly score matrix and calculate the median and maximum value of the anomaly score matrix, and update the anomaly score matrix s l =[s l1 s l2 s l3 s l4 ] is s m =[s mid s max ], and the final matrix anomaly score calculation is completed based on formula (11): s ml =(a×s mid +b×s max ) / (a+b+a)(11) In the formula, a and b are constants, α is a very small number close to 0, and s mid Select the matrix for the median of the anomaly score matrix; s max Select the matrix for the maximum value of the anomaly score matrix; s ml is the calculated matrix anomaly score.

4. The distributed photovoltaic abnormal data score decision method based on time series clustering according to claim 3 is characterized in that: In formula (11), a=3, b=2; α=0.

01.

5. The distributed photovoltaic abnormal data score decision method based on time series clustering according to claim 3 is characterized in that: The specific process of making the best decision on the anomaly score matrix of the model based on the distance method of superior and inferior solutions selected based on matrix statistical information is as follows: Find the optimal solution for the matrix anomaly score sl + and the worst solution sl - : In the formula, sl + is the optimal solution for the matrix anomaly score; sl - is the worst solution for the matrix anomaly score; s ml is the calculated matrix anomaly score; max() is the maximum value, and min() is the maximum value; Finally, solve the final decision anomaly score value of the anomaly detection model:

6. The distributed photovoltaic abnormal data score decision method based on time series clustering according to claim 1 is characterized in that: The optimal number of clusters is 4.

Images