Data defect detection and correction method for electric power scene

Abstract

The invention provides a data defect detection and correction method for an electric power scene, and the method comprises the steps: collecting the data of a sensor, extracting spatial-temporal features, constructing a global high-dimensional feature representation, and carrying out the dimension reduction, thereby obtaining a feature vector after the dimension reduction; time sequence anomaly detection and space anomaly detection are carried out on the feature vectors after dimension reduction, detection results are synthesized, and a comprehensive anomaly score is calculated; classifying the abnormal data according to the comprehensive abnormal score, and constructing a self-adaptive correction model to preliminarily correct all the abnormal data and iteratively optimize the model; monitoring the corrected data in real time, calculating an anomaly detection error, updating parameters of anomaly detection and correction models, and applying the parameters to the models; the optimal learning rate is dynamically determined to accelerate the convergence of the model, the abnormal detection and correction model parameters are migrated and updated, the model is dynamically updated, and the performance index of the system is monitored in real time, so that the operation safety and stability of the power system are improved.

Description

Technical Field

[0001] The present invention belongs to the field of data management, and in particular relates to a data defect detection and correction method for power scenarios. Background Art

[0002] In modern power systems, the accuracy and integrity of data are key factors in ensuring the safe and stable operation of power systems. Power data mainly comes from various sensors and measuring devices, which are widely distributed in different links such as power stations, substations, transmission lines and power consumption ends. With the development of smart grids, the amount and types of data in power systems are growing exponentially, and the variables involved are complex and diverse, including voltage, current, frequency, phase, temperature, meteorological information, etc. These data are not only large in volume and complex in structure, but also have the characteristics of real-time and high-frequency update. However, due to sensor failures, communication delays or interruptions, equipment aging, environmental interference and other reasons, data in power systems often have defects, such as data loss, outliers, noise and inconsistency. If these data defects are not discovered and corrected in time, they may have a serious impact on the monitoring, analysis and decision-making of the power system, leading to incorrect fault diagnosis, distorted energy efficiency evaluation, and even safety accidents in the power system.

[0003] Existing power data defect detection and correction technologies mainly include methods based on statistical analysis, methods based on physical models, and methods based on machine learning. Methods based on statistical analysis (such as mean method, variance method, sliding window detection, etc.) usually identify anomalies by detecting changes in the statistical characteristics of data, but these methods rely on static assumptions about the data and cannot effectively cope with the complex and changeable power data environment, and are prone to false positives and false negatives. Methods based on physical models (such as power flow calculation models, electromagnetic transient models, etc.) rely on the physical laws of the power system and detect anomalies by analyzing the physical consistency of the system state. Although these methods have a solid theoretical basis, they often require precise model parameters and are difficult to cope with the uncertainty and complexity in actual operation, especially when the system is affected by large-scale disturbances or nonlinear events, the physical model may fail. Methods based on machine learning (such as K-means clustering, support vector machines, random forests, etc.) have received widespread attention in recent years. They can automatically learn complex patterns of data, but traditional machine learning models usually require a large amount of labeled data for training, and are insufficient in dealing with the spatiotemporal correlation and non-stationary characteristics of power data.

[0004] In addition, most current methods lack the systematic integration of data defect detection and correction. Detection and correction are usually performed separately, which leads to low detection accuracy and low correction efficiency. In terms of multi-level data fusion and feature enhancement, existing technologies also fail to fully utilize the spatiotemporal information of power system data and the complementary characteristics of multi-source data. Therefore, it is particularly important to develop a new method that can efficiently integrate the advantages of existing technologies while overcoming their limitations to improve the data quality and overall security of power systems. Summary of the invention

[0005] The present invention comprehensively improves the effect of power data defect detection and correction through innovative multi-source data fusion, detection method combining physical model and data-driven model, adaptive data correction strategy of reinforcement learning, and real-time monitoring and feedback mechanism. This method not only overcomes various deficiencies of existing technologies in data defect detection and correction, but also has the advantages of high precision, high efficiency and strong adaptability, and is suitable for multi-scenario application of power data in modern smart grids.

[0006] In order to achieve the above object, a first aspect of the present invention provides a data defect detection and correction method for a power scenario, the method comprising:

[0007] S1, collect sensor data and extract spatiotemporal features by combining time series characteristics and spatial correlation, fuse the sensor data to construct a global high-dimensional feature representation, and then perform PCA dimensionality reduction processing on the global high-dimensional feature representation to obtain the reduced-dimensional feature vector;

[0008] S2. Build a custom convolutional gated recurrent unit model to perform time series anomaly detection on the reduced feature vectors. Use a custom graph convolutional network combined with the network topology of the power system to perform spatial anomaly detection on the reduced feature vectors. Design a custom fusion decision function to integrate the results of time series anomaly detection and spatial anomaly detection and calculate the comprehensive anomaly score.

[0009] S3. Classify the abnormal data according to the comprehensive anomaly score, and design a correction weight function to set different correction weights for each anomaly type. According to the correction weight function, build an adaptive correction model, and use the adaptive correction model to make preliminary corrections to all abnormal data. Then iteratively optimize the adaptive correction model. During each iteration, update the results of time series anomaly detection and spatial anomaly detection according to the corrected data, and recalculate the comprehensive anomaly score and correction weight. At the end of each iteration of the adaptive correction model, check whether the corrected data meets the convergence condition; if the change amplitude of the corrected data is lower than a preset threshold, it is considered that the correction strategy has converged and the iteration is stopped; wherein, the convergence condition is expressed as follows:

[0010] ||X corr (t)-X corr (t-1)||<∈

[0011] Among them, ||X corr (t)-X corr (t-1)|| is the change range between the correction values; ∈ is the convergence threshold, which controls the convergence speed and accuracy of the correction strategy;

[0012] S4, monitor the corrected data in real time and build an anomaly detection and correction model. Use sliding window anomaly detection to calculate the anomaly detection error. If the anomaly detection error δ(t) in the sliding window exceeds the preset threshold θ, execute the adaptive optimization strategy to update the parameters of the anomaly detection and correction model, and apply the optimized parameters to the anomaly detection and correction model;

[0013] S5. Analyze the optimized anomaly detection and correction data to dynamically determine the optimal learning rate to accelerate model convergence, then introduce a transfer learning strategy to migrate and update the anomaly detection and correction model parameters, and design an online learning mechanism for long-term adaptive optimization strategies to dynamically update the model when new data streams arrive. Finally, after each parameter update, monitor the system's performance indicators in real time to determine subsequent actions; if the system performance continues to improve, continue online learning; if the performance decreases or remains unchanged, check the data characteristics and model structure;

[0014] The custom convolutional gated recurrent unit model takes time t as the unit and transforms the feature vector F pca (t) Input into the custom convolutional gated recurrent unit model, the initial state h of the defined convolutional gated recurrent unit model 0 Set it to zero vector; perform convolution operation on the time series, and combine the state update mechanism of the gated recurrent unit to calculate the abnormal probability vector P at the current moment t , which is expressed as follows:

[0015] P t =σ(W c *F pca (t)+W r ·h t-1 +b+λ||h t-1 -F pca (t)|| 2 )

[0016] Among them, P t is the abnormal probability vector at the current time t; σ(·) is the Sigmoid activation function, which is used to output the probability; W c is the time convolution kernel matrix, extracting time features; * is the convolution operator; W ris the state weight matrix of the gated recurrent unit; h t-1 is the hidden state vector of the previous moment; b is the bias term, which is used to adjust the state update; λ is the regularization parameter, which adjusts the sensitivity of anomaly detection; ||h t-1 -F pca (t)|| 2 is the state difference regularization term, which is used to capture sudden anomalies and state changes and emphasize the smoothness and continuity of the time series;

[0017] The custom graph convolutional network takes time t as the unit and transforms the feature vector F pca (t) and the adjacency matrix A of the power network are used as input to construct a graph convolution layer, which fuses the features of each node with the information of adjacent nodes, as shown below:

[0018] F gcn (t)=σ((D -1 / 2 AD -1 / 2 )·F pca (t)·W g +γ||F pca (t)-F pca (t-1)|| 2 )

[0019] Among them, F gcn (t) is the feature matrix updated by the GCN model, which represents the spatial dependency feature; D is the degree matrix; A is the power network adjacency matrix, which represents the connection relationship between sensors; W g is the weight matrix of the graph convolutional layer; σ(·) represents the activation function, which is used to introduce nonlinearity; γ is the spatial regularization parameter, which adjusts the sensitivity of spatial anomaly detection; ||F pca (t)-F pca (t-1)|| 2 It is a feature difference regularization term, which emphasizes the smooth transition of features in time and reduces anomalies that are misjudged due to instantaneous changes.

[0020] Furthermore, in S1, in order to remove noise and retain the dynamic change characteristics of the data, the spatiotemporal features of the data are first extracted, and the steps are as follows:

[0021] Assume X i (t) represents the data vector collected by sensor i at time t. First, the spatiotemporal features of the data are extracted. Combining the time series characteristics and spatial correlation, the convolution kernel is used to perform convolution operations to define the spatiotemporal convolution operation. for:

[0022]

[0023] Among them, F i(t) is the spatiotemporal characteristic output of sensor i at time t; X i (t+k,l) is the data point of sensor i at time t+k and spatial location l; W k,l is the spatiotemporal convolution kernel weight, K and L are the convolution kernel sizes in the time and space directions respectively;

[0024] The global high-dimensional feature representation F(t) uses a custom fusion function To integrate the data characteristics of all sensors, it is expressed as follows:

[0025]

[0026] Among them, F(t) is the global high-dimensional feature representation at time t, F i (t) is the characteristic output of sensor i at time t, and n is the number of sensors.

[0027] Furthermore, the comprehensive abnormality score is calculated using the following formula:

[0028]

[0029] Among them, P t is the abnormal probability vector at the current time t, F gcn (t) is the feature matrix updated using the graph convolutional network model, S(t) is the comprehensive anomaly score, is the decision fusion function, α is the weighting factor of the temporal anomaly detection result, is the graph convolution feature of sensor i at time t, n is the number of sensors, and β is the dynamic feature difference adjustment parameter.

[0030] Furthermore, the correction weight function is used to determine the correction strength of each abnormality; the more serious the abnormality, the greater the correction strength; the function correction weight function is as follows:

[0031]

[0032] Among them, ω(S(t)) is the correction weight at time t, which determines the strength of the anomaly correction; α is the adjustment parameter used to control the sensitivity of the weight function; S(t) is the comprehensive anomaly score, which indicates the severity of the anomaly at time t.

[0033] Furthermore, the adaptive correction model combines the dynamic changes of time series and the abnormal dependence in space to correct the data by dynamically adjusting the correction amplitude and direction. It is expressed as follows:

[0034]

[0035] Among them, Xcorr (t) is the corrected data, which means the data value at time t after the correction model is applied, X(t) is the original data, which means the observed value at time t, ω(S(t)) is the correction weight, and λ 1 ,λ 2 is the adjustment parameter that controls the influence weight of time and space correction terms, is the reverse mapping function used to transform the time anomaly probability vector P t Converted into the corrected value in the original data space, is the spatial anomaly correction term, representing the Laplacian operator of the graph convolution feature.

[0036] Furthermore, after the adaptive correction model corrects all abnormal data, in each model iteration process, the anomaly detection result is updated according to the corrected data, and the comprehensive anomaly score S(t) and the correction weight ω(S(t)) are recalculated until the correction value converges. The correction value iteration formula is as follows:

[0037]

[0038] Among them, X corr (t) is the correction data of the current iteration, η is the learning rate, is the gradient of the loss function, which represents the deviation between the corrected data and the original data.

[0039] Furthermore, the sliding window anomaly detection calculates an anomaly detection error, which is expressed as follows:

[0040]

[0041] Where δ(t) is the sliding window anomaly detection error at time t, T is the length of the sliding window, and X corr (k) is the corrected data, which represents the data value at time k, and X(k) is the original data, which represents the observed value at time k;

[0042] According to the results of real-time monitoring, if the anomaly detection error δ(t) within the sliding window exceeds the preset threshold θ, the adaptive optimization strategy is triggered to improve the performance of the anomaly detection and correction model by adjusting its parameters;

[0043] A parameter update formula based on gradient descent is introduced, and a custom regularization term λ is combined to prevent overfitting. The parameter update formula is expressed as follows:

[0044]

[0045] in, are the model parameters at time t, η is the learning rate, is the loss function and λ is the regularization parameter.

[0046] Furthermore, the optimized parameters are used to update the model and recalculate the anomaly detection results. and revised strategy X corr,new (t), the calculation formula is as follows:

[0047]

[0048] Among them, X corr,new (t) is the optimized corrected data, which indicates the corrected value of the data after using the updated model. is the new anomaly detection result calculated using the optimized parameters, F gcn,new (t) is the new spatial dependency feature calculated using the optimized parameters, ω(S(t)) is the correction weight, and λ 1 ,λ 2 It is the adjustment parameter that controls the influence weight of time and space correction terms;

[0049] After the optimized model parameters are applied to the system, the performance is continuously monitored and the model effect is evaluated again based on the updated new data. If the optimization effect meets the expectations, the current parameters are maintained; if the performance still does not meet the requirements, the adaptive optimization strategy is continued until the system is stable. The adaptive evaluation conditions are expressed as follows:

[0050] δ new (t)<δ old (t)-∈

[0051] Among them, δ new (t) is the sliding window anomaly detection error after using the optimized parameters, δ old (t) is the sliding window anomaly detection error before optimization, and ∈ is the performance improvement threshold.

[0052] Further, the learning rate η(t) adjustment for dynamically determining the optimal learning rate is expressed as follows:

[0053]

[0054] Among them, η(t) is the learning rate at time t, which is used to control the step size of model parameter update; η 0 is the initial learning rate, usually set to a small positive number; β is a tuning parameter that controls the rate at which the learning rate decreases; δ new (t) is the sliding window anomaly detection error after using the optimized parameters;

[0055] The transfer learning strategy specifically includes:

[0056] Assume there is a source model Θ sIt has been trained on similar power data, and the knowledge of the source model is transferred to the current model Θ(t). The migration update formula of the model parameters is expressed as follows:

[0057] Θ(t)=γ·Θ s +(1-γ)·Θ(t)

[0058] Among them, Θ(t) is the model parameter at the current time t, which represents the model weight for anomaly detection and correction, Θ s is the source model parameter, which represents the model parameter trained on similar power data, and γ is the migration coefficient, which controls the degree of fusion between the source model and the current model.

[0059] Furthermore, the online learning update strategy of the long-term adaptive optimization strategy is expressed as follows:

[0060]

[0061] Among them, Θ(t+1) is the model parameter of the next time step, Θ(t) is the model parameter of the current time step, η(t) is the adaptive learning rate, is the loss function, λ is the regularization parameter, is the gradient of the regularization term; wherein, after each parameter update of the online learning update strategy, the performance indicators of the system are monitored in real time to determine the subsequent actions; wherein, the performance monitoring is expressed as follows:

[0062]

[0063] Among them, Δ perf (t) is the rate of change of performance at time t, X pred (k) is the power data value predicted using the current model, X(k) is the original observation data, and T is the time window length for performance monitoring.

[0064] The beneficial technical effects of the present invention are at least as follows:

[0065] (1) The present invention proposes a data defect detection and correction method for power scenarios, which effectively solves the problems existing in traditional methods through multi-level data fusion and adaptive algorithms. First, the present invention adopts multi-source data fusion and feature enhancement technology to fuse different types of power data (such as voltage, current, temperature, meteorological information, etc.), and uses a multimodal deep learning network to fully explore the spatiotemporal correlation between data and the complementary characteristics of multi-source data, significantly improving the accuracy of data defect detection. By introducing a spatiotemporal convolutional neural network (ST-GCN), the present invention can capture the complex spatiotemporal dependencies in the power system, overcoming the limitations of traditional statistical and machine learning methods in processing multidimensional, time-varying data.

[0066] (2) The present invention innovatively combines the physical model with the data-driven model. When performing data defect detection, it not only relies on the statistical characteristics of the data and the pattern recognition of the machine learning model, but also considers the physical laws of the power system and further verifies the abnormality of the data through the consistency check of the physical model. This multi-level, multi-model joint detection method not only improves the detection accuracy, but also ensures the physical rationality and system consistency of data correction, effectively avoiding the misjudgment and underreporting problems of a single model.

[0067] (3) The present invention proposes an adaptive data correction strategy based on reinforcement learning. Through continuous learning and adjustment, the system can automatically select the optimal data correction strategy under different power operation scenarios. The reinforcement learning model can dynamically optimize the correction strategy during the data correction process, adapt to the complex changes of the power system in real time, and improve the flexibility and accuracy of data correction. At the same time, the present invention also introduces a real-time monitoring and feedback mechanism. By online monitoring of the power data flow and real-time adjustment of the model parameters, it ensures that data defects can still be efficiently detected and corrected under the dynamic changes of the power system. This mechanism not only improves the efficiency of data processing, but also greatly reduces the need for manual intervention, enhancing the adaptability and intelligence of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0069] Figure 1 The present invention is a flow chart of a method for detecting and correcting data defects in power scenarios. DETAILED DESCRIPTION

[0070] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0071] like Figure 1 As shown, an embodiment of the present invention provides an intelligent monitoring method for a power distribution cabinet, the method comprising:

[0072] S1. Collect sensor data and extract spatiotemporal features by combining time series characteristics and spatial correlation. Fuse the sensor data to construct a global high-dimensional feature representation. Then perform PCA dimensionality reduction on the global high-dimensional feature representation to obtain the reduced-dimensional feature vector.

[0073] Furthermore, in said S1, the specific steps are:

[0074] S101. In order to remove noise and retain the dynamic change characteristics of the data, the spatiotemporal features of the data are first extracted. The steps are as follows:

[0075] Assume X i (t) represents the data vector collected by sensor i at time t. First, the spatiotemporal features of the data are extracted. Combining the time series characteristics and spatial correlation, the convolution kernel is used to perform convolution operations to define the spatiotemporal convolution operation. for:

[0076]

[0077] Among them, F i (t) is the spatiotemporal characteristic output of sensor i at time t; X i (t+k,l) is the data point of sensor i at time t+k and spatial location l; W k,l is the spatiotemporal convolution kernel weight, K and L are the convolution kernel sizes in the time and space directions, respectively (for example, when K = 3, L = 3, it means a 3x3 spatiotemporal convolution kernel).

[0078] S102, after extracting the spatiotemporal features, construct a global high-dimensional feature representation F(t), and use a custom fusion function F to integrate the data features of all sensors, as shown below:

[0079]

[0080] Among them, F(t) is the global high-dimensional feature representation at time t, F i (t) is the feature output of sensor i at time t, and n is the number of sensors. This formula means that the spatiotemporal features of all sensors are averaged and fused in time series to form a unified feature vector.

[0081] S103. In order to reduce the dimension and computational complexity of the data and ensure the scale consistency of different data, principal component analysis (PCA) is used to reduce the dimension of the fused high-dimensional features. PCA selects the main components by calculating the covariance matrix and eigenvalues, and reduces the eigenvector F(t) to a new feature space F pca (t), expressed as follows:

[0082] F pca (t) = U T ·F(t)

[0083] Among them, F pca(t) is the reduced feature vector, U is the matrix of the first m principal component vectors of the covariance matrix, m is the number of features after dimensionality reduction (for example, if 95% of the cumulative variance needs to be retained, the first m principal components need to be selected), and F(t) is the fused high-dimensional feature vector.

[0084] S2. Construct a custom convolutional gated recurrent unit model to perform time series anomaly detection on the feature vector after dimensionality reduction. Use a custom graph convolutional network combined with the network topology of the power system to perform spatial anomaly detection on the feature vector after dimensionality reduction. Design a custom fusion decision function to integrate the results of time series anomaly detection and spatial anomaly detection and calculate the comprehensive anomaly score.

[0085] In the power system, anomalies may be caused by sudden events (such as short circuits and overloads) or by gradual failures (such as equipment aging). To this end, a custom convolutional gated recurrent unit network (CG-RNN) is used to extract the feature vector F output from step 1. pca (t) Perform time series anomaly detection. CG-RNN combines the feature extraction capability of convolution operation and the sequence learning capability of gated recurrent unit (GRU), which can effectively capture the dynamic changes of data in the time dimension.

[0086] Specifically, in S2, the specific steps are:

[0087] S201, using a custom convolutional gated recurrent unit network (CG-RNN) to the feature vector F pca (t) For time series anomaly detection, CG-RNN combines the feature extraction capability of convolution operation and the sequence learning capability of gated recurrent unit (GRU), which can effectively capture the dynamic changes of data in the time dimension;

[0088] The model input and initialization are based on time t, and the feature vector F pca (t) is input into CG-RNN, the initial state of the model is h 0 Set to zero vector, time convolution and state update is to perform convolution operation on the time series, and combine the state update mechanism of GRU unit to calculate the abnormal probability vector P at the current moment t , which is expressed as follows:

[0089] P t =σ(W c *F pca (t)+W r ·h t-1 +b+λ||h t-1 -F pca (t)|| 2 )

[0090] Among them, Pt is the abnormal probability vector at the current time t, σ(·) is the Sigmoid activation function used to output the probability, W c is the time convolution kernel matrix, extracting time features, * is the convolution operator, W r is the state weight matrix of the GRU unit, h t-1 is the hidden state vector of the previous moment, b is the bias term used to adjust the state update, λ is the regularization parameter, which adjusts the sensitivity of anomaly detection (for example, λ = 0.1), ||h t-1 -F pca (t)|| 2 It is a state difference regularization term, which is used to capture sudden anomalies and state changes and emphasize the smoothness and continuity of the time series.

[0091] S202. The devices and sensors in the power system are usually interconnected in a complex topological structure, which makes the anomaly have obvious spatial dependence. The present invention uses a custom graph convolutional network (GCN) combined with the network topological structure of the power system to detect spatial anomalies. The GCN model uses the connection information of the power system and the adjacency matrix A to capture the spatial dependency and detect the spatial distribution of anomalies.

[0092] A custom graph convolutional network (GCN) is used to detect spatial anomalies in combination with the network topology of the power system. The GCN model uses the connection information and adjacency matrix A of the power system to capture spatial dependencies and detect the spatial distribution of anomalies. The model input and graph construction are based on time t. The feature vector F pca (t) and the adjacency matrix A of the power network are used as input to construct a graph convolution layer, which fuses the features of each node (i.e., sensor location) and the information of adjacent nodes, as shown below:

[0093] F gcn (t)=σ((D -1 / 2 AD -1 / 2 )·F pca (t)·W g +γ||F pca (t)-F pca (t-1)|| 2 )

[0094] Among them, F gcn (t) is the feature matrix updated by the GCN model, which represents the spatial dependency feature. D is the degree matrix (i.e., the diagonal matrix composed of the elements and each row in the adjacency matrix A). A is the adjacency matrix of the power network, which represents the connection relationship between sensors. W gis the weight matrix of the graph convolutional layer, σ(·) is the activation function (such as ReLU) used to introduce nonlinearity, γ is the spatial regularization parameter that adjusts the sensitivity of spatial anomaly detection (for example, γ = 0.05), and ||F pca (t)-F pca (t-1)|| 2 It is a feature difference regularization term, which emphasizes the smooth transition of features in time and reduces anomalies that are misjudged due to instantaneous changes.

[0095] S203, the result of time anomaly detection P t The result of spatial dependence detection F gcn (t) Merge through a custom fusion decision function To integrate the anomaly detection results in time and space, calculate the comprehensive anomaly score S(t), the formula is as follows:

[0096]

[0097] Among them, P t is the abnormal probability vector at the current time t, F gcn (t) is the feature matrix updated using the graph convolutional network model, S(t) is the comprehensive anomaly score, is the decision fusion function, α is the weighting factor of the temporal anomaly detection result, is the graph convolution feature of sensor i at time t, n is the number of sensors, and β is the dynamic feature difference adjustment parameter that controls the impact of feature mutations in time on the comprehensive anomaly score (e.g., β = 0.1).

[0098] Specifically, this method is designed specifically for the diversity and complexity of power data, and can more accurately identify and classify various potential anomalies, providing high-quality input for subsequent data correction steps.

[0099] S3. Classify the abnormal data according to the comprehensive anomaly score, and design a correction weight function to set different correction weights for each anomaly type. Build an adaptive correction model according to the correction weight function, and use the adaptive correction model to make preliminary corrections to all abnormal data. Then iteratively optimize the adaptive correction model. During each iteration, update the results of time series anomaly detection and spatial anomaly detection according to the corrected data, and recalculate the comprehensive anomaly score and correction weight. At the end of each iteration of the adaptive correction model, check whether the corrected data meets the convergence conditions; if the change range of the corrected data is lower than a preset threshold, it is considered that the correction strategy has converged and the iteration is stopped.

[0100] Specifically, the specific steps of S3 include:

[0101] S301. First, classify the abnormal data according to the comprehensive abnormality score S(t) output by S2, and set different correction weights for each abnormality type. Set a custom correction weight function ω(S(t)) to determine the correction strength of each abnormality. The more serious the abnormality, the greater the correction strength; the function correction weight function is as follows:

[0102]

[0103] Among them, ω(S(t)) is the correction weight at time t, which determines the strength of the anomaly correction; α is an adjustment parameter used to control the sensitivity of the weight function (for example, α=5 indicates a higher sensitivity to the anomaly score); S(t) is the comprehensive anomaly score, which indicates the severity of the anomaly at time t.

[0104] S302, using the correction weights, construct an adaptive correction model. The adaptive correction model combines the dynamic changes of the time series and the abnormal dependence in space, and corrects the data by dynamically adjusting the correction amplitude and direction. It is expressed as follows:

[0105]

[0106] Among them, X corr (t) is the corrected data, which means the data value at time t after the correction model is applied, X(t) is the original data, which means the observed value at time t, ω(S(t)) is the correction weight, and λ 1 ,λ 2 is a tuning parameter (e.g., λ 1 =0.7,λ 2 = 0.3), controlling the influence weight of time and space correction terms, is the reverse mapping function used to transform the time anomaly probability vector P t Converted into the corrected value in the original data space, is the spatial anomaly correction term, representing the Laplacian operator of the graph convolution feature.

[0107] S303, after the adaptive correction model corrects all abnormal data, in each model iteration process, the abnormal detection result is updated according to the corrected data, and the comprehensive abnormal score S(t) and the correction weight ω(S(t)) are recalculated until the correction value converges. The correction value iteration formula is as follows:

[0108]

[0109] Among them, X corr (t) is the correction data of the current iteration, η is the learning rate (for example, η = 0.01), is the gradient of the loss function, which represents the deviation between the corrected data and the original data;

[0110] S304. At the end of each iteration, check whether the correction value has reached the convergence condition. If the change range of the correction value is lower than a preset threshold ∈, it is considered that the correction strategy has converged and the iteration can be stopped. Convergence judgment condition:

[0111] ||X corr (t)-X corr (t-1)||<∈ (10)

[0112] Among them, ||X corr (t)-X corr (t-1)|| is the amplitude of change between the correction values, and ∈ is the convergence threshold, which controls the convergence speed and accuracy of the correction strategy (for example, ∈=0.001).

[0113] S4. Monitor the corrected data in real time and build an anomaly detection and correction model. Use sliding window anomaly detection to calculate the anomaly detection error. If the anomaly detection error δ(t) in the sliding window exceeds the preset threshold θ, execute the adaptive optimization strategy to update the parameters of the anomaly detection and correction model, and apply the optimized parameters to the anomaly detection and correction model.

[0114] Specifically, the specific steps of S4 include:

[0115] S401, firstly, the corrected data X corr (t) Real-time monitoring is performed to detect new anomalies or potential abnormal trends. The sliding window method is used to continuously track changes in power data and calculate the error of the anomaly detection model. If multiple abnormal points are detected within a time window T, it means that the current model may not adapt to the changes in the power system and needs to be optimized. The sliding window anomaly detection calculates the anomaly detection error, which is expressed as follows:

[0116]

[0117] Where δ(t) is the sliding window anomaly detection error at time t, T is the length of the sliding window (for example, T = 10 means the error is calculated over the past 10 time points), and X corr (k) is the corrected data, which represents the data value at time k, and X(k) is the original data, which represents the observed value at time k.

[0118] S402: According to the results of real-time monitoring, if the anomaly detection error δ(t) in the sliding window exceeds the preset threshold θ, the adaptive optimization strategy is triggered to improve the performance of the anomaly detection and correction model by adjusting the parameters of the anomaly detection and correction model;

[0119] A parameter update formula based on gradient descent is introduced, and a custom regularization term λ is combined to prevent overfitting. The parameter update formula is expressed as follows:

[0120]

[0121] in, are the model parameters at time t (e.g., the weight matrix of the anomaly detection model), η is the learning rate (e.g., η = 0.01), which is used to control the step size of parameter updates, is a loss function used to evaluate the anomaly detection effect of the current model, and λ is a regularization parameter to prevent overfitting (for example, λ = 0.05).

[0122] S403. Each time the model parameters are updated, the optimized parameters need to be applied to the anomaly detection and correction model. The updated model will be used for the next real-time data processing to ensure that the system can adapt to the new power data environment. Through the real-time feedback mechanism and parameter adaptive optimization, the system can continuously adjust itself to maintain efficient anomaly detection and data correction capabilities.

[0123] Use the optimized parameters to update the model and recalculate the anomaly detection results and revised strategy X corr ,new (t), the calculation formula is as follows:

[0124]

[0125] Among them, X corr,new (t) is the optimized corrected data, which indicates the corrected value of the data after using the updated model. is the new anomaly detection result calculated using the optimized parameters, F gcn,new (t) is the new spatial dependency feature calculated using the optimized parameters, ω(S(t)) is the correction weight, and λ 1 ,λ 2 It is an adjustment parameter that controls the influence weight of time and space correction terms.

[0126] S404. After the optimized model parameters are applied to the system, the performance is continuously monitored and the model effect after the new data is updated is evaluated again. If the optimization effect meets the expectation, the current parameters are maintained; if the performance still does not meet the requirements, the adaptive optimization strategy is continued until the system is stable; wherein the adaptive evaluation conditions are expressed as follows:

[0127] δ new (t)<δ old (t)-∈

[0128] Among them, δ new(t) is the sliding window anomaly detection error after using the optimized parameters, δ old (t) is the sliding window anomaly detection error before optimization, and ∈ is the performance improvement threshold (e.g., ∈=0.001).

[0129] S5. Analyze the optimized anomaly detection and correction data to dynamically determine the optimal learning rate to accelerate model convergence, then introduce a transfer learning strategy to migrate and update the anomaly detection and correction model parameters, and design an online learning mechanism for long-term adaptive optimization strategies to dynamically update the model when new data streams arrive. Finally, after each parameter update, monitor the system's performance indicators in real time to determine subsequent actions; if the system performance continues to improve, continue online learning; if the performance decreases or remains unchanged, check the data characteristics and model structure;

[0130] Specifically, the specific steps of S5 include:

[0131] S501, based on the optimized abnormality detection and correction data X output in step 4 corr,new (t), the learning rate is adaptively adjusted during the model training process. By analyzing the data change trend and anomaly detection error, the optimal learning rate is determined to accelerate model convergence and prevent overfitting. The learning rate η(t) that dynamically determines the optimal learning rate is adjusted as follows:

[0132]

[0133] Among them, η(t) is the learning rate at time t, which is used to control the step size of model parameter update; η 0 is the initial learning rate, usually set to a small positive number such as η 0 =0.01; β is a tuning parameter that controls the rate at which the learning rate decreases; δ new (t) is the sliding window anomaly detection error after using the optimized parameters.

[0134] S502: To ensure that the model can adapt to the new power data characteristics, a migration learning strategy is introduced to migrate and update the model parameters. Assume that there is a source model Θ s It has been trained on similar power data, and the knowledge of the source model is transferred to the current model Θ(t). The migration update formula of the model parameters is expressed as follows:

[0135] Θ(t)=γ·Θ s +(1-γ)·Θ(t)

[0136] Among them, Θ(t) is the model parameter at the current time t, which represents the model weight for anomaly detection and correction, Θ sis the source model parameter, which represents the model parameter trained on similar power data. γ is the migration coefficient, which controls the degree of fusion between the source model and the current model. It is usually in the range of [0,1]. For example, γ = 0.5 represents uniform fusion.

[0137] S503. For long-term model optimization, an online learning mechanism is designed to dynamically update the model when new data streams arrive. This mechanism can ensure that the model continuously learns new data characteristics and quickly adapts when the power system environment changes. The online learning update strategy of the long-term adaptive optimization strategy is expressed as follows:

[0138]

[0139] Among them, Θ(t+1) is the model parameter of the next time step, Θ(t) is the model parameter of the current time step, η(t) is the adaptive learning rate, is the loss function, λ is the regularization parameter used to prevent overfitting, usually a small positive number, such as λ = 0.01, is the gradient of the regularization term, which is used to introduce the penalty term to control the complexity of the model; after each parameter update, the system performance indicators (such as detection accuracy, correction error, etc.) are monitored in real time. If the system performance continues to improve, continue online learning; if the performance decreases or remains unchanged, check the data characteristics and model structure, and adjust the migration coefficient γ and learning rate η(t) if necessary, as shown below:

[0140]

[0141] Among them, Δ perf (t) is the rate of change of performance at time t, X pred (k) is the power data value predicted by the current model, X(k) is the original observation data, and T is the time window length of performance monitoring, for example, T=10, which means that the performance change rate is calculated within the past 10 time points.

[0142] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a division of logical functions. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0143] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0144] Although embodiments of the present invention have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A data defect detection and correction method for power scenarios, characterized in that: The method comprises: S1, collect sensor data and extract spatiotemporal features by combining time series characteristics and spatial correlation, fuse the sensor data to construct a global high-dimensional feature representation, and then perform PCA dimensionality reduction processing on the global high-dimensional feature representation to obtain the reduced-dimensional feature vector; S2. Build a custom convolutional gated recurrent unit model to perform time series anomaly detection on the reduced feature vectors. Use a custom graph convolutional network combined with the network topology of the power system to perform spatial anomaly detection on the reduced feature vectors. Design a custom fusion decision function to integrate the results of time series anomaly detection and spatial anomaly detection and calculate the comprehensive anomaly score. S3. Classify the abnormal data according to the comprehensive anomaly score, and design a correction weight function to set different correction weights for each anomaly type. According to the correction weight function, build an adaptive correction model, and use the adaptive correction model to make preliminary corrections to all abnormal data. Then iteratively optimize the adaptive correction model. During each iteration, update the results of time series anomaly detection and spatial anomaly detection according to the corrected data, and recalculate the comprehensive anomaly score and correction weight. At the end of each iteration of the adaptive correction model, check whether the corrected data meets the convergence condition; if the change amplitude of the corrected data is lower than a preset threshold, it is considered that the correction strategy has converged and the iteration is stopped; wherein, the convergence condition is expressed as follows: ∥X corr (t)-X corr (t-1)∥<∈ Among them, ∥X corr (t)-X corr (t-1)∥ is the change range between the correction values;∈ is the convergence threshold, which controls the convergence speed and accuracy of the correction strategy; S4, monitor the corrected data in real time and build an anomaly detection and correction model. Use sliding window anomaly detection to calculate the anomaly detection error. If the anomaly detection error δ(t) in the sliding window exceeds the preset threshold θ, execute the adaptive optimization strategy to update the parameters of the anomaly detection and correction model, and apply the optimized parameters to the anomaly detection and correction model; S5. Analyze the optimized anomaly detection and correction data to dynamically determine the optimal learning rate to accelerate model convergence, then introduce a transfer learning strategy to migrate and update the anomaly detection and correction model parameters, and design an online learning mechanism for long-term adaptive optimization strategies to dynamically update the model when new data streams arrive. Finally, after each parameter update, monitor the system's performance indicators in real time to determine subsequent actions; if the system performance continues to improve, continue online learning; if the performance decreases or remains unchanged, check the data characteristics and model structure; The custom convolutional gated recurrent unit model takes time t as the unit and transforms the feature vector F pca (t) is input into the custom convolutional gated recurrent unit model, and the initial state h0 of the defined convolutional gated recurrent unit model is set to zero vector; the convolution operation is performed on the time series, and combined with the state update mechanism of the gated recurrent unit, the abnormal probability vector P at the current moment is calculated. t , which is expressed as follows: P t =σ(W c *F pca (t)+W r ·h t-1 +b+λ∥h t-1 -F pca (t)∥ 2 ) Among them, P t is the abnormal probability vector at the current time t; σ(·) is the Sigmoid activation function, which is used to output the probability; W c is the time convolution kernel matrix, extracting time features; * is the convolution operator; W r is the state weight matrix of the gated recurrent unit; h t-1 is the hidden state vector of the previous moment; b is the bias term, which is used to adjust the state update; λ is the regularization parameter, which adjusts the sensitivity of anomaly detection; ∥h t-1 -F pca (t)∥ 2 is the state difference regularization term, which is used to capture sudden anomalies and state changes and emphasize the smoothness and continuity of the time series; The custom graph convolutional network takes time t as the unit and transforms the feature vector F pca (t) and the adjacency matrix A of the power network are used as input to construct a graph convolution layer, which fuses the features of each node with the information of adjacent nodes, as shown below: F gcn (t)=σ((D -1 / 2 AD -1 / 2 )·F pca (t)·W g +γ∥F pca (t)-F pca (t-1)∥ 2 ) Among them, F gcn (t) is the feature matrix updated by the GCN model, which represents the spatial dependency feature; D is the degree matrix; A is the power network adjacency matrix, which represents the connection relationship between sensors; W g is the weight matrix of the graph convolutional layer; σ(·) represents the activation function, which is used to introduce nonlinearity; γ is the spatial regularization parameter, which adjusts the sensitivity of spatial anomaly detection; ∥F pca (t)-F pca (t-1)∥ 2 It is a feature difference regularization term, which emphasizes the smooth transition of features in time and reduces anomalies that are misjudged due to instantaneous changes.

2. A data defect detection and correction method for power scenarios according to claim 1, characterized in that: In S1, in order to remove noise and retain the dynamic change characteristics of the data, the spatiotemporal features of the data are first extracted, and the steps are as follows: Assume X i (t) represents the data vector collected by sensor i at time t. First, the spatiotemporal features of the data are extracted. Combining the time series characteristics and spatial correlation, the convolution kernel is used to perform convolution operations to define the spatiotemporal convolution operation. for: Among them, F i (t) is the spatiotemporal characteristic output of sensor i at time t; X i (t+k,l) is the data point of sensor i at time t+k and spatial location l; W k,l is the spatiotemporal convolution kernel weight, K and L are the convolution kernel sizes in the time and space directions respectively; The global high-dimensional feature representation F(t) uses a custom fusion function F to integrate the data features of all sensors, as shown below: Among them, F(t) is the global high-dimensional feature representation at time t, F i (t) is the characteristic output of sensor i at time t, and n is the number of sensors.

3. The data defect detection and correction method for power scenarios according to claim 1 is characterized in that: The comprehensive abnormality score is calculated using the following formula: Among them, P t is the abnormal probability vector at the current time t, F gcn (t) is the feature matrix updated using the graph convolutional network model, S(t) is the comprehensive anomaly score, is the decision fusion function, α is the weighting factor of the temporal anomaly detection result, is the graph convolution feature of sensor i at time t, n is the number of sensors, and β is the dynamic feature difference adjustment parameter.

4. The data defect detection and correction method for power scenarios according to claim 1 is characterized in that: The correction weight function is used to determine the correction strength of each abnormality; the more serious the abnormality, the greater the correction strength; the function correction weight function is as follows: Among them, ω(S(t)) is the correction weight at time t, which determines the strength of the anomaly correction; α is the adjustment parameter used to control the sensitivity of the weight function; S(t) is the comprehensive anomaly score, which indicates the severity of the anomaly at time t.

5. A data defect detection and correction method for power scenarios according to claim 4, characterized in that: The adaptive correction model combines the dynamic changes of time series and the abnormal dependence in space to correct the data by dynamically adjusting the correction amplitude and direction. It is expressed as follows: Among them, X corr (t) is the corrected data, which means the data value at time t after the correction model is applied. X(t) is the original data, which means the observed value at time t. ω(S(t)) is the correction weight. λ1 and λ2 are adjustment parameters that control the influence weight of the time and space correction terms. is the reverse mapping function used to transform the time anomaly probability vector P t Converted into the corrected value in the original data space, is the spatial anomaly correction term, representing the Laplacian operator of the graph convolution feature.

6. A data defect detection and correction method for power scenarios according to claim 5, characterized in that: After the adaptive correction model corrects all abnormal data, in each model iteration, the anomaly detection results are updated according to the corrected data, and the comprehensive anomaly score S(t) and the correction weight ω(S(t)) are recalculated until the correction value converges. The correction value iteration formula is as follows: Among them, X corr (t) is the correction data of the current iteration, η is the learning rate, is the gradient of the loss function, which represents the deviation between the corrected data and the original data.

7. A data defect detection and correction method for power scenarios according to claim 1, characterized in that: The sliding window anomaly detection calculates the anomaly detection error, which is expressed as follows: Where δ(t) is the sliding window anomaly detection error at time t, T is the length of the sliding window, and X corr (k) is the corrected data, which represents the data value at time k, and X(k) is the original data, which represents the observed value at time k; According to the results of real-time monitoring, if the anomaly detection error δ(t) within the sliding window exceeds the preset threshold θ, the adaptive optimization strategy is triggered to improve the performance of the anomaly detection and correction model by adjusting its parameters; A parameter update formula based on gradient descent is introduced, and a custom regularization term λ is combined to prevent overfitting. The parameter update formula is expressed as follows: in, are the model parameters at time t, η is the learning rate, is the loss function and λ is the regularization parameter.

8. A data defect detection and correction method for power scenarios according to claim 7, characterized in that: Use the optimized parameters to update the model and recalculate the anomaly detection results and revised strategy X corr,new (t), the calculation formula is as follows: Among them, X corr,new (t) is the optimized corrected data, which indicates the corrected value of the data after using the updated model. is the new anomaly detection result calculated using the optimized parameters, F gcn,new (t) is the new spatial dependency feature calculated using the optimized parameters, ω(S(t)) is the correction weight, and λ1, λ2 are adjustment parameters that control the influence weights of the time and space correction terms; After the optimized model parameters are applied to the system, the performance is continuously monitored and the model effect is evaluated again based on the updated new data. If the optimization effect meets the expectations, the current parameters are maintained; if the performance still does not meet the requirements, the adaptive optimization strategy is continued until the system is stable. The adaptive evaluation conditions are expressed as follows: δ new (t)<δ old (t)-∈ Among them, δ new (t) is the sliding window anomaly detection error after using the optimized parameters, δ old (t) is the sliding window anomaly detection error before optimization, and ∈ is the performance improvement threshold.

9. The data defect detection and correction method for power scenarios according to claim 1 is characterized in that: The learning rate η(t) adjustment for dynamically determining the optimal learning rate is expressed as follows: Among them, η(t) is the learning rate at time t, which is used to control the step size of model parameter update; η0 is the initial learning rate, which is usually set to a small positive number; β is the adjustment parameter that controls the rate at which the learning rate decreases; δ new (t) is the sliding window anomaly detection error after using the optimized parameters; The transfer learning strategy specifically includes: Assume there is a source model Θ s It has been trained on similar power data, and the knowledge of the source model is transferred to the current model Θ(t). The migration update formula of the model parameters is expressed as follows: Θ(t)=γ·Θ s +(1-γ)·Θ(t) Among them, Θ(t) is the model parameter at the current time t, which represents the model weight for anomaly detection and correction, Θ s is the source model parameter, which represents the model parameter trained on similar power data, and γ is the migration coefficient, which controls the degree of fusion between the source model and the current model.

10. A data defect detection and correction method for power scenarios according to claim 9, characterized in that: The online learning update strategy of the long-term adaptive optimization strategy is expressed as follows: Among them, Θ(t+1) is the model parameter of the next time step, Θ(t) is the model parameter of the current time step, η(t) is the adaptive learning rate, is the loss function, λ is the regularization parameter, is the gradient of the regularization term; wherein, after each parameter update of the online learning update strategy, the performance indicators of the system are monitored in real time to determine the subsequent actions; wherein, the performance monitoring is expressed as follows: Among them, Δ perf (t) is the rate of change of performance at time t, X pred (k) is the power data value predicted using the current model, X(k) is the original observation data, and T is the time window length for performance monitoring.

Images