Maha-DMVAE-based power grid operation mode data iterative cleaning method and terminal

Abstract

The invention discloses a Maha-DMVAE-based power grid operation mode data iterative cleaning method and a terminal. The method comprises the following steps: collecting topological structure data and dynamic load measurement data related to a power grid operation mode; constructing a feature matrix based on a topological quantile characterization coefficient based on the topological structure data and the dynamic load measurement data; designing and training a Mahaa-DMVAE model which comprises a Mahalanobis distance measurement module by using the characteristic matrix, wherein the Mahaa-DMVAE model comprises the Mahalanobis distance measurement module; taking the adjustment of the Mahalanobis distance of the submerged space as a target, taking the feature matrix as input, and outputting a final matrix by using a Maha-DMVAE model; and based on a power flow residual value confidence dual-mode evaluation system, evaluating the effectiveness of the final matrix, if the final matrix passes the effectiveness evaluation, considering that the Maha-DMVAE model is effective, and using the Maha-DMVAE model to clean the data related to the power grid operation mode. According to the method, accurate identification and high-confidence intelligent correction of the abnormal data are realized, and the purification effect of the power grid data and the physical authenticity and availability of the final data are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of power grid data processing and intelligent operation and maintenance, and in particular to an iterative cleaning method and terminal for power grid operation mode data based on Maha-DMVAE. Background Technology

[0002] As the power grid continues to expand and its operation becomes increasingly complex, power grid operation data exhibits characteristics such as high dimensionality, strong correlation, and continuous time series. During the data collection process, the power grid energy management system, control system, and various intelligent terminals are frequently affected by factors such as communication delays, sensor drift, sampling failures, and network congestion, often resulting in abnormal data, missing data, and pseudo-measurements inconsistent with the actual operating status. These problems directly affect the accuracy of upper-level operations such as power flow calculation, state estimation, and load forecasting, and may even lead to biased operational decisions, posing a threat to the safe and stable operation of the power grid.

[0003] Existing data cleaning methods mostly rely on threshold-based rule verification or simple elimination strategies based on statistical characteristics. These methods often fail to fully utilize the topological relationships and temporal correlations of power grid operation data, making it difficult to effectively identify "minor anomalies," "combined anomalies," or "interrelated anomaly samples." Furthermore, traditional cleaning methods typically focus on deleting anomalous data, which can easily lead to the loss of valuable information and fail to meet the requirements of power grid operation analysis for data integrity and high fidelity. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a method and terminal for iterative cleaning of power grid operation mode data based on Maha-DMVAE, which can achieve accurate identification and high-confidence intelligent correction of abnormal data.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: An iterative data cleaning method for power grid operation modes based on Maha-DMVAE includes the following steps: S1. Collect topology data and dynamic load measurement data related to power grid operation mode; S2. Based on the topology data and the dynamic load measurement data, construct a feature matrix based on the topology quantile characteristic coefficients; S3. Design and train a Maha-DMVAE model including a Mahalanobis distance metric module using the feature matrix; S4. With the goal of adjusting the Mahalanobis distance of the latent space, and using the feature matrix as input, the final matrix is ​​output using the Maha-DMVAE model. S5. Based on the power flow residual confidence dual-mode evaluation system, evaluate the validity of the final matrix. If the final matrix passes the validity evaluation, the Maha-DMVAE model is considered valid, and the Maha-DMVAE model is used to clean the data related to the power grid operation mode.

[0006] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows: A terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement steps in an iterative data cleaning method for power grid operation modes based on Maha-DMVAE.

[0007] The beneficial effects of this invention are as follows: It provides an iterative cleaning method and terminal for power grid operation mode data based on Maha-DMVAE. By integrating the latent space Mahalanobis distance anomaly score during the data cleaning stage, it achieves accurate identification and high-confidence intelligent correction of abnormal data. This not only fundamentally changes the traditional logic of passive deletion and preserves data integrity, but also constructs a closed-loop self-optimizing feedback mechanism, which significantly improves the cleaning effect of power grid data and the physical authenticity and usability of the final data. Attached Figure Description

[0008] Figure 1 This is a flowchart of an iterative data cleaning method for power grid operation modes based on Maha-DMVAE, according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating the construction process of the Maha-DMVAE model according to an embodiment of the present invention; Figure 3 A flowchart illustrating the adjustment of the Mahalanobis distance in the latent space according to an embodiment of the present invention. Figure 4 This is a schematic diagram of a terminal according to an embodiment of the present invention; Label Explanation: 1. A terminal; 2. A memory; 3. A processor. Detailed Implementation

[0009] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0010] Before detailing the embodiments of this application, some related concepts will first be explained: Mahalanobis distance is a statistical distance measure that uses the mean vector and covariance matrix to measure the degree to which a data point deviates from the overall distribution. It considers not only the numerical differences in each dimension but also the correlations between these dimensions, making the same numerical deviation appear more significant in different directions. In this way, the degree to which a data point matches the overall shape of the dataset can be quantified. When a point shows inconsistencies across multiple dimensional combinations, the distance value increases significantly, thus enabling more accurate identification of outliers in multidimensionally correlated scenarios than ordinary distance.

[0011] The Maha-DMVAE model is a variational autoencoder model consisting of an encoder, sampling layer, and decoder. By establishing a latent space distribution within the model, it enables the input data to form a statistically structured representation in the latent space. This is further combined with a self-attention module to extract correlations between features, ultimately allowing for the reconstruction of the input data. Furthermore, the model incorporates Mahalanobis distance in the latent space anomaly scoring stage, enabling it to both learn the compact distribution of normal samples and identify anomalous samples based on the degree of deviation, thus achieving a joint cleaning capability of reconstruction and statistical measurement.

[0012] Quantile transformation is a nonlinear normalization method based on the feature distribution pattern. It maps the original value to its corresponding quantile by calculating the empirical cumulative distribution function of each feature, and then projects the quantile onto a predetermined target distribution. This results in a stable and consistent distribution structure for different feature dimensions after processing. This approach significantly reduces the impact of dimensional differences and skewed distributions on model training, ensuring a smoother and more learnable numerical feature structure in the feature matrix when input into the model. This improves the stability of subsequent model reconstruction and anomaly detection capabilities.

[0013] Please refer to Figure 1 This invention provides an iterative data cleaning method for power grid operation modes based on Maha-DMVAE, comprising the following steps: S1. Collect topology data and dynamic load measurement data related to power grid operation mode; S2. Based on the topology data and the dynamic load measurement data, construct a feature matrix based on the topology quantile characteristic coefficients; S3. Design and train a Maha-DMVAE model including a Mahalanobis distance metric module using the feature matrix; S4. With the goal of adjusting the Mahalanobis distance of the latent space, and using the feature matrix as input, the final matrix is ​​output using the Maha-DMVAE model. S5. Based on the power flow residual confidence dual-mode evaluation system, evaluate the validity of the final matrix. If the final matrix passes the validity evaluation, the Maha-DMVAE model is considered valid, and the Maha-DMVAE model is used to clean the data related to the power grid operation mode.

[0014] As can be seen from the above description, the beneficial effects of this invention are as follows: By unifying the static topology information and dynamic measurement information of the power grid through the feature matrix, the model can learn different types of feature associations under the same data structure; by reconstructing the internal structure of the data through the Maha-DMVAE model, the model can establish a data distribution in the latent space; by using an iterative correction strategy based on Mahalanobis distance, the deviation of abnormal samples in the latent space can be weakened round by round; by using dual verification of power flow residuals and confidence indices, the cleaning results are ensured to be consistent in both statistical characteristics and physical constraints, thereby effectively improving the availability of power grid operation mode data and enabling subsequent power flow calculations, state estimations, and other processes to be built on a more stable data foundation.

[0015] In some implementations, step S2 includes: S21. Instantiate the topology data and extract static features; S22. Convert dynamic load measurement data into a time series matrix based on time sections; S23. The static features are concatenated with the time series matrix to form a preliminary feature matrix, and the preliminary feature matrix is ​​subjected to quantile transformation to form a feature matrix based on topological quantile feature coefficients.

[0016] As described above, after instantiation, topological data can be used to extract static features, clearly expressing the structural relationships between nodes; dynamic load data, after time-section reshaping, forms a time-series matrix, enabling the model to learn changes in operating status over time; the concatenation of static and dynamic features ensures that each time section has a complete expression of node characteristics; quantile transformation unifies the numerical differences of different feature dimensions, thereby improving the stability of model training. Thus, through a unified data representation method, the model can simultaneously learn node location attributes, connectivity relationships, and operating status, making subsequent anomaly identification and correction more targeted and accurate.

[0017] In some implementations, the topology data includes at least device IDs, device parameters, and device connection relationships; the dynamic load measurement data includes at least active power, reactive power, current, and voltage bound to the device IDs. Step S21 specifically includes: S211. Convert the device ID and device parameters in the topology data into device nodes, convert the device connection relationship into edges, and instantiate the device nodes and edges into graph objects; S212. For each device node in the graph object, calculate the degree centrality, betweenness centrality, proximity centrality, and clustering coefficient. S213. Using a log-geometric mean fusion strategy, the degree centrality, betweenness centrality, proximity centrality, and clustering coefficients are nonlinearly fused into a topological centrality index, as shown in the following formula:

[0018] In the formula, TCI Represents the topological centrality index; C i These represent the degree centrality, the betweenness centrality, the proximity centrality, and the clustering coefficient, respectively. w i These are preset weighting coefficients; This represents the smallest positive constant that avoids taking the logarithm of zero. S214. Based on the power grid graph theory model, the edges are transformed into adjacency state vectors of each device node, and the topological centrality index and the adjacency state vectors are used as static features of each device node. In other words, by calculating degree centrality, betweenness centrality, proximity centrality, and clustering coefficient, the status of a node in the power grid topology can be quantified; by forming a topology centrality index through a fusion strategy, the combined effect of each index on the node can be reasonably reflected; and by expressing the connection relationship of nodes through adjacency state vectors, the local features of the power grid structure can be transformed into a learnable numerical form. Step S22 specifically includes: The active power, reactive power, current and voltage in the dynamic load measurement data are converted into a time series matrix based on a time segment. Each row of the time series matrix corresponds to an independent time segment, and each column of the time series matrix corresponds to the active power, reactive power, current and voltage of the equipment node in the time segment. Step S23 specifically includes: S231. Using the time cross-section as a reference, the static features and the time series matrix are spliced ​​together to form a preliminary feature matrix; S232. Calculate the empirical accumulation distribution function of each column of the preliminary feature matrix using quantile transformation; S233. The preliminary feature matrix is ​​transformed into a feature matrix based on topological quantile characteristic coefficients using the empirical accumulation distribution function.

[0019] The time attributes and numerical scale of the running data are normalized by using time series matrices and quantile transformations.

[0020] In some implementations, step S3 includes: S31. Stack a multi-head DMSA layer, a residual connection layer, a normalization layer, a feedforward neural network, a residual connection layer, and a normalization layer in sequence to form a DMSA module; S32. The two DMSA modules are spliced ​​together to form an encoder, and the two DMSA modules are combined to form a decoder that matches the structure of the encoder; S33. Insert a sampling layer between the encoder and the decoder to form a Maha-DMVAE model including the Mahalanobis distance metric module; S34. Using the feature matrix as input, train the Maha-DMVAE model with the goal of minimizing the preset loss function.

[0021] As described above, by constructing an encoder and decoder containing multiple DMSA modules in the Maha-DMVAE model and inserting a sampling layer between them, the model gains the ability to extract internal correlations within feature sequences and establish a latent space probability distribution. Its working principle is as follows: the DMSA module forces the model to learn contextual information from other time steps by ignoring the attention of the time step itself; the encoder, through the stacking of multiple modules, can extract deeper temporal structures; the decoder maps the original data by reconstructing latent vectors; and the sampling layer forms a learnable Gaussian distribution in the latent space through reparameterization. This step effectively improves the model's generalization ability in outlier repair, enabling the model to not only reconstruct normal samples but also provide reasonable estimates for missing or outlier samples.

[0022] In some implementations, step S4 includes: S41. Transform the feature matrix into an initial dataset, input the initial dataset into the trained Maha-DMVAE model, and output the latent space mean vector and its set. S42. Based on the latent space mean vector, calculate the mean vector and covariance matrix; S43. Calculate the Mahalanobis distance of the corresponding latent space mean vector using the mean vector and the covariance matrix, and mark the Mahalanobis distance as the outlier score, as follows:

[0023] In the formula, Z mean Represents the latent space mean vector;D M Indicates abnormal scores; ∑ represents the mean vector; ∑ represents the covariance matrix; K Indicates the number of iterations; S44. Sort the abnormal scores in descending order, select a preset number of the abnormal scores with the highest ranking, and mark them as an abnormal set. S45. Based on the initial dataset and the anomaly set, generate a mask matrix including the anomaly set, input the mask matrix and the initial dataset into the trained Maha-DMVAE model, and output a correction matrix.

[0024] As described above, an iterative cleaning process for latent space is formed by calculating the mean vector and covariance matrix in the latent space, using Mahalanobis distance to measure anomalies in the latent space mean vector, and combining sorting and masking mechanisms for correction. The principle is as follows: the mean vector in the latent space is more likely to form a compact distribution; the covariance matrix can reflect the correlation between different latent variables; Mahalanobis distance is particularly suitable for anomaly quantification of datasets with covariance correlation; by labeling the anomaly set and inputting it into the model for correction, the deviation between anomalies and the normal distribution can be reduced round by round, thus making anomaly identification independent of subjective thresholds and automatically determined by statistical distribution; the correction process is naturally compatible with the model structure and can be optimized in multiple rounds; ultimately, a stable latent space distribution is formed, effectively repairing anomalies.

[0025] In some implementations, step S45 specifically includes: S451. Based on the initial dataset, generate a mask matrix with the same dimensions, and mark all samples of the anomaly set as to be corrected in the mask matrix; S452. Input the initial dataset and the mask matrix into the trained Maha-DMVAE model, perform forward propagation, and output the correction matrix; S453. Replace the feature matrix with the correction matrix and return to step S41 for iterative calculation until a preset number of iterations is reached or the difference in the anomaly set obtained in any adjacent iteration calculation is less than a preset threshold. The correction matrix generated by the last iteration calculation is regarded as the final matrix.

[0026] As described above, by constructing a mask matrix, combining it with the correction matrix output by the Maha-DMVAE model, and performing multiple iterations, the correction process achieves continuity and convergence. The principle is as follows: the mask matrix clearly tells the model which locations are regions to be corrected, giving the correction action directionality; the forward propagation process generates correction values ​​consistent with the overall data based on the latent space vector; by replacing the original feature matrix with the correction matrix and repeating the calculation, the latent space distribution is continuously updated, gradually stabilizing the anomaly identification results. Successive rounds of correction avoid the accumulation of biases caused by a single correction; the anomaly set difference judgment mechanism effectively terminates unnecessary iterations, improving efficiency; and the final matrix formed is more consistent in statistical distribution and operational patterns.

[0027] In some implementations, step S5 includes: S51. Convert the final matrix into a normal dataset; S52. Construct a benchmark test set and inject a preset anomaly into the benchmark test set; compare the normal dataset and the benchmark test set, and calculate the repair confidence fusion index; S53. Construct a node admittance matrix based on the topology data; calculate the power flow residual using the AC power flow equation based on the node admittance matrix and the normal dataset, and define the physical constraint compliance degree based on the power flow residual; S54. The validity of the final matrix is ​​evaluated based on the repair confidence fusion index and physical constraint compliance. If the final matrix passes the validity evaluation, the Maha-DMVAE model is considered valid, and the Maha-DMVAE model is used to clean the data related to the power grid operation mode.

[0028] As described above, by constructing a benchmark test set based on a normal dataset and combining the repair confidence fusion index and physical constraint compliance to evaluate the final matrix, the evaluation criteria for the cleaning results become more comprehensive. The principle is as follows: by injecting preset anomalies into the benchmark test set, the model's performance in identifying and repairing anomalies can be quantitatively measured; the node admittance matrix and power flow equations can verify whether the physical relationships of the power grid remain consistent. In other words, dual-mode evaluation can avoid the problem of the model only exhibiting high reconfiguration capability but with large physical deviations, thus improving the credibility of the data in engineering scenarios.

[0029] In some implementations, step S52 specifically includes: S521. Construct a benchmark test set and inject a preset exception into the benchmark test set; S522. Compare the normal dataset and the benchmark test set to identify the precision, recall, and F1 score of the preset anomaly, and use a weighted harmonic average method to fuse the precision, recall, and F1 score into a repair confidence fusion index, as follows:

[0030] In the formula, RCFI This indicates the need to repair the confidence level fusion index; P Indicates accuracy; R Indicates recall rate; F1 Represents F1 score ； These are the preset weighting coefficients for accuracy, recall, and F1 score, respectively.

[0031] As can be seen from the above description, different types of data anomalies have different impacts on power grid analysis. Weighted fusion of the three types of indicators can make the evaluation results more comprehensive and applicable. Weighted harmonic average can enhance the consistency between indicators, so that the results are not distorted due to the low value of a single indicator, and the repair effect can be intuitively quantified, providing a reference for model iteration and parameter adjustment. This indicator can also be used to monitor the performance changes of the model in different time periods and improve the efficiency of model maintenance.

[0032] In some implementations, step S53 specifically includes: S531. Extract the reported active power, reported reactive power and reported voltage vector of all device nodes at any time segment in the normal dataset; S532. Construct a node admittance matrix based on the topology data; based on the node admittance matrix and the reported voltage vector, calculate the required active power and reactive power of all the device nodes according to the AC power flow equation, and calculate the power flow residual, as shown in the following formula:

[0033] In the formula, This represents the power flow residual of device node i; Indicates the reported active power; This indicates that there should be active power; This indicates the reported reactive power; This indicates that there should be reactive power; S533. The average value of the power flow residuals of all device nodes at all times in the normal dataset is defined as the physical constraint compliance degree, as shown in the following formula:

[0034] In the formula, PCCThe degree of compliance with physical constraints is represented by T, which represents the total number of time segments in the normal dataset; and N represents the total number of device nodes.

[0035] As described above, by constructing a node admittance matrix based on topology data and calculating the required power using AC power flow equations, power flow residuals are obtained, and physical constraint compliance is calculated, thus ensuring that the repair results are constrained at the level of power system physical laws. The node admittance matrix has a definite mathematical relationship with voltage, current, and power; the power flow residuals directly reflect the consistency between operating data and the actual power grid model; and the physical constraint compliance, by averaging all power flow residuals, can comprehensively measure the degree to which the data conforms to the laws of the power grid.

[0036] Please refer to Figure 4 A terminal 1 includes a memory 2, a processor 3, and a computer program stored in the memory 2 and running on the processor 3. When the processor 3 executes the computer program, it implements the steps in a Maha-DMVAE-based power grid operation mode data iterative cleaning method.

[0037] Please refer to Figure 1 Embodiment 1 of the present invention is as follows: An iterative data cleaning method for power grid operation modes based on Maha-DMVAE includes the following steps: S1. Collect topology data and dynamic load measurement data related to power grid operation mode; The topology data and dynamic load measurement data related to power grid operation can come from data source systems. The acquisition process involves: identifying and connecting to core systems that can provide historical power grid operation data, mainly including the Power Grid Energy Management System (EMS), Production Control Cloud (Dispatch Cloud), and Marketing PIMS system. A stable and reliable data acquisition channel is established through methods such as direct database connection, API interface calls, or batch file export. The topology data includes at least device ID, device parameters, and device connection relationships; the dynamic load measurement data includes at least active power, reactive power, current, and voltage bound to the device ID. At the same time, the topology data and dynamic load measurement data are precisely aligned based on the device ID and timestamp to establish the association between the two types of data, ensuring data consistency and traceability.

[0038] S2. Based on the topology data and the dynamic load measurement data, construct a feature matrix based on the topology quantile characteristic coefficients; S3. Design and train a Maha-DMVAE model including a Mahalanobis distance metric module using the feature matrix; S4. With the goal of adjusting the Mahalanobis distance of the latent space, and using the feature matrix as input, the final matrix is ​​output using the Maha-DMVAE model. S5. Based on the power flow residual confidence dual-mode evaluation system, evaluate the validity of the final matrix. If the final matrix passes the validity evaluation, the Maha-DMVAE model is considered valid, and the Maha-DMVAE model is used to clean the data related to the power grid operation mode.

[0039] Specifically, by unifying static topology information and dynamic measurement information of the power grid through feature matrices, the model can learn different types of feature associations under the same data structure; by reconstructing the internal structure of the data through the Maha-DMVAE model, the model can establish data distribution in the latent space; by using an iterative correction strategy based on Mahalanobis distance, the deviation of outlier samples in the latent space can be weakened round by round; and by using dual verification of power flow residuals and confidence indices, the cleaning results are ensured to be consistent in both statistical characteristics and physical constraints, thereby effectively improving the usability of power grid operation mode data and enabling subsequent power flow calculations, state estimations, and other processes to be built on a more stable data foundation.

[0040] Step S2 includes: S21. Instantiate the topology data and extract static features; Step S21 specifically includes: S211. Convert the device ID and device parameters in the topology data into device nodes V, convert the device connection relationship into edges E, and instantiate the device nodes and edges into graph objects G=(V,E). S212. For each device node in the graph object, calculate the degree centrality, betweenness centrality, proximity centrality, and clustering coefficient. S213. Using a log-geometric mean fusion strategy, the degree centrality, betweenness centrality, proximity centrality, and clustering coefficients are nonlinearly fused into a topological centrality index, as shown in the following formula:

[0041] In the formula, TCI Represents the topological centrality index; C i These represent the degree centrality, the betweenness centrality, the proximity centrality, and the clustering coefficient, respectively. w i These are preset weighting coefficients; This represents the smallest positive constant that avoids taking the logarithm of zero. S214. Based on the power grid graph theory model, the edges are transformed into adjacency state vectors of each device node, and the topological centrality index and the adjacency state vectors are used as static features of each device node. In other words, by calculating degree centrality, betweenness centrality, proximity centrality, and clustering coefficient, the status of a node in the power grid topology can be quantified; by forming a topology centrality index through a fusion strategy, the combined effect of each index on the node can be reasonably reflected; and by expressing the connection relationship of nodes through adjacency state vectors, the local features of the power grid structure can be transformed into a learnable numerical form. S22. Convert dynamic load measurement data into a time series matrix based on time sections; Step S22 specifically includes: The active power, reactive power, current and voltage in the dynamic load measurement data are converted into a time series matrix based on a time segment. Each row of the time series matrix corresponds to an independent time segment, and each column of the time series matrix corresponds to the active power, reactive power, current and voltage of the equipment node in the time segment. S23. The static features are concatenated with the time series matrix to form a preliminary feature matrix, and the preliminary feature matrix is ​​subjected to quantile transformation to form a feature matrix based on topological quantile feature coefficients.

[0042] Step S23 specifically includes: S231. Using the time cross-section as a reference, the static features and the time series matrix are spliced ​​together to form a preliminary feature matrix; S232. Calculate the empirical cumulative distribution function of each column of the preliminary feature matrix using quantile transformation; for each feature in the matrix, the quantile transformer calculates its empirical cumulative distribution function based on all data points of that feature, sorts each data point under that feature, and determines its quantile position in the overall distribution of that feature.

[0043] S233. The preliminary feature matrix is ​​transformed into a feature matrix based on topological quantile characteristic coefficients using the empirical accumulation distribution function.

[0044] The time attributes and numerical scale of the running data are normalized by using time series matrices and quantile transformations.

[0045] In other words, after instantiation, topological data can be used to extract static features, which clearly expresses the structural relationships between nodes; dynamic load data, after time-section reshaping, forms a time-series matrix, enabling the model to learn changes in operating status over time; the concatenation of static and dynamic features ensures that each time section has a complete expression of node characteristics; quantile transformation unifies the numerical differences of different feature dimensions, thereby improving the stability of model training. Thus, through a unified data representation method, the model can simultaneously learn node location attributes, connectivity relationships, and operating status, making subsequent anomaly identification and correction more targeted and accurate.

[0046] Please refer to Figure 2 Step S3 includes: S31. Stack a multi-head DMSA layer, a residual connection layer, a normalization layer, a feedforward neural network, a residual connection layer, and a normalization layer in sequence to form a DMSA module; S32. The two DMSA modules are concatenated to form an encoder, and the two DMSA modules are combined to form a decoder that matches the structure of the encoder; the encoder also includes an output layer, which consists of two independent fully connected layers. These two fully connected layers do not share weights and are used to map the high-dimensional features extracted by the DMSA modules to generate the mean vector and log-variance vector of the latent space, respectively. S33. Insert a sampling layer between the encoder and the decoder to form a Maha-DMVAE model including the Mahalanobis distance metric module; The mean vector and log-variance vector of the encoder output are used to define a Gaussian probability distribution:

[0047] in, Indicates standard deviation, The mean of the Gaussian distribution can be represented by the above mean vector, which can be directly equal to the mean of the Gaussian distribution.

[0048] in, Represents the logarithmic variance vector; To generate the latent vector z, a random noise vector with the same dimension as z is first sampled from a standard normal distribution. Then, the latent vector z is generated as follows:

[0049] S34. Using the feature matrix as input, train the Maha-DMVAE model with the objective of minimizing the preset loss function. The preset loss function consists of three weighted parts, as shown in the following formula:

[0050] The specific calculation steps are as follows: (1) Calculate the mask interpolation loss From the known input observations, a subset is randomly selected and temporarily masked. The mean squared error (MSE) between the model's predictions for these temporarily masked locations and their true values ​​is calculated. The calculated MSE value is defined as the masking interpolation loss. ; (2) Calculate the observation reconstruction loss With KL divergence loss The mean squared error (MSE) between the reconstructed data output by the model and the true observations of the input data at the original, unmasked locations is calculated, and this value is defined as the observation reconstruction loss. The KL divergence between the latent space distribution of the encoder output and the standard normal distribution is calculated, and this value is defined as the KL divergence loss. .

[0051] In the standard self-attention mechanism, an attention score matrix is ​​calculated. All elements on the diagonal of this matrix are set to a very large negative number (-∞). Through a diagonal mask, the attention weights of each time step during information reconstruction are made close to 0 after passing through a softmax function.

[0052] Furthermore, the feedforward neural network consists of two fully connected layers. The first layer expands the feature dimension from the model's internal working dimension to a higher dimension and uses the ReLU activation function. The second fully connected layer then maps it back to the original working dimension to achieve a non-linear transformation of the features.

[0053] The Maha-DMVAE model constructs an encoder and decoder containing multiple DMSA modules, with a sampling layer inserted between them. This enables the model to both extract internal correlations within feature sequences and establish a latent space probability distribution. Its working principle is as follows: the DMSA modules, by ignoring the attention of each time step itself, force the model to learn contextual information from other time steps; the encoder, through the stacking of multiple modules, can extract deeper temporal structures; the decoder maps the original data through the reconstruction of latent vectors; and the sampling layer, through reparameterization, forms a learnable Gaussian distribution in the latent space. This step effectively improves the model's generalization ability in outlier repair, enabling the model to not only reconstruct normal samples but also provide reasonable estimates for missing or outlier samples.

[0054] Please refer to Figure 3 Step S4 includes: S41. Transform the feature matrix into an initial dataset, input the initial dataset into the trained Maha-DMVAE model, and output the latent space mean vector and its set. S42. Based on the latent space mean vector, calculate the mean vector and covariance matrix; S43. Calculate the Mahalanobis distance of the corresponding latent space mean vector using the mean vector and the covariance matrix, and mark the Mahalanobis distance as the outlier score, as follows:

[0055] In the formula, Z mean Represents the latent space mean vector; D M Indicates abnormal scores; ∑ represents the mean vector; ∑ represents the covariance matrix; K Indicates the number of iterations; S44. Sort the abnormal scores in descending order, and select a preset number of the abnormal scores that rank highest, marking them as an abnormal set; specifically, the preset number is a proportion, such as the top 2%; S45. Based on the initial dataset and the anomaly set, generate a mask matrix including the anomaly set, input the mask matrix and the initial dataset into the trained Maha-DMVAE model, and output a correction matrix.

[0056] Step S45 specifically includes: S451. Based on the initial dataset, generate a mask matrix with the same dimensions, and mark all samples of the anomaly set as to be corrected in the mask matrix; that is, in the mask matrix, the mask value to be corrected is 0, and the other positions are 1; S452. Input the initial dataset and the mask matrix into the trained Maha-DMVAE model, perform forward propagation, and output the correction matrix; S453. Replace the feature matrix with the correction matrix, and return to step S41 for iterative calculation until a preset number of iterations is reached or the difference in the anomaly set obtained in any adjacent iterations is less than a preset threshold. The correction matrix generated by the last iteration is regarded as the final matrix. Specifically, the preset number of iterations is 5.

[0057] By constructing a mask matrix, combining it with the correction matrix output by the Maha-DMVAE model, and performing multiple iterations, the correction process achieves continuity and convergence. The principle is as follows: the mask matrix clearly tells the model which locations are regions to be corrected, giving the correction action directionality; the forward propagation process generates correction values ​​consistent with the overall data based on the latent space vector; by replacing the original feature matrix with the correction matrix and repeating the calculation, the latent space distribution is continuously updated, gradually stabilizing the anomaly identification results. Successive rounds of correction avoid the accumulation of biases caused by a single correction; the anomaly set difference judgment mechanism effectively terminates unnecessary iterations, improving efficiency; and the final matrix is ​​more consistent in statistical distribution and operational patterns.

[0058] In step S4, an iterative cleaning process oriented towards the latent space is formed by calculating the mean vector and covariance matrix in the latent space, using Mahalanobis distance to measure anomalies in the latent space mean vector, and combining sorting and masking mechanisms for correction. The principle is as follows: the mean vector in the latent space is more likely to form a compact distribution; the covariance matrix can reflect the correlation between different latent variables; Mahalanobis distance is particularly suitable for anomaly quantification of datasets with covariance correlation; by labeling the anomaly set and inputting it into the model for correction, the deviation between anomalies and the normal distribution can be reduced round by round, thus making anomaly identification independent of subjective thresholds and automatically determined by statistical distribution; the correction process is naturally compatible with the model structure and can be optimized in multiple rounds; finally, a stable latent space distribution is formed, effectively repairing anomalies.

[0059] Step S5 includes: S51. Convert the final matrix into a normal dataset; S52. Construct a benchmark test set and inject preset anomalies (such as spikes, drifts, missing values, etc.) into the benchmark test set; compare the normal dataset and the benchmark test set, and calculate the repair confidence fusion index. Step S52 specifically includes: S521. Construct a benchmark test set and inject a preset exception into the benchmark test set; S522. Compare the normal dataset and the benchmark test set to identify the precision, recall, and F1 score of the preset anomaly, and use a weighted harmonic average method to fuse the precision, recall, and F1 score into a repair confidence fusion index, as follows:

[0060] In the formula, RCFI This indicates the need to repair the confidence level fusion index; P Indicates accuracy; R Indicates recall rate; F1Represents F1 score ； These are the preset weighting coefficients for accuracy, recall, and F1 score, respectively.

[0061] S53. Construct a node admittance matrix based on the topology data; calculate the power flow residual using the AC power flow equation based on the node admittance matrix and the normal dataset, and define the physical constraint compliance degree based on the power flow residual; Step S53 specifically includes: S531. Extract the reported active power, reported reactive power and reported voltage vector of all device nodes at any time segment in the normal dataset; S532. Construct a node admittance matrix based on the topology data; based on the node admittance matrix and the reported voltage vector, calculate the required active power and reactive power of all the device nodes according to the AC power flow equation, and calculate the power flow residual, as shown in the following formula:

[0062] In the formula, This represents the power flow residual of device node i; Indicates the reported active power; This indicates that there should be active power; This indicates the reported reactive power; This indicates that there should be reactive power; S533. The average value of the power flow residuals of all device nodes at all times in the normal dataset is defined as the physical constraint compliance degree, as shown in the following formula:

[0063] In the formula, PCC The degree of compliance with physical constraints is represented by T, which represents the total number of time segments in the normal dataset; and N represents the total number of device nodes.

[0064] S54. The validity of the final matrix is ​​evaluated based on the repair confidence fusion index and physical constraint compliance. If the final matrix passes the validity evaluation, the Maha-DMVAE model is considered valid, and the Maha-DMVAE model is used to clean the data related to the power grid operation mode.

[0065] In step S5, a benchmark test set is constructed based on a normal dataset, and the final matrix is ​​evaluated by combining the repair confidence fusion index and physical constraint compliance, making the evaluation criteria for the cleaning results more comprehensive. The principle is as follows: by injecting preset anomalies into the benchmark test set, the model's performance in identifying and repairing anomalies can be quantitatively measured; the node admittance matrix and power flow equations can verify whether the physical relationships of the power grid remain consistent. In other words, dual-mode evaluation can avoid the problem of the model only exhibiting high reconfiguration capability but with large physical deviations, thus improving the credibility of the data in engineering scenarios.

[0066] Specifically, it also includes step S6: S6.1: Periodically collect the normal operation data accumulated during this period and verified by humans, and combine it with the original normal dataset to achieve incremental learning and retraining; S6.2, Set the update strategy. Based on factors such as changes in power grid structure and seasonal load changes, set the model update cycle to be every quarter and every six months. S6.3 completely re-executes the entire iterative cleaning and training process to generate a new generation of data anomaly detection models that are better adapted to the current power grid operation characteristics, and replaces the old model that is running online. S6.4 Use automated scripts and statistical analysis tools to check the consistency and accuracy of the cleaned data, ensuring that all data meets the set conditions.

[0067] Please refer to Figure 4 Embodiment two of the present invention is as follows: A terminal 1 includes a memory 2, a processor 3, and a computer program stored in the memory 2 and running on the processor 3. When the processor 3 executes the computer program, it implements the steps in a Maha-DMVAE-based power grid operation mode data iterative cleaning method.

[0068] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for iterative cleaning of power grid operating mode data based on Maha-DMVAE, characterized in that, The method comprises the steps of: S1, collecting topological structure data and dynamic load measurement data related to power grid operation mode; S2, constructing a feature matrix based on topological quantile characteristic coefficients based on the topological structure data and the dynamic load measurement data; S3, using the feature matrix to design and train a Maha-DMVAE model comprising a Mahalanobis distance measurement module; S4, taking the Mahalanobis distance of the latent space as the target, taking the feature matrix as the input, and using the Maha-DMVAE model to output a final matrix; S5, based on the confidence double-mode evaluation system of power flow residual value, evaluating the effectiveness of the final matrix, if the final matrix passes the effectiveness evaluation, considering that the Maha-DMVAE model is effective, and using the Maha-DMVAE model to clean the data related to power grid operation mode.

2. The Maha-DMVAE-based power grid operating mode data iterative cleaning method according to claim 1, characterized in that, The step S2 comprises: S21, instantiating the topological structure data and extracting static features; S22, converting the dynamic load measurement data into a time series matrix based on time sections; S23, splicing the static features and the time series matrix to form a preliminary feature matrix, and performing quantile transformation on the preliminary feature matrix to form a feature matrix based on topological quantile characteristic coefficients.

3. The Maha-DMVAE-based power grid operating mode data iterative cleaning method according to claim 2, characterized in that, The topological structure data at least includes device ID, device parameters and device connection relationship; the dynamic load measurement data at least includes active power, reactive power, current and voltage bound with device ID; The step S21 specifically comprises: S211, converting the device ID and the device parameters in the topological structure data into device nodes, converting the device connection relationship into edges, and instantiating the device nodes and the edges into graph objects; S212, calculating the degree centrality, betweenness centrality, closeness centrality and clustering coefficient of each device node in the graph object; S213, using a log-geometric average fusion strategy to nonlinearly fuse the degree centrality, betweenness centrality, closeness centrality and clustering coefficient into a topological centrality index, the formula is as follows: In the formula, TCI denotes a topological centrality index; C i denote the degree centrality, the betweenness centrality, the closeness centrality and the clustering coefficient, respectively; w i is a preset weight coefficient; denotes a small positive number that avoids taking the logarithm of zero. S214, based on the power grid graph theory model, converting the edges into the adjacency state vector of each device node, and taking the topological centrality index and the adjacency state vector as the static features of each device node; The step S22 specifically comprises: Converting the active power, reactive power, current and voltage in the dynamic load measurement data into a time series matrix based on time sections, each row of the time series matrix corresponds to an independent time section, and each column of the time series matrix corresponds to the active power, reactive power, current and voltage of the device node of the time section; The step S23 specifically comprises: S231, splicing the static features and the time series matrix based on the time sections to form a preliminary feature matrix; S232, using quantile transformation to calculate the empirical cumulative distribution function of each column of the preliminary feature matrix; S233, transform the preliminary feature matrix into a feature matrix based on topological quantile feature coefficients using the empirical cumulative distribution function.

4. The Maha-DMVAE-based power grid operating mode data iterative cleaning method according to claim 1, characterized in that, The step S3 comprises: S31, stack a multi-head DMSA layer, a residual connection layer, a normalization layer, a feedforward neural network, a residual connection layer and a normalization layer in sequence to form a DMSA module; S32, splice two DMSA modules to form an encoder, and parallel two DMSA modules to form a decoder matching the structure of the encoder; S33, insert a sampling layer between the encoder and the decoder to constitute a Maha-DMVAE model comprising a Mahalanobis distance measurement module; S34, train the Maha-DMVAE model by taking the feature matrix as input and minimizing a preset loss function.

5. The method of claim 4, wherein the Maha-DMVAE based power grid operating mode data iterative cleaning method is characterized by, The step S4 comprises: S41, convert the feature matrix into an initial data set, input the initial data set into the trained Maha-DMVAE model, and output a latent space mean vector and a set thereof; S42, calculate a mean vector and a covariance matrix based on the latent space mean vector; S43, calculate the Mahalanobis distance of the corresponding latent space mean vector using the mean vector and the covariance matrix, and mark the Mahalanobis distance as an anomaly score, as follows: wherein Z mean denotes the latent space mean vector; D M denotes the anomaly score; denotes the mean vector;∑ denotes the covariance matrix; K denotes the number of iterations; S44, sort the anomaly scores in descending order, select a preset number of anomaly scores ranked at the top, and mark them as an anomaly set; S45, generate a mask matrix comprising the anomaly set based on the initial data set and the anomaly set, and input the mask matrix and the initial data set into the trained Maha-DMVAE model, and output a correction matrix.

6. The method of claim 5, wherein the method is based on Maha-DMVAE. The step S45 specifically comprises: S451, generate a mask matrix with the same dimension as the initial data set, and mark all samples in the anomaly set as to be corrected in the mask matrix; S452, input the initial data set and the mask matrix into the trained Maha-DMVAE model, and perform forward propagation to output a correction matrix; S453, replace the feature matrix with the correction matrix, and return to step S41 for iterative calculation until a preset number of iterations is reached or the difference between any adjacent iterations is less than a preset threshold, and the correction matrix generated in the last iteration is regarded as the final matrix.

7. The method of claim 6, wherein the method is based on Maha-DMVAE. The step S5 comprises: S51, convert the final matrix into a normal data set; S52, construct a benchmark test set and inject a preset anomaly into the benchmark test set, compare the normal data set and the benchmark test set, and calculate a repair confidence fusion index; S53, construct a node admittance matrix according to the topological structure data, calculate a power flow residual using an alternating current power flow equation based on the node admittance matrix and the normal data set, and define a physical constraint compliance degree according to the power flow residual; S54, evaluate the effectiveness of the final matrix according to the repair confidence fusion index and the physical constraint compliance degree, if the final matrix passes the effectiveness evaluation, consider that the Maha-DMVAE model is effective, and use the Maha-DMVAE model to clean the data related to power grid operation mode.

8. The method of claim 7, wherein the method is based on Maha-DMVAE. The step S52 specifically comprises: S521, construct a benchmark test set, and inject a preset anomaly in the benchmark test set; S522, compare the normal data set and the benchmark test set, identify the accuracy, recall rate and F1 score of the preset anomaly, and fuse the accuracy, recall rate and F1 score into a repair confidence fusion index by using a weighted harmonic mean method, and the formula is as follows: In the formula, RCFI represents a repair confidence fusion index; P represents accuracy; R represents recall; F1 represents F1 score ； are preset weight coefficients of accuracy, recall, and F1 score, respectively.

9. The method of claim 7, wherein the method is based on Maha-DMVAE. The step S53 specifically comprises: S531, extract the reported active power, reported reactive power and reported voltage vector of all device nodes in any time section in the normal data set; S532, construct a node admittance matrix according to the topological structure data; based on the node admittance matrix and the reported voltage vector, calculate the required active power and required reactive power of all the device nodes according to the alternating current flow equation, and calculate the flow residual error, and the formula is as follows: In the formula, represents the power flow residual of the device node i; represents the reported active power; represents the required active power; represents the reported reactive power; represents the required reactive power; S533, define the average value of the flow residual error of all the device nodes at all times in the normal data set as the physical constraint compliance degree, and the formula is as follows: wherein PCC represents the physical constraint compliance degree, T represents the total number of time sections of the normal data set; N represents the total number of device nodes.

10. A terminal comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to realize the steps in the power grid operation mode data iterative cleaning method based on Maha-DMVAE in any one of claims 1-9.

Images