Power load data missing value filling model based on MDAN and construction method thereof

Abstract

The application relates to an MDAN-based power load data missing value filling model and a construction method thereof. The filling model comprises a time-time relationship capturing block Module 1, a time-feature relationship capturing block Module 2 and a dynamic weighted fusion block. Module 1 can obtain the correlation between the timestamp dimensions, Module 2 can obtain the correlation between the feature dimensions and the multi-dimensional correlation between time and features, the dynamic weighted fusion module gives a learnable weight to the obtained feature representation, then performs fusion to obtain the final feature representation, then replaces the values of the non-missing positions, and finally obtains the final filling value. The dynamic joint optimization method is used to adjust the importance of the sub-tasks, so that the generalization capability of the model is improved. The model can fill the missing readings of the power load data and achieve a more accurate filling effect, and opens up a new way for the subsequent load data missing value filling task.

Description

Technical Field

[0001] This invention relates to the field of power load data missing value processing technology, specifically to a power load data missing value filling model based on MDAN and its construction method. Background Technology

[0002] Load data refers to the total electrical power drawn by electrical equipment from the power system at a given moment. When imputing missing values, relying solely on historical load data is insufficient for high-precision filling. Therefore, load data and meteorological data (including temperature and humidity) that significantly influence it are defined as power load data. It is multidimensional time-series data, possessing temporal, sequential, and seasonally varying periodicity. Over years of research and development, methods for handling missing values ​​have evolved from simple deletion to methods that use existing data to impute missing values. Imputation methods include mean replacement (MEAN), previous value replacement, linear regression (LR), multi-layer perceptron (MLP), K-nearest neighbors (KNN), and recurrent neural networks (RNN). In the era of big data, data quality profoundly impacts decision-making and scientific development; its importance is self-evident. Coupled with its influence on downstream tasks, missing data imputation remains a field of ongoing exploration for many researchers.

[0003] Early methods for handling missing data involved direct deletion, which deleted the entire record containing the missing data. This not only resulted in the loss of a large amount of information but also damaged the structure of the data, leading to a reduction in usable data. In datasets with high missing data rates, effective analysis and processing were even impossible.

[0004] Most methods use different data to fill missing values. For example, the mean replacement method uses the average of all observations of the attribute to fill the missing values, while the previous value replacement method uses data from a time period before the missing value. Both of these methods ignore the variance of the data and the correlation between the various attributes, so the filling results are not accurate.

[0005] Linear regression models are feedforward neural networks consisting of a single fully connected layer, while multilayer perceptrons build upon linear regression by adding activation functions and more fully connected layers. Compared to mean substitution, both methods preserve the variance and covariance of the data. However, all imputed values ​​follow a single regression curve, considering only single-feature time-series information and ignoring the structure of the data matrix; therefore, they cannot represent the inherent variations within the data.

[0006] K-Nearest Neighbors (KNN) is a machine learning model that does not have parameters to be learned. For data with missing values, KNN calculates the "distance" between the target data and the known data according to a certain distance metric algorithm (such as Euclidean distance), and selects the K closest samples to predict the missing values. Its disadvantage is that for each sample to be estimated, all known samples need to be traversed once. When the amount of missing data is large, it will be very time-consuming. In addition, the performance of the model is highly dependent on the distance algorithm.

[0007] Recurrent Neural Networks (RNNs) can handle temporal problems because they have a "memory" capability, enabling them to effectively handle the relationships between temporal features. However, RNNs also have significant drawbacks. On one hand, the forward propagation of an RNN is sequential, making it difficult to utilize the parallel capabilities of a GPU. This results in a significant time consumption and low computational efficiency when processing long-sequence data. On the other hand, RNNs struggle to address long-range dependencies, making it difficult to extract information corresponding to all timestamps when dealing with long-sequence data.

[0008] MDAN is a deep learning model based on an attention mechanism. First, the self-attention mechanism effectively uncovers potential interrelationships within data dimensions. When dealing with multi-feature electricity load data, the multi-dimensional attention mechanism can extract internal connections across multiple dimensions, resulting in better prediction performance. Second, the attention mechanism effectively addresses long-range dependency issues, paying attention to information from all timestamps, enabling connections to be established even between distant timestamps. Finally, the attention mechanism effectively utilizes the parallel processing capabilities of GPUs, reducing time consumption when processing long-series data.

[0009] In summary, the Attention mechanism is well-suited for handling the task of filling in missing values ​​in power load data.

[0010] With the continuous optimization of the power system and the ongoing upgrading of smart grid construction, power companies are increasingly demanding higher quality load data. However, due to reasons such as equipment downtime, damage, communication errors, and unexpected interruptions (e.g., power outages), the data obtained by sensors often contains missing values. Missing data not only affects real-time numerical monitoring but also interferes with data analysis and load forecasting. Complete and usable load data is crucial for power companies to formulate reasonable plans and scheduling schemes, improve equipment utilization, and ensure the safe and stable operation of the system. As an important branch of load data preprocessing, load missing value imputation has significant research and application value and has attracted widespread attention in the field of load data mining. Traditional data analysis methods (such as the MEAN method and the pre-replacement method) cannot meet the challenges posed by large-scale load data and are inefficient. The use of big data analytics is beneficial for in-depth mining of this data to extract patterns and regularities in load data changes and apply them to the task of imputing missing data to achieve better results. Among big data-based methods, LR and MLP only consider single-feature time series information and ignore the structure of the data matrix; KNN needs to compare all known data when predicting each missing reading, which is very time-consuming and the algorithm is highly dependent on distance metric methods; although RNN can perform time series prediction tasks well, its running efficiency is low and it cannot solve long-range dependency problems. Summary of the Invention

[0011] The purpose of this invention is to provide a missing value filling model for power load data based on MDAN and its construction method, so as to solve the problem of low filling accuracy of existing missing value filling models for power load data.

[0012] To achieve the above objectives, the present invention adopts the following technical solution:

[0013] A method for constructing a missing value imputation model for power load data based on MDAN includes the following steps:

[0014] A Data fusion layer is constructed, which is used to concatenate multidimensional load data and a mask matrix to obtain the input matrix of the model;

[0015] An input layer is constructed, which includes a multivariate time series position encoding and a fully connected linear layer; the multivariate time series position encoding is used to add position information to the time series, and the fully connected linear layer is used to map the input data to a dense vector;

[0016] Construct Module1, which includes a multi-head diagonal mask self-attention module and a linear layer. The multi-head diagonal mask self-attention module is used to predict Z1 from the time dimension as V for the multi-head diagonal mask multi-dimensional attention of Module2. The linear layer is used to obtain the first predicted value.

[0017] Construct Module2, which includes a multi-headed diagonal mask multi-dimensional attention module and a linear layer; the multi-headed diagonal mask multi-dimensional attention module is used to obtain the potential relationship between time and features to generate feature representation Z2 and attention score A, and the linear layer is used to obtain a second prediction value;

[0018] A weighted fusion layer and a replacement layer are constructed. The weighted fusion layer is used to dynamically weight and fuse the intra-temporal prediction values ​​obtained from Module 1 with the feature-temporal multi-dimensional prediction values ​​obtained from Module 2 to obtain the final feature representation Z3. The replacement layer is used to fill the feature representation with missing positions and fill the non-missing positions with the original data to obtain the final value.

[0019] Furthermore, the Data fusion layer is used to stitch together multidimensional load data, relevant meteorological data, and a mask matrix to obtain the input matrix of the model.

[0020] Furthermore, the Module1 layer includes a multi-head diagonal mask self-attention module, a residual connection layer, a feedforward layer, and a linear layer; the multi-head diagonal mask self-attention module includes: the input data has not undergone transpose operation, and the attention mechanism can capture the potential connections within the first dimension, while the diagonal mask prevents the data itself from contributing to its estimation, and then multiple attention heads are used to obtain its features in different representation spaces and fuse them;

[0021] The residual connection layer passes the output of the multi-head diagonal mask from the attention module to the next layer through residual connection, which can retain more information.

[0022] The feedforward layer linearly maps the output of the multi-head diagonal mask self-attention module;

[0023] The linear layer performs a dimensionality transformation operation on the obtained feature representation to obtain X1, which is data of the same dimension as the input for subsequent loss calculation.

[0024] Furthermore, the Module 2 layer includes a multi-head diagonal mask multidimensional attention module, a residual connection layer, a feedforward layer, and a ReLU activation function with Dropout random deactivation. The multi-head diagonal mask multidimensional attention module includes: input data undergoing a transpose operation, and then... f ·K f It can capture the potential connections within the second dimension, while preventing the data itself from contributing to its estimation through a diagonal mask. Then, by multiplying it with Vt' which contains information from the first dimension, it can obtain the potential connections between data in multiple dimensions. Then, it can obtain and fuse the features of the data in different representation spaces through multiple attention heads.

[0025] The residual connection layer passes the output of the multi-head diagonal mask multidimensional attention module to the next layer through residual connection, which can retain more information.

[0026] The feedforward layer linearly maps the output of the multi-head diagonal mask self-attention module;

[0027] The ReLU activation function and Dropout random deactivation are used to perform nonlinear transformation and random deactivation on the obtained feature representation, so as to achieve better feature representation while avoiding overfitting.

[0028] Furthermore, the method includes constructing a dynamic weight module, which concatenates the attention score of the multi-head diagonal mask multidimensional attention mechanism module with the mask matrix and then passes it through a linear layer to obtain the final weight coefficient γ. The weights {1-γ,γ} are assigned to the special representations Z1 and Z2 obtained in the above process to obtain the final feature representation Z3.

[0029] The replace operation includes replacing the values ​​at non-missing locations in the final feature representation Z3 with the observations in the original data X to obtain the final filled data Iimp.

[0030] A missing value imputation model for power load data based on MDAN includes:

[0031] The Data fusion layer is used to concatenate multidimensional load data and a mask matrix to obtain the input matrix of the model.

[0032] The input layer includes a multivariate time series position encoding and a fully connected linear layer; the multivariate time series position encoding is used to add position information to the time series, and the fully connected linear layer is used to map the input data to a dense vector.

[0033] Module 1 layer includes a multi-head diagonal mask self-attention module and a linear layer; the multi-head diagonal mask self-attention module is used to predict Z1 from the time dimension as V for the multi-head diagonal mask multi-dimensional attention of Module 2, and the linear layer is used to obtain the first predicted value;

[0034] Module 2 layer includes a multi-headed diagonal mask multi-dimensional attention module and a linear layer; the multi-headed diagonal mask multi-dimensional attention module is used to obtain the potential relationship between time and features to generate feature representation Z2 and attention score A, and the linear layer is used to obtain a second prediction value;

[0035] The system consists of a weighted fusion layer and a replacement layer. The weighted fusion layer is used to dynamically weight and fuse the intra-temporal prediction values ​​obtained from Module 1 with the feature-temporal multi-dimensional prediction values ​​obtained from Module 2 to obtain the final feature representation. The replacement layer is used to fill in the feature representation at missing positions and fill the non-missing positions with the original data to obtain the final filled value Iimp.

[0036] Furthermore, the Data fusion layer is used to stitch together multidimensional load data, relevant meteorological data, and a mask matrix to obtain the input matrix of the model.

[0037] Furthermore, the Module1 layer includes a multi-head diagonal mask self-attention module, a residual connection layer, a feedforward layer, and a linear layer; the multi-head diagonal mask self-attention module includes: the input data has not undergone transpose operation, and the attention mechanism can capture the potential connections within the first dimension, while the diagonal mask prevents the data itself from contributing to its estimation, and then multiple attention heads are used to obtain its features in different representation spaces and fuse them;

[0038] The residual connection layer passes the output of the multi-head diagonal mask from the attention module to the next layer through residual connection, which can retain more information.

[0039] The feedforward layer linearly maps the output of the multi-head diagonal mask self-attention module;

[0040] The linear layer performs a dimensionality transformation operation on the obtained feature representation to obtain X1, which is data of the same dimension as the input for subsequent loss calculation.

[0041] Furthermore, the Module 2 layer includes a multi-head diagonal mask multidimensional attention module, a residual connection layer, a feedforward layer, and a ReLU activation function with Dropout random deactivation. The multi-head diagonal mask multidimensional attention module includes: input data undergoing a transpose operation, and then...f ·K f It can capture the potential connections within the second dimension, while preventing the data itself from contributing to its estimation through a diagonal mask. Then, by multiplying it with Vt' which contains information from the first dimension, it can obtain the potential connections between data in multiple dimensions. Then, it can obtain and fuse the features of the data in different representation spaces through multiple attention heads.

[0042] The residual connection layer passes the output of the multi-head diagonal mask multidimensional attention module to the next layer through residual connection, which can retain more information.

[0043] The feedforward layer linearly maps the output of the multi-head diagonal mask self-attention module;

[0044] The ReLU activation function and Dropout random deactivation are used to perform nonlinear transformation and random deactivation on the obtained feature representation, so as to achieve better feature representation while avoiding overfitting.

[0045] Furthermore, the method includes constructing a dynamic weight module, which concatenates the attention score of the multi-head diagonal mask multidimensional attention mechanism module with the mask matrix and then passes it through a linear layer to obtain the final weight coefficient γ. The weights {1-γ,γ} are assigned to the special representations Z1 and Z2 obtained in the above process to obtain the final feature representation Z3.

[0046] The replace operation includes replacing the values ​​at non-missing locations in the final feature representation Z3 with the observations in the original data X to obtain the final filled data Iimp.

[0047] The beneficial effects of this invention are:

[0048] The system constructs Module 1, which captures the potential relationships between the time dimensions of the workload data, resulting in the first feature representation X1 containing time dimension information. Module 2 captures the potential relationships between feature dimensions, captures the multi-dimensional relationships between features and timestamps, and generates a feature representation X2 containing multi-dimensional information. The capture of potential relationships in the timestamp dimension uses a self-attention mechanism to generate feature representations from the time dimension, while the capture of multi-dimensional relationships between features and features and timestamps uses a multi-dimensional attention mechanism. A dynamic weighted fusion module is constructed, which concatenates the attention score from Module 2 with the mask matrix and then reduces the dimensionality to obtain learnable weight coefficients. These weights are then used to weight the feature representations obtained from Module 1 and Module 2 to obtain a more accurate feature representation, better suited for subsequent workload missing value imputation, thus improving the accuracy of missing value imputation.

[0049] Compared with other traditional models and models based on big data technology, the multivariate air quality data missing value imputation model provided by this invention can achieve higher prediction accuracy, opening up new ideas for subsequent multivariate air quality data missing value imputation tasks. Attached Figure Description

[0050] Figure 1 This is a flowchart of the method for constructing a missing value imputation model for power load data based on MDAN, as described in this invention;

[0051] Figure 2 This is a specific data execution flowchart of the power load data missing value filling model based on MDAN of the present invention;

[0052] Figure 3 This is a schematic diagram of the missing value filling model for power load data based on MDAN of the present invention;

[0053] Figure 4 This is a specific network structure diagram of the power load data missing value filling model based on MDAN in this invention;

[0054] Figure 5 It is a way to evaluate the accuracy of missing value imputation. Detailed Implementation

[0055] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0056] Example:

[0057] This invention relates to a method for constructing a missing value filling model for power load data based on MDAN. The hardware execution subject can be a desktop computer, a laptop computer, a server device, a smart mobile terminal (tablet computer, smartphone, etc.), etc., and this embodiment does not limit it.

[0058] like Figure 1 As shown, the construction method includes:

[0059] Step 1: Build the Data Fusion layer:

[0060] A Data Fusion layer is constructed to normalize the power load data. Since the differences between different units of power load data are too large, the Min-Max normalization function is used.

[0061] Normalization function:

[0062]

[0063] in For input data, For the normalized data, and These represent the maximum and minimum values ​​of a sample under the same feature dimension, respectively.

[0064] The Data Fusion layer is used to stitch together multidimensional load data, relevant meteorological data, and a mask matrix to obtain the input matrix of the model.

[0065] Step 2: Construct the Input layer, which includes a multivariate time series position encoding and a fully connected linear layer; the multivariate time series position encoding is used to add position information to the time series, and the fully connected linear layer is used to map the input data to a dense vector.

[0066] Since this model requires missing value imputation, the original data X needs to be concatenated with a Mask matrix containing the location information of missing values ​​as input. Because the input data has a small time span, with each value representing only an indicator of a certain attribute at a specific time stamp, a fully connected layer is added to map the sparse vector into a high-dimensional dense vector. Since the self-attention mechanism cannot recognize the temporal location information of multi-dimensional time series, positional encoding is added to encode the temporal information.

[0067] Fully connected layer: e1 = Linear(concat(X, Mask))

[0068] Location coding: Where p is the location index of a feature in the load data, 2i is the even-numbered dimension, 2i+1 represents the odd-numbered dimension, and d model This is the dimension of the positional encoding vector, and the elements are mapped to d using the PE(p,i) function. model Dimension. Step 3: Construct Module 1, which includes a multi-head diagonal mask self-attention module and a linear layer; the multi-head diagonal mask attention module is used to predict from the time dimension to obtain Z1 as V of the multi-head diagonal mask multi-dimensional attention of Module 2, and the linear layer is used to obtain the first feature representation X1.

[0069] It also includes a residual connection layer, which feeds the output of the multi-head diagonal mask self-attention module into the next layer through residual connections, thus retaining more information. A feedforward layer performs a linear mapping on the output of the multi-head diagonal mask self-attention module.

[0070] First, load data is a type of time series data, and its numerical prediction over future times depends on its temporal variation characteristics. Second, load data has multiple feature dimensions, where each feature dimension's data is a univariate time series, and there are potential relationships between different features. This example designs DMSA, which uses a self-attention mechanism to calculate correlations.

[0071]

[0072]

[0073]

[0074] Where Qt, Kt, and Vt represent the Query vector, Key vector, and Value vector of each sequence, respectively.

[0075] Diagonal mask self-attention mechanism:

[0076] Multiple attention heads are constructed to learn features in different representation spaces, focusing on correlation calculations across different spaces and concatenating the results of multiple attention heads as the output vector. Following the multi-head attention, a ReLU non-linear activation function and Dropout random deactivation are applied. ReLU adds non-linearity to the neural network, accelerating the model's learning speed and enhancing its fitting ability, while random deactivation reduces overfitting.

[0077] Step 4: Construct Module2, which includes a multi-headed diagonal mask multidimensional attention module, a residual connection layer, a feedforward layer, and a linear layer. The multi-headed diagonal mask multidimensional attention module is used to obtain potential connections between different dimensions of data from the feature-time dimension, and then obtains X2 containing multidimensional connections through the feedforward layer.

[0078] To focus on the correlations between data across different dimensions, this example uses a multidimensional attention module to calculate the correlation between features and the time dimension. Furthermore, to prevent itself from contributing to the estimation, a diagonal mask is used before the softmax activation function.

[0079] Diagonal mask:

[0080]

[0081] Diagonal mask multidimensional attention mechanism:

[0082]

[0083] Module 1: Temporal Dimension Potential Connection Capture Block: To focus on potential connections between data from different time periods, the model calculates an attention score matrix across all timestamps; a higher score indicates a higher correlation. A multi-head attention mechanism can focus on correlation calculations across different representation spaces and concatenates multiple attention results as the output vector. ReLU non-linear activation and Dropout random deactivation are then applied. ReLU adds non-linearity to the neural network, accelerating the model's learning speed and enhancing its fitting ability, while random deactivation reduces overfitting.

[0084] like Figure 2 The diagram shows a specific data processing flowchart for capturing potential connections in the time dimension. First, a linear layer is used to map the normalized power load data into a dense vector. Then, position encoding is added to it. Next, timestamp Q, K, and V matrices are generated to calculate the correlation and obtain the timestamp attention matrix, which is the feature representation matrix in the time dimension.

[0085] Module 2 Feature - Temporal Dimension Potential Connection Capture Block:

[0086] To focus on the potential relationships between different features, the model calculates an attention score matrix across all timestamps, with higher scores indicating higher relevance. This score is then multiplied by a V matrix containing the potential relationships between timestamps to obtain the potential relationships between features and the time dimension. The multi-head attention mechanism can focus on the correlation calculation across different representation spaces and concatenate multiple attention results as the output vector. Subsequently, ReLU non-linear activation and Dropout random deactivation are applied, along with the addition of linear layers. ReLU adds non-linearity to the neural network, accelerating the model's learning speed and enhancing its fitting ability; random deactivation reduces overfitting; and the linear layers increase the model's depth, enabling better learning of feature representations.

[0087] like Figure 2The diagram shows a specific data processing flowchart for feature-time multidimensional data. First, the normalized meteorological load data is transposed. Then, a linear layer is used to map the standardized transposed multivariate air quality data into a dense vector. Next, positional encoding is added to the multivariate time series (dense vector). Then, feature Q and K matrices are generated for correlation calculation. Finally, the matrix is ​​operated with the V matrix, which contains potential connections in the timestamp dimension, to obtain the feature Attention matrix, which is the feature representation of multidimensional potential connections. Step 5: Construct a Weighted Fusion Layer. The Weighted Fusion Layer performs a weighted fusion of feature representation X1 containing potential connections in the time dimension and feature representation X2 containing potential connections in the feature-time dimension. The weight matrix γ comes from Module 2. After concatenating the Attention matrix and the Mask matrix, it is first passed through a linear layer for dimensional transformation. Then, the values ​​are mapped to the [0,1] interval using a sigmoid function to obtain γ as the weight of feature representation X2. 1-γ is used as the weight of feature representation X1 to obtain the final feature representation X3.

[0088] like Figure 2 The diagram illustrates the detailed processing flow of weighted fusion. The final feature representation is obtained by dynamically weighting and fusing two feature representations, and then this final representation replaces the missing values ​​in the original data to arrive at the final result. The first two modules yield feature representations containing potential temporal relationships and feature-time relationship potentials. To learn better feature representations, learnable weights are added to the first two feature representations. These weights are obtained by concatenating the Attention matrix from the multi-head diagonal mask multi-dimensional attention module with the mask matrix, followed by a linear layer and a sigmoid function. Weighted fusion yields the final feature representation X3. Then, the missing values ​​in the original data are replaced with the corresponding values ​​from X3 to obtain the final filled data I. imp .

[0089] The model provided by this invention uses a multi-dimensional attention mechanism to capture the relationship between features and the time dimension, and a self-attention mechanism to capture the time dimension dependency, thus uncovering the potential interrelationships within the data. Simultaneously, it dynamically weights and fuses the feature representations obtained from the two modules to achieve better imputation results. Furthermore, the attention mechanism solves the long-distance dependency problem inherent in RNN-based missing value imputation models. Compared to other models based on big data technologies, it achieves higher accuracy in missing value imputation tasks, opening up new avenues for future missing value imputation tasks.

[0090] Example of a model for imputing missing values ​​in power load data based on MDAN:

[0091] This embodiment provides a missing value imputation model for power load data based on MDAN, which corresponds to the construction method of the aforementioned missing value imputation model for power load data based on MDAN. Figure 3 As shown, the missing value imputation model for power load data based on MDAN includes:

[0092] The Data fusion layer is used to concatenate multidimensional load data and a mask matrix to obtain the input matrix of the model.

[0093] The input layer includes a multivariate time series position encoding and a fully connected linear layer; the multivariate time series position encoding is used to add position information to the time series, and the fully connected linear layer is used to map the input data to a dense vector.

[0094] The Module 1 layer includes a multi-head diagonal mask self-attention module and a linear layer. The multi-head diagonal mask self-attention module is used to predict Z1 from the time dimension as V for the multi-head diagonal mask multi-dimensional attention of Module 2. The linear layer is used to obtain the first predicted value.

[0095] Module 2 includes a multi-headed diagonal mask multi-dimensional attention module and a linear layer. The multi-headed diagonal mask multi-dimensional attention module is used to obtain the potential relationship between time and features, generate feature representation Z2 and attention score A, and the linear layer is used to obtain a second prediction value.

[0096] The system consists of a weighted fusion layer and a replacement layer. The weighted fusion layer fuses the feature representations within the time dimension obtained from Module 1 with the feature representations between the feature and time dimensions obtained from Module 2 using a dynamic weighting method to obtain the final feature representation Z3. The replacement layer fills missing positions with the value of feature representation Z3 at those positions, and fills non-missing positions with the original data, obtaining the final filled value as the output I. imp。

[0097] Figure 4 A specific network structure diagram for a model that fills in missing values ​​in MDAN-based power load data.

[0098] Figure 5In practical applications, missing values ​​are imputed in two different sets of data. The main idea is to first normalize the original data X, then apply an artificial mask with a certain proportion, and finally use the model proposed in this invention to impute the missing data in the mask. By simulating missing values ​​in complete data, the imputation accuracy of the model can be directly evaluated. For data that already has missing values, indirect evaluation is required.

[0099] This invention is not limited to the preferred embodiments described above. Anyone can derive other products in various forms under the guidance of this invention. However, regardless of any changes in shape or structure, any technical solution that is the same as or similar to this application falls within the protection scope of this invention.

Claims

1. A method for constructing an MDAN-based power load data missing value filling model, characterized in that, Includes the following steps: A Data fusion layer is constructed, which is used to concatenate multidimensional load data and a mask matrix to obtain the input matrix of the model; An input layer is constructed, which includes a multivariate time series position encoding and a fully connected linear layer; the multivariate time series position encoding is used to add position information to the time series, and the fully connected linear layer is used to map the input data to a dense vector; Construct Module 1, which includes a multi-headed diagonal mask self-attention module and a linear layer. The multi-headed diagonal mask self-attention module is used to predict Z1 from the time dimension as V for the multi-headed diagonal mask multidimensional attention of Module 2. The linear layer is used to obtain the first predicted value. Construct Module 2, which includes a multi-headed diagonal mask multidimensional attention module and a linear layer. The multi-headed diagonal mask multidimensional attention module is used to obtain the potential relationship between time and features, generate a feature representation Z2 and an attention score A. The linear layer is used to obtain the second predicted value. A weighted fusion layer and a replacement layer are constructed. The weighted fusion layer is used to dynamically weight and fuse the intra-temporal prediction values ​​obtained from Module 1 with the feature-temporal multi-dimensional prediction values ​​obtained from Module 2 to obtain the final feature representation Z3. The replacement layer is used to fill the feature representation at missing positions and fill the non-missing positions with the original data to obtain the final value. 2.The method of claim 1, wherein, The Data fusion layer is used to stitch together multidimensional load data, relevant meteorological data, and a mask matrix to obtain the input matrix of the model. 3.The method of claim 1, wherein, The Module1 layer includes a multi-head diagonal mask self-attention module, a residual connection layer, a feedforward layer, and a linear layer; The multi-head diagonal mask self-attention module includes: input data that has not undergone transpose operation, which can capture the potential connections within the first dimension through an attention mechanism, while preventing the data itself from contributing to its estimation through a diagonal mask, and then using multiple attention heads to obtain and fuse its features in different representation spaces; The residual connection layer passes the output of the multi-head diagonal mask from the attention module to the next layer through residual connection. The feedforward layer linearly maps the output of the multi-head diagonal mask self-attention module; The linear layer performs a dimensionality transformation operation on the obtained feature representation to obtain X1, which is data of the same dimension as the input for subsequent loss calculation. 4.The method of claim 1, wherein, The Module 2 layer includes a multi-head diagonal mask multidimensional attention module, a residual connection layer, a feedforward layer, and a ReLU activation function with Dropout random deactivation. The multi-head diagonal mask multidimensional attention module includes: input data transposed and then processed by Q... f ·K f It can capture the potential connections within the second dimension, while preventing the data itself from contributing to its estimation through diagonal masking. Then, it performs a product operation with Vt', which contains information from the first dimension, and then uses multiple attention heads to obtain and fuse its features in different representation spaces. The residual connection layer passes the output of the multi-head diagonal mask multidimensional attention module to the next layer through residual connection. The feedforward layer linearly maps the output of the multi-head diagonal mask self-attention module; The ReLU activation function and Dropout random deactivation are used to perform nonlinear transformation and random deactivation on the obtained feature representation, so as to achieve better feature representation while avoiding overfitting.

5. The method for constructing a missing value imputation model for power load data based on MDAN according to claim 1, characterized in that, The dynamic weighting module is constructed by concatenating the attention score of the multi-head diagonal mask multidimensional attention mechanism module with the mask matrix and then passing it through a linear layer to obtain the final weight coefficient γ. The weights {1-γ,γ} are assigned to the special representations Z1 and Z2 obtained in the above process to obtain the final feature representation Z3. The replace operation includes replacing the values ​​at non-missing locations in the final feature representation Z3 with the observations from the original data X, resulting in the final filled data I. imp . 6.A model for filling missing values of power load data based on MDAN, characterized in that, include: The Data fusion layer is used to concatenate multidimensional load data and a mask matrix to obtain the input matrix of the model. The input layer includes a multivariate time series position encoding and a fully connected linear layer; the multivariate time series position encoding is used to add position information to the time series, and the fully connected linear layer is used to map the input data to a dense vector. Module 1 layer includes a multi-head diagonal mask self-attention module and a linear layer; the multi-head diagonal mask self-attention module is used to predict Z1 from the time dimension as V for the multi-head diagonal mask multi-dimensional attention of Module 2, and the linear layer is used to obtain the first predicted value; Module 2 layer includes a multi-headed diagonal mask multi-dimensional attention module and a linear layer; the multi-headed diagonal mask multi-dimensional attention module is used to obtain the potential relationship between time and features to generate feature representation Z2 and attention score A, and the linear layer is used to obtain a second prediction value; The weighted fusion layer and the replacement layer are used to dynamically weight and fuse the intra-temporal prediction values ​​obtained from Module 1 with the feature-temporal multi-dimensional prediction values ​​obtained from Module 2 to obtain the final feature representation. The replace layer is used to fill in the missing locations with feature representations, and fill the non-missing locations with the original data to obtain the final filled value I. imp .

7. The MDAN-based power load data missing value filling model according to claim 6, characterized in that, The Data fusion layer is used to stitch together multidimensional load data, relevant meteorological data, and a mask matrix to obtain the input matrix of the model.

8. The power load data missing value imputation model based on MDAN according to claim 6, characterized in that, The Module1 layer includes a multi-head diagonal mask self-attention module, a residual connection layer, a feedforward layer, and a linear layer; The multi-head diagonal mask self-attention module includes: input data that has not undergone transpose operation, which can capture the potential connections within the first dimension through an attention mechanism, while preventing the data itself from contributing to its estimation through a diagonal mask, and then using multiple attention heads to obtain and fuse its features in different representation spaces; The residual connection layer passes the output of the multi-head diagonal mask from the attention module to the next layer through residual connection. The feedforward layer linearly maps the output of the multi-head diagonal mask self-attention module; The linear layer performs a dimensionality transformation operation on the obtained feature representation to obtain X1, which is data of the same dimension as the input for subsequent loss calculation. 9.The MDAN-based power load data imputation model of claim 6, wherein, The Module 2 layer includes a multi-head diagonal mask multidimensional attention module, a residual connection layer, a feedforward layer, and a ReLU activation function with Dropout random deactivation. The multi-head diagonal mask multidimensional attention module includes: input data transposed and then processed by Q... f ·K f It can capture the potential connections within the second dimension, while preventing the data itself from contributing to its estimation through a diagonal mask. Then, by multiplying it with Vt' which contains information from the first dimension, it can obtain the potential connections between data in multiple dimensions. Then, it can obtain and fuse the features of the data in different representation spaces through multiple attention heads. The residual connection layer passes the output of the multi-head diagonal mask multidimensional attention module to the next layer through residual connection. The feedforward layer linearly maps the output of the multi-head diagonal mask self-attention module; The ReLU activation function and Dropout random deactivation are used to perform nonlinear transformation and random deactivation on the obtained feature representation, so as to achieve better feature representation while avoiding overfitting. 10.The MDAN-based power load data missing value filling model according to claim 6, wherein, The dynamic weighting module is constructed by concatenating the attention score of the multi-head diagonal mask multidimensional attention mechanism module with the mask matrix and then passing it through a linear layer to obtain the final weight coefficient γ. The weights {1-γ,γ} are assigned to the special representations Z1 and Z2 obtained in the above process to obtain the final feature representation Z3. the replace operation includes: replacing the value of the non-missing position in the obtained final feature representation Z3 with the observed value in the original data X to obtain the final filled data I imp .

Images