A time series data completion method and system

Abstract

The present invention provides a time series data completion method and system, the method comprising: collecting time series related operation data of a DC converter transformer, constructing an original time series defect data set X 0; for the original time series defective data set X 0 performs time correlation preprocessing to obtain the time correlation preprocessing matrix X 1; The time-dependent preprocessing matrix X 1 Input the CNN network to integrate the data sequence to obtain the integration matrix X 2. Integrate the matrix X 2. Send it to the bidirectional long short-term memory neural network Bi-LSTM for time correlation prediction learning to obtain the time correlation prediction result X 3. Predict the time correlation results X 3. Send it to the anti-attention layer to reduce the attention of the pre-filled data to obtain the reconstructed attention matrix X 4; Reconstruct the attention matrix X 4 is put into the fully connected output layer to obtain the data prediction completion result #imgabs0#. The present invention solves the technical problems of gradient disappearance, overfitting and low accuracy of completed data.

Description

Technical Field

[0001] The present invention relates to the field of power grid control, and in particular to a time series data completion method and system. Background Art

[0002] The transformers of UHV DC converter stations play an important role in the power transmission process and are expensive. However, once manufactured by the manufacturer, their structure is fixed, and the positions and number of sensors that can be arranged are also determined, resulting in limited data collected by the sensors. For example, the transformer oil chromatography and the volume fraction of gas in the transformer oil change over time. The sensor may detect anomalies at some point in the process, resulting in data missing. It is necessary to effectively supplement the transformer data that is missing in the data set over time. The use of the supplemented data will be beneficial to the subsequent transformer fault detection work.

[0003] The traditional long short-term memory network (LSTM) can only process data in a unidirectional manner from front to back, and cannot integrate the beneficial effects brought by processing data in the reverse direction, resulting in the problem of gradient vanishing. In the application process of traditional network models, the aggregation and convergence efficiency is low and there is an overfitting problem. It is difficult to retain data features in the time dimension, which is not conducive to better prediction and completion of data. At the same time, the traditional model has poor effect in distinguishing between predicted data and real data, and the accuracy of completed data is low.

[0004] The existing invention patent application document "A Code Completion Method, Model and Tool" with publication number CN114327609A, the existing method includes: obtaining real projects from real code repositories and classifying data sets based on the project source codes of real projects, and constructing code completion data sets by selecting a set of real projects used in the work; constructing a semantic-based code completion model, which includes building a sequence-to-sequence neural network with an attention mechanism, performing source code segmentation through a bit pair encoding BPE (Byte-Pair Encoding) segmentation algorithm, performing context dependency learning through a bidirectional long short-term memory neural network, using an attention mechanism to learn text weights, and adapting based on a sequence-to-sequence neural network according to different prediction scenarios; constructing a grammar-based code completion model; and reordering the prediction results to obtain the target prediction text and arrangement order. However, the attention mechanism used in the aforementioned prior art will result in the generated code being replaced by a false value of the original missing position, so that the false information and the real information are replaced equally, resulting in a low accuracy rate of the completed information.

[0005] The existing invention patent application document "Knowledge Graph Completion Method Based on Neural Network" with publication number CN110347847A, the existing method includes: converting the triples in the knowledge graph K into sentences in the embedding layer, and converting the sentences into k-dimensional dense vector representation using vector representation technology; S2, using the bidirectional long short-term memory neural network in the circulation layer to learn the long-term dependency of the sentence from the input feature vector to obtain the circulation layer feature vector; S3, using the convolutional neural network in the convolutional layer to learn the local structure information from the circulation layer feature vector to obtain the convolutional layer feature vector; S4, converting the convolutional layer feature vector into the score of each triple in the fully connected layer. However, the bidirectional long short-term memory neural network used in the above-mentioned prior art can only use bidirectional information flow, and there may be a problem of overfitting.

[0006] In summary, the existing technology has technical problems such as gradient disappearance, overfitting and low accuracy of completed data. Summary of the invention

[0007] The technical problem to be solved by the present invention is: how to solve the technical problems of gradient vanishing, overfitting and low accuracy of supplemented data in the prior art.

[0008] The present invention adopts the following technical solution to solve the above technical problem: A time series data completion method includes:

[0009] S1. For DC converter transformers, collect and obtain timing-related operation data and construct the original timing defect data set X 0, the original time series defective dataset X 0 is divided into training set and test set;

[0010] S2. For the original time series defective dataset X 0, perform time correlation preprocessing operation to obtain the time correlation preprocessing matrix X 1;

[0011] S3, the time-dependent preprocessing matrix X 1 Input into the CNN (Convolutional Neural Network) network structure, perform data sequence integration operation, and obtain the integration matrix X 2;

[0012] S4. Integrate the matrix X 2. Send it to the bidirectional long short-term memory neural network Bi-LSTM to perform time correlation prediction learning and obtain the time correlation prediction result X 3;

[0013] S5. Time correlation prediction results X3. Send it to the anti-attention layer to reduce the attention on the pre-filled data to obtain the reconstructed attention matrix X 4;

[0014] S6. Reconstruct the attention matrix X 4 Put it into the fully connected output layer and process it to get the data prediction completion result X 5.

[0015] The present invention can use the time series data completion method of the Bi-LSTM (bidirectional long short-term memory neural network, Bi-Long Short-Term Memory) model based on the self-attention mechanism to complete the missing data collected by the DC converter transformer in the time series. X The preprocessing result of the time correlation of 0 is input into the CNN network with a residual mechanism to extract the key features of the data set and organize it into a sequence. It is then sent to the Bi-LSTM model with an attention reconstruction mechanism. Under the internal and external complementary attention mechanism, the missing data set is effectively completed and data is output in the fully connected output layer, thereby completing the missing data of the DC converter transformer data.

[0016] In a more specific technical solution, S2 includes:

[0017] S21. For the original time series defective dataset X 0, find the incomplete matrix to be completed A , to calculate the temporal correlation matrix and mark missing values ​​NaN using the following logic:

[0018] (1)

[0019] In the formula, Represents an incomplete matrix to be completed A the number of rows, Represents an incomplete matrix to be completed A The number of columns, Represents a one-dimensional column matrix T Corresponding to i The time value of the row, Represents a one-dimensional column matrix T Corresponding to i-1 The time value of the row; Represents an incomplete matrix to be completed A The same type matrix The Row, No. The value of the column, Indicates the time correlation matrix Row, No. The value of the column, Indicates the time correlation matrix Row, No. The value of the column;

[0020] S22, according to the time correlation matrix C , the attenuation matrix is ​​obtained by the following logical processing D :

[0021] (2)

[0022] In the formula, Represents the time correlation matrix C Middle i Row, No. j The value of the column; Represents the attenuation matrix D No. i Row, No. j The value of the column, e represents the calculation error vector;

[0023] S23. Completing the incomplete matrix A Perform pre-filling processing to fill the missing value NaN into the incomplete matrix to be completed according to the moment value of the preset time series A , get the pre-filled matrix :

[0024] (3)

[0025] In the formula, Represents an incomplete matrix to be completed A exist t The value of the moment; Represents a pre-filled matrix exist The value of the moment, Represents a pre-filled matrix exist t The value of the moment;

[0026] S24, horizontal splicing pre-filled matrix , attenuation matrix D , and obtain the time-dependent preprocessing matrix X 1.

[0027] In a more specific technical solution, in S3, a CNN network structure based on a residual mechanism is used to add an identity mapping between different convolutional layers through a residual network, wherein the residual block output of the convolutional layer is expressed using the following logic:

[0028] (4)

[0029] (5)

[0030] In the formula, represents the residual block output, Represents the direct mapping part, l Indicates the number of layers of the CNN network, represents the residual mapping part to be learned, F represents the residual mapping, represents the network layer parameters to be learned, represents the activation function, represents the weight of the first layer of the CNN network, Represents the weights of the second layer of the CNN network.

[0031] The residual block in the CNN network of the present invention can achieve faster and better aggregation and convergence of the residual network, so that the data in the CNN network can flow across layers to overcome the overfitting problem.

[0032] Specifically, the present invention adopts a CNN network based on a residual mechanism as the output layer, adds an identity mapping between different convolutional layers through the residual network, and uses residual blocks to achieve aggregation and convergence of the residual network, so that the data in the CNN network can flow across layers to avoid overfitting.

[0033] In a more specific technical solution, S4 includes:

[0034] S41, setting model parameters of the bidirectional long short-term memory neural network Bi-LSTM;

[0035] S42, integrate the matrix X 2 is put into the 2-layer reverse LSTM of the bidirectional long short-term memory neural network Bi-LSTM, and the integration matrix X 2. At different time points in different directions of the data sequence, two-way prediction learning is performed to obtain the time correlation prediction results. X 3.

[0036] Compared with the traditional recurrent neural network LSTM, which can only process data unidirectionally from front to back and cannot integrate the beneficial effects brought by processing data in the reverse direction, the bidirectional long short-term memory neural network Bi-LSTM adopted in the present invention is a bidirectional allowing model, which can learn forward and reverse context information at the same time during the training process, which helps to extract richer and more accurate feature representations. During the back propagation process, the reverse LSTM can capture the gradient information in the long-term dependency relationship and reduce the gradient disappearance problem.

[0037] In a more specific technical solution, in S42, in the bidirectional long short-term memory neural network Bi-LSTM, the following logic is used to calculate t The output hidden state vector at time:

[0038] (6)

[0039] (7)

[0040] (8)

[0041] (9)

[0042] (10)

[0043] (11)

[0044] In the formula, for t The door of forgetfulness of time, for t The input gate of the moment, for t The output gate at time, for t The cell state at a moment in time, for t The memory cells of time, for Memory cells of the moment; for t The output hidden state vector at time , for The output hidden state vector at time , is the activation function, tanh is the hyperbolic tangent activation function, is the weight matrix corresponding to the forget gate, is the weight matrix corresponding to the input gate, is the weight matrix corresponding to the output gate, for t Cell state at a moment The weight matrix of is the bias vector corresponding to the forget gate, is the bias vector corresponding to the input gate, is the bias vector corresponding to the output gate, for t Cell state at a moment The bias vector of for t The input vector at time instant.

[0045] In a more specific technical solution, S5 includes:

[0046] S51. In the bidirectional long short-term memory neural network Bi-LSTM, The output hidden state vector forward at time , the output hidden state vector of the backward Perform splicing processing to obtain the splicing vector:

[0047] (12)

[0048] In the formula, for t The concatenated vector of time, is the forward output hidden state vector at the first moment, is the forward output hidden state vector at the second moment, for t The forward output hidden state vector at time t is, is the output hidden state vector of the first moment, is the backward output hidden state vector at the second moment, for t The backward output hidden state vector at time t, is the component of the splicing vector at the first moment, is the component of the splicing vector at the second moment, for t The components of the concatenation vector at time instant;

[0049] S52. Perform attention scoring on each concatenated vector to obtain an initial attention score g t :

[0050] (13)

[0051] S53, according to the attenuation matrix D , using the following logic, calculate the vector missing degree , vector integrity :

[0052] (14)

[0053] (15)

[0054] In the formula, N is the dimension of the data vector at the current moment in the decay matrix, is the attenuation matrix t Moment h The data value of the row, is the attenuation matrix t Time 1st row to h The accumulation of row data values;

[0055] S54. Re-score the initial attention score according to the vector missingness and vector completeness to calculate the new attention score of each concatenated vector using the following logic:

[0056] (16)

[0057] In the formula, for The new attention score obtained at the moment, for The initial attention score at time t, for t The initial attention score at time t, for t The vector missing degree at time, for The vector integrity at the moment, Represents the time series divided by The accumulation of vector integrity at other times except the time;

[0058] S55, use the softmax function to process the time step n Attention distribution ;

[0059] S56. Through the new attention mechanism, according to the attention distribution and splicing vector, calculate the proportion of the input sequence in the total sequence ;

[0060] S57, assign different attentions to different time steps in the input sequence to select applicable prediction information for the current task goal and construct a reconstructed attention matrix X 4.

[0061] The present invention calculates the attenuation matrix D It can retain the data features in each time dimension, which is conducive to better prediction and completion of data. In the anti-attention layer, the attention score of the pre-filled data value is weakened, which is conducive to distinguishing the predicted data from the real data and improving the accuracy of the completed data.

[0062] The anti-attention layer in the present invention is used to solve the problem that false information and real information in the data are treated equally. According to the missing data of each splicing vector input to the encoder, according to the proportion of the input sequence in the total sequence, the attention of the decoder is appropriately diverged to other vectors. The effectiveness of the data completion operation is improved.

[0063] In the present invention, by adding attention mechanism and reconstructing attention mechanism, the time-dependent preprocessing matrix is ​​used X1Redistribute the neural network attention, and refer to the weight of each hidden state in the encoding process during the decoding process.

[0064] In a more specific technical solution, in S55, the following logic is used to process the time step n Attention distribution :

[0065] (17)

[0066] In the formula, is the time step n The corresponding new attention score, Represents the activation function.

[0067] In a more specific technical solution, in S56, the following logic is used to process and obtain the proportion of the input sequence in the total sequence: :

[0068] (18)

[0069] In the formula, The amount of input data is k The attention distribution at The amount of input data is k The corresponding splicing vector is Indicates the amount of input data is k The attention distribution and input data volume are k The product of the corresponding concatenated vectors is n The sum of time steps.

[0070] In a more specific technical solution, in S6, the following logic is used to adjust the network parameters of the bidirectional long short-term memory neural network Bi-LSTM in the fully connected output layer, perform optimization training, and update the parameters using the mean square error:

[0071] (20)

[0072] (twenty one)

[0073] In the formula, IN is the activation function The input value of For the l1 Fully connected layers with training weights, For the l2 Fully connected layers with training weights, The amount of input data is k The output prediction value of is the proportion of the input sequence in the total sequence , represents the sum of the product of the first fully connected layer with training weights and the proportion of the input sequence in the total sequence, represents the sum of the product of the second fully connected layer with training weights and the proportion of the input sequence in the total sequence, n Represents the time step.

[0074] In a more specific technical solution, a time series data completion system includes:

[0075] The data set construction and partitioning module is used to collect and obtain timing-related operation data for DC converter transformers and construct the original timing defect data set X 0, the original time series defective dataset X 0 is divided into training set and test set;

[0076] Time correlation preprocessing module is used to process the original time series defective data set X 0, perform time correlation preprocessing operation to obtain the time correlation preprocessing matrix X 1. The time correlation preprocessing module is connected with the data set construction and partitioning module;

[0077] Sequence integration module, used to convert the time-dependent preprocessing matrix X 1 Input into the CNN network structure, perform data sequence integration operation, and obtain the integration matrix X 2. The sequence integration module is connected with the time correlation preprocessing module;

[0078] The time correlation prediction learning module is used to integrate the matrix X 2. Send it to the bidirectional long short-term memory neural network Bi-LSTM to perform time correlation prediction learning and obtain the time correlation prediction result X 3. The time correlation prediction learning module is connected with the sequence integration module;

[0079] Lower the attention module to predict the results of temporal correlation X 3. Send it to the anti-attention layer to reduce the attention on the pre-filled data to obtain the reconstructed attention matrix X 4. Reduce the connection between the attention module and the time correlation prediction learning module;

[0080] Prediction completion module, used to reconstruct the attention matrix X 4 Put it into the fully connected output layer and process it to get the data prediction completion result X 5. The prediction completion module is connected to the attention reduction module.

[0081] Compared with the prior art, the present invention has the following advantages:

[0082] The present invention can use the time series data completion method of the Bi-LSTM (bidirectional long short-term memory neural network, Bi-Long Short-Term Memory) model based on the self-attention mechanism to complete the missing data collected by the DC converter transformer in the time series. X The preprocessing result of the time correlation of 0 is input into the CNN network with a residual mechanism to extract the key features of the data set and organize it into a sequence. It is then sent to the Bi-LSTM model with an attention reconstruction mechanism. Under the internal and external complementary attention mechanism, the missing data set is effectively completed and data is output in the fully connected output layer, thereby completing the missing data of the DC converter transformer data.

[0083] The residual block in the CNN network of the present invention can achieve faster and better aggregation and convergence of the residual network, so that the data in the CNN network can flow across layers to overcome the overfitting problem.

[0084] Compared with the traditional recurrent neural network LSTM, which can only process data unidirectionally from front to back and cannot integrate the beneficial effects brought by processing data in the reverse direction, the bidirectional long short-term memory neural network Bi-LSTM adopted in the present invention is a bidirectional allowing model, which can learn forward and reverse context information at the same time during the training process, which helps to extract richer and more accurate feature representations. During the back propagation process, the reverse LSTM can capture the gradient information in the long-term dependency relationship and reduce the gradient disappearance problem.

[0085] The present invention calculates the attenuation matrix D It can retain the data features in each time dimension, which is conducive to better prediction and completion of data. In the anti-attention layer, the attention score of the pre-filled data value is weakened, which is conducive to distinguishing the predicted data from the real data and improving the accuracy of the completed data.

[0086] The anti-attention layer in the present invention is used to solve the problem that false information and real information in the data are treated equally. According to the missing data of each splicing vector input to the encoder, according to the proportion of the input sequence in the total sequence, the attention of the decoder is appropriately diverged to other vectors. The effectiveness of the data completion operation is improved.

[0087] In the present invention, by adding attention mechanism and reconstructing attention mechanism, the time-dependent preprocessing matrix is ​​used X 1Redistribute the neural network attention, and refer to the weight of each hidden state in the encoding process during the decoding process.

[0088] The present invention solves the technical problems of gradient disappearance, overfitting and low accuracy of supplemented data existing in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 A schematic diagram of basic steps of a time series data completion method according to Embodiment 1 of the present invention;

[0090] Figure 2 The calculation and time-dependent preprocessing matrix of Embodiment 1 of the present invention is X 1. Schematic diagram of the specific steps;

[0091] Figure 3 Schematic diagram of the overall structure of the time series data completion model of Example 1 of the present invention;

[0092] Figure 4 This is a schematic diagram of specific steps for performing time correlation prediction learning in Example 1 of the present invention;

[0093] Figure 5 This is a schematic diagram of the specific steps of reconstructing attention on pre-filled data using the anti-attention layer in Example 1 of the present invention. DETAILED DESCRIPTION

[0094] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0095] Example 1

[0096] like Figure 1 As shown, a time series data completion method includes the following specific steps:

[0097] Step S1: For the DC converter transformer, collect and obtain the timing correlation operation data to construct the original timing defect data set X 0, the original time series defective dataset X 0 is divided into training set and test set;

[0098] In this embodiment, in this embodiment, the time series operation data with time series correlation includes but is not limited to: oil temperature, gas concentration;

[0099] Step S2: original time series defective data set X 0, perform time correlation preprocessing operation to obtain the time correlation preprocessing matrix X 1;

[0100] like Figure 2As shown, in this embodiment, the time-dependent preprocessing matrix is ​​calculated X Step S2 of 1 further includes the following specific steps:

[0101] Step S21: For the original time series defective data set X 0, find the incomplete matrix to be completed A , calculate the time correlation matrix and mark missing values ​​NaN;

[0102] In this embodiment, an incomplete matrix to be completed is established. A The same type matrix B ,use Represents an incomplete matrix to be completed A The data is missing, 0 means missing, 1 means not missing; establish an incomplete matrix to be completed A One-dimensional column matrices with the same number of columns T Represents a time series, the sampling time interval of adjacent rows can be set to, for example, 1 second; A and a one-dimensional column matrix T Calculate the time correlation matrix C , can be calculated as follows:

[0103] (1).

[0104] Step S22: According to the time correlation matrix C , and the attenuation matrix is ​​obtained by processing D ;

[0105] In this embodiment, the attenuation matrix D Calculate according to the following formula:

[0106] (2).

[0107] S23. Completing the incomplete matrix A Perform pre-filling processing to fill the missing value NaN into the incomplete matrix to be completed according to the moment value of the preset time series A , get the pre-filled matrix ;

[0108] In this embodiment, the incomplete matrix to be completed A Perform pre-filling processing to replace the missing value NaN with the previous moment value on the time series to obtain the incomplete matrix to be completed A , as follows:

[0109] (3).

[0110] S24, horizontal splicing pre-filled matrix , attenuation matrixD , and obtain the time-dependent preprocessing matrix X 1.

[0111] Step S3: Time-dependent preprocessing matrix X 1 Input into the CNN (Convolutional Neural Network) network structure, perform data sequence integration operation, and obtain the integration matrix X 2;

[0112] In this embodiment, the time-dependent preprocessing matrix X 1 Input the one-dimensional CNN network to integrate the data sequence to obtain the integration matrix X 2. The CNN network structure based on the residual mechanism adds an identity mapping between different convolutional layers through the residual network. The convolutional neural network parameters of CNN can be set as follows: the convolutional layer size is 3; the activation functions of the convolutional layer and the pooling layer are both ReLU activation functions; the residual block output of the convolutional layer is as follows:

[0113] (4)

[0114] (5).

[0115] Step S4: integrate the matrix X 2. Send it to the bidirectional long short-term memory neural network Bi-LSTM to perform time correlation prediction learning and obtain the time correlation prediction result X 3;

[0116] In this embodiment, the bidirectional long short-term memory neural network Bi-LSTM uses a gate mechanism to combine long-term memory with short-term memory, and can obtain data features from both positive and negative directions. It can not only characterize past data, but also predict future development trends, and has the advantage of bidirectional time series data prediction. The time correlation of missing values ​​is used to perform internal and external attention, and the prediction results of the bidirectional long short-term memory neural network Bi-LSTM are improved in accuracy in the time dimension.

[0117] like Figure 3 and Figure 4 As shown, in this embodiment, step S4 of performing time correlation prediction learning also includes the following specific steps:

[0118] Step S41, setting model parameters of a bidirectional long short-term memory neural network Bi-LSTM;

[0119] In this embodiment, the LSTM hidden layer can be set to, for example, 2 layers; the activation function can be selected as, for example, tanh; Dropout can be set to, for example, 0.3; the time step can be set to, for example, 4, the batch size can be set to, for example, 2, and the initial learning rate can be set to, for example, 0.015; the optimization function can be Adam; the loss function can be selected as, for example, MSE Loss (MeanS square ErrorLoss, mean square error).

[0120] Step S42: integrate the matrix X 2 is put into the 2-layer reverse LSTM of the bidirectional long short-term memory neural network Bi-LSTM, and the integration matrix X 2. At different time points in different directions of the data sequence, two-way prediction learning is performed to obtain the time correlation prediction results. X 3;

[0121] In this embodiment, the matrix is ​​integrated X The data sequence of 2 is put into the two layers of reverse LSTM of the bidirectional long short-term memory neural network Bi-LSTM. The first layer LSTM inputs data from left to right, and the second layer LSTM inputs data from right to left. Bidirectional prediction learning is performed with the forward and backward time points on the data sequence respectively to obtain the time correlation prediction result. X 3.

[0122] In this embodiment, the bidirectional long short-term memory neural network Bi-LSTM calculates t The output hidden state vector at time t is shown in the following equations (6)-(11):

[0123] (6)

[0124] (7)

[0125] (8)

[0126] (9)

[0127] (10)

[0128] (11).

[0129] Step S5: Time correlation prediction results X 3. Send it to the anti-attention layer to reduce the attention on the pre-filled data to obtain the reconstructed attention matrix X 4;

[0130] like Figure 5As shown, in this embodiment, step S5 of reconstructing the attention of the pre-filled data using the anti-attention layer anti-attention also includes the following specific steps:

[0131] Step S51: In a bidirectional long short-term memory neural network Bi-LSTM, t The output hidden state vector forward at time , the output hidden state vector of the backward Perform splicing processing to obtain a splicing vector;

[0132] In this embodiment, the bidirectional long short-term memory neural network Bi-LSTM in the above step S4 is The output hidden state vector forward at time , the output hidden state vector of the backward The splicing is performed as shown in the following formula (12):

[0133] (12).

[0134] Step S52: Perform attention scoring on each concatenated vector to obtain an initial attention score. g t ;

[0135] In this embodiment, the following formula (13) is used to score the hidden state of each concatenated vector:

[0136] (13)

[0137] Step S53: According to the attenuation matrix D , calculate the vector missing degree , vector integrity ;

[0138] In this embodiment, the attenuation matrix in the aforementioned step S22 is D The vector missing degree is calculated using the following formula (14) (15): , vector integrity :

[0139] (14)

[0140] (15).

[0141] Step S54: re-score the initial attention score according to the vector missing degree and the vector completeness, and calculate the new attention score of each spliced ​​vector;

[0142] In this embodiment, the new attention score of each concatenated vector is calculated according to the following formula (16):

[0143] (16).

[0144] Step S55: Use the softmax function to process the time step n Attention distribution :

[0145] (17).

[0146] Step S56: Calculate the proportion of the input sequence in the total sequence based on the attention distribution and the concatenation vector through the new attention mechanism :

[0147] (18).

[0148] Step S57: assign different attentions to different time steps in the input sequence to select applicable prediction information for the current task target and construct a reconstructed attention matrix. X 4;

[0149] In this embodiment, different degrees of attention are given to different time steps in the data sequence, so as to select more critical prediction information for the current task goal;

[0150] In this embodiment, the training process takes minimizing the reconstruction error as the network parameter optimization goal:

[0151] (19)

[0152] in, is the target loss function; is the actual data value of the concatenated vector; is the predicted completion data value of the concatenated vector, N is the dimension of the data vector at the current moment in the decay matrix.

[0153] Step S6: Reconstruct the attention matrix X 4 Put it into the fully connected output layer and process it to get the data prediction completion result X 5;

[0154] In this embodiment, the network optimization training is performed by adjusting the number of iterations and network nodes in the fully connected output layer, and the output concatenated vector is used as the input to calculate, as shown in the following formula (20) (21), and the parameters are updated using the mean square error:

[0155] (20)

[0156] (twenty one).

[0157] In summary, the present invention can use the time series data completion method of the Bi-LSTM (bidirectional long short-term memory neural network, Bi-Long Short-Term Memory) model based on the self-attention mechanism to complete the missing data collected by the DC converter transformer in the time series. X The preprocessing result of the time correlation of 0 is input into the CNN network with a residual mechanism to extract the key features of the data set and organize it into a sequence. It is then sent to the Bi-LSTM model with an attention reconstruction mechanism. Under the internal and external complementary attention mechanism, the missing data set is effectively completed and data is output in the fully connected output layer, thereby completing the missing data of the DC converter transformer data.

[0158] The residual block in the CNN network of the present invention can achieve faster and better aggregation and convergence of the residual network, so that the data in the CNN network can flow across layers to overcome the overfitting problem.

[0159] Compared with the traditional recurrent neural network LSTM, which can only process data unidirectionally from front to back and cannot integrate the beneficial effects brought by processing data in the reverse direction, the bidirectional long short-term memory neural network Bi-LSTM adopted in the present invention is a bidirectional allowing model, which can learn forward and reverse context information at the same time during the training process, which helps to extract richer and more accurate feature representations. During the back propagation process, the reverse LSTM can capture the gradient information in the long-term dependency relationship and reduce the gradient disappearance problem.

[0160] The present invention calculates the attenuation matrix D It can retain the data features in each time dimension, which is conducive to better prediction and completion of data. In the anti-attention layer, the attention score of the pre-filled data value is weakened, which is conducive to distinguishing the predicted data from the real data and improving the accuracy of the completed data.

[0161] The anti-attention layer in the present invention is used to solve the problem that false information and real information in the data are treated equally. According to the missing data of each splicing vector input to the encoder, according to the proportion of the input sequence in the total sequence, the attention of the decoder is appropriately diverged to other vectors. The effectiveness of the data completion operation is improved.

[0162] In the present invention, by adding attention mechanism and reconstructing attention mechanism, the time-dependent preprocessing matrix is ​​used X 1Redistribute the neural network attention, and refer to the weight of each hidden state in the encoding process during the decoding process.

[0163] The present invention solves the technical problems of gradient disappearance, overfitting and low accuracy of supplemented data existing in the prior art.

[0164] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A time series data completion method, characterized in that: The method comprises: S1. For DC converter transformers, collect and obtain timing-related operation data and construct the original timing defect data set X 0, the original time series defective data set X 0 is divided into training set and test set; S2. the original time series defective data set X 0, perform time correlation preprocessing operation to obtain the time correlation preprocessing matrix X 1; S3, the time-dependent preprocessing matrix X 1 Input into the CNN network structure, perform data sequence integration operation, and obtain the integration matrix X 2; S4, the integration matrix X 2. Send it to the bidirectional long short-term memory neural network Bi-LSTM to perform time correlation prediction learning and obtain the time correlation prediction result X 3; S5, the time correlation prediction result X 3. Send it to the anti-attention layer to reduce the attention on the pre-filled data to obtain the reconstructed attention matrix X 4; Wherein, the S5 includes: S51. In the bidirectional long short-term memory neural network Bi-LSTM, The output hidden state vector forward at time , the output hidden state vector of the backward Perform splicing processing to obtain the splicing vector: (12) In the formula, for t The concatenated vector of time, is the forward output hidden state vector at the first moment, is the forward output hidden state vector at the second moment, For the t The forward output hidden state vector at time t, is the backward output hidden state vector at the first moment, is the backward output hidden state vector at the second moment, for t The backward output hidden state vector at time t, is the component of the splicing vector at the first moment, is the component of the splicing vector at the second moment, For the t The components of the splicing vector at time instant; S52: Perform attention scoring on each of the splicing vectors to obtain an initial attention score g t : (13) S53, according to the attenuation matrix D , using the following logic, calculate the vector missing degree , vector integrity : (14) (15) In the formula, N is the dimension of the data vector at the current moment in the attenuation matrix, is the attenuation matrix t Moment h The data value of the row, The attenuation matrix is ​​described in t Time 1st row to h The accumulation of row data values; S54. Re-score the initial attention score according to the vector missing degree and the vector completeness, so as to calculate a new attention score for each of the concatenated vectors using the following logic: (16) In the formula, for The new attention score obtained at the moment, for The initial attention score at time t, For the t The initial attention score at time t, For the t The degree of missingness of the vector at time, for The vector integrity at time, Represents the time series divided by The accumulation of the vector integrity at other times except the time; S55, use the softmax function to process the time step n Attention distribution ; S56. Calculate the proportion of the input sequence in the total sequence according to the attention distribution and the concatenation vector through the new attention mechanism. ; S57, assigning different attentions to different time steps in the input sequence to select applicable prediction information for the current task target, and constructing the reconstructed attention matrix X 4; S6, reconstructing the attention matrix X 4 Put it into the fully connected output layer and process it to get the data prediction completion result X 5.

2. A time series data completion method according to claim 1, characterized in that: The S2 includes: S21, for the original time series defective data set X 0, find the incomplete matrix to be completed A , to calculate the temporal correlation matrix and mark missing values ​​NaN using the following logic: (1) In the formula, Represents the incomplete matrix to be completed A the number of rows, Represents the incomplete matrix to be completed A The number of columns, Represents a one-dimensional column matrix T Corresponding to i The time value of the row, Represents the one-dimensional column matrix T Corresponding to i-1 The time value of the row; Represents the incomplete matrix to be completed A The same type matrix The Row, No. The value of the column, Indicates the time correlation matrix Row, No. The value of the column, Indicates the time correlation matrix Row, No. The value of the column; S22, according to the time correlation matrix C , the attenuation matrix is ​​obtained by the following logical processing D : (2) In the formula, Represents the time correlation matrix C Middle i Row, No. j The value of the column; Denotes the attenuation matrix D No. i Row, No. j The value of the column, e represents the calculation error vector; S23, the incomplete matrix to be completed A Perform pre-filling processing to fill the missing value NaN into the incomplete matrix to be completed according to the moment value of the preset time series A , get the pre-filled matrix : (3) In the formula, Represents the incomplete matrix to be completed A exist t The value of the moment; Represents the pre-filled matrix exist The value of the moment, Represents the pre-filled matrix exist t The value of the moment; S24, horizontally splicing the pre-filled matrix , the attenuation matrix D , get the time-dependent preprocessing matrix .

3. A time series data completion method according to claim 1, characterized in that: In S3, the CNN network structure based on the residual mechanism is used to add an identity mapping between different convolutional layers through the residual network, wherein the residual block output of the convolutional layer is expressed using the following logic: (4) (5) In the formula, represents the residual block output, Represents the direct mapping part, l represents the number of layers of the CNN network, represents the residual mapping part to be learned, F represents the residual mapping, represents the network layer parameters to be learned, represents the activation function, represents the weight of the first layer of the CNN network, Represents the weights of the second layer of the CNN network.

4. The method for completing time series data according to claim 1, characterized in that: The S4 includes: S41, setting model parameters of the bidirectional long short-term memory neural network Bi-LSTM; S42, integrating the matrix X 2 is put into the 2-layer reverse LSTM of the bidirectional long short-term memory neural network Bi-LSTM, and in the integration matrix X 2, bidirectional prediction learning is performed at different time points of the data sequence in different directions, and the time correlation prediction results are obtained by processing X 3.

5. The method for completing time series data according to claim 1, characterized in that: In S42, in the bidirectional long short-term memory neural network Bi-LSTM, the following logic is used to calculate t The output hidden state vector at time: (6) (7) (8) (9) (10) (11) In the formula, for t The door of forgetfulness of time, For the t The input gate of the moment, For the t The output gate at time, For the t The cell state at a moment in time, For the t The memory cells of time, for Memory cells of the moment; For the t The output hidden state vector at time t, for The output hidden state vector at time t, is the activation function, tanh is the hyperbolic tangent activation function, is the weight matrix corresponding to the forget gate, is the weight matrix corresponding to the input gate, is the weight matrix corresponding to the output gate, For the t Cell state at a moment The weight matrix of is the bias vector corresponding to the forget gate, is the bias vector corresponding to the input gate, is the bias vector corresponding to the output gate, For the t Cell state at a moment The bias vector of For the t The input vector at time instant.

6. A time series data completion method according to claim 5, characterized in that: In S55, the following logic is used to process the time step n Attention distribution : (17) In the formula, is the time step n The corresponding new attention score, Represents the activation function.

7. A time series data completion method according to claim 5, characterized in that: In S56, the following logic is used to process and obtain the proportion of the input sequence in the total sequence: : (18) In the formula, The amount of input data is k The attention distribution at The amount of input data is k The corresponding splicing vector is, Indicates that the amount of input data is k The attention distribution and the input data volume when k The product of the corresponding splicing vectors is n The sum of the time steps.

8. The method for completing time series data according to claim 1, characterized in that: In S6, the following logic is used to adjust the network parameters of the bidirectional long short-term memory neural network Bi-LSTM in the fully connected output layer, perform optimization training, and update the parameters using the mean square error: (20) (21) In the formula, IN is the activation function The input value of For the l1 Fully connected layers with training weights, For the l2 Fully connected layers with training weights, The amount of input data is k The output prediction value of is the proportion of the input sequence in the total sequence , represents the sum of the products of the training weights of the first fully connected layer and the proportion of the input sequence in the total sequence, represents the sum of the products of the training weights of the second fully connected layer and the proportion of the input sequence in the total sequence, n Represents the time step.

9. A time series data completion system, characterized in that: A system for executing a time series data completion method according to any one of claims 1 to 8, comprising: The data set construction and division module is used to collect and obtain timing-related operation data for DC converter transformers and construct the original timing defect data set X 0, the original time series defective data set X 0 is divided into training set and test set; A time correlation preprocessing module is used to process the original time series defective data set. X 0, perform time correlation preprocessing operation to obtain the time correlation preprocessing matrix X 1. The time correlation preprocessing module is connected to the data set construction and partitioning module; A sequence integration module is used to convert the time-dependent preprocessing matrix X 1 Input into the CNN network structure, perform data sequence integration operation, and obtain the integration matrix X 2. The sequence integration module is connected to the time correlation preprocessing module; A time correlation prediction learning module is used to integrate the integration matrix X 2. Send it to the bidirectional long short-term memory neural network Bi-LSTM to perform time correlation prediction learning and obtain the time correlation prediction result X 3. The time correlation prediction learning module is connected to the sequence integration module; Reduce the attention module to predict the time correlation results X 3. Send it to the anti-attention layer to reduce the attention on the pre-filled data to obtain the reconstructed attention matrix X 4. The attention reduction module is connected to the time correlation prediction learning module; A prediction completion module is used to reconstruct the attention matrix X 4 Put it into the fully connected output layer and process it to get the data prediction completion result X 5. The prediction completion module is connected to the attention reduction module.

Images