Transformer prediction method based on mha-cnn-slstm and error compensation

Abstract

The transformer prediction method based on MHA-CNN-SLSTM and error compensation comprises the following steps: decomposing transformer error data into different high and low frequency components by VMD; optimizing the key parameters of VMD based on the improved seagull optimization algorithm ISOA to realize optimal decomposition of error data; cross processing the error influence characteristics based on the multi-head attention mechanism MHA to mine the correlation between features, and establishing the deep connection between weakly correlated features and errors through the relationship between strongly correlated features and errors; placing CNN at the back end of the multi-head attention mechanism MHA to reduce the feature dimension of the multi-head attention mechanism MHA output; improving the LSTM neural network by considering the correlation between the training set and the test set, dynamically adjusting the network weight and bias, and constructing a MHA-CNN-SLSTM combined prediction model; taking the error between the predicted value and the actual value as the input of the combined prediction model constructed by the training set to generate compensation data for compensating the predicted value. The method can accurately predict the future short-term error state change of the transformer, and has the characteristics of high prediction accuracy and strong generalization.

Description

Technical Field

[0001] This invention relates to the field of current transformer verification technology, specifically to a current transformer prediction method based on MHA-CNN-SLSTM and error compensation. Background Technology

[0002] With the rapid development of power systems, the value of power data is becoming increasingly prominent in areas such as electricity trading and settlement, power grid planning, and security protection. Instrument transformers, as commonly used metering devices, are crucial for the safe and stable operation of the power grid due to their measurement accuracy. According to my country's standards for periodic calibration of measuring equipment, instrument transformers need to undergo periodic accuracy calibration to ensure that their measurement errors meet the corresponding accuracy requirements. Traditional calibration methods require connecting the standard instrument and the device under test to the same circuit, which consumes a significant amount of manpower and resources, especially for high-voltage lines, where there are difficulties such as power outages and inconvenient transportation of calibration equipment. While live-line calibration can effectively solve these problems, this method poses significant safety risks. Therefore, there is an urgent need to research a method for predicting the future error state of instrument transformers to guide the calibration work. Summary of the Invention

[0003] To address the issue of low prediction accuracy in existing current transformer error prediction methods, this invention provides a current transformer prediction method based on MHA-CNN-SLSTM and error compensation. This method can accurately predict changes in the error state of current transformers in the short term and features high prediction accuracy and strong generalization ability.

[0004] The technical solution adopted in this invention is as follows:

[0005] The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation includes the following steps:

[0006] Step 1: Use the Variational Mode Decomposition (VMD) algorithm to decompose the original instrument transformer error data into different high and low frequency components;

[0007] Step 2: Optimize the key parameters of the variational mode decomposition algorithm (VMD) based on the improved Seagull Optimization Algorithm (ISOA) to achieve optimal decomposition of error data;

[0008] Step 3: Based on the multi-head attention mechanism (MHA), cross-process the error-affecting features to explore the correlation between each feature. By establishing a deep connection between weakly correlated features and errors through the relationship between strongly correlated features and errors, we can avoid the reduction in prediction accuracy due to data waste.

[0009] Step 4: Place the Convolutional Neural Network (CNN) at the back end of the Multi-Head Attention (MHA) mechanism to reduce the dimensionality of the MHA output features, reduce training parameters, and improve the model's computation speed.

[0010] Step 5: Consider the correlation between the training set and the test set, improve the LSTM neural network, dynamically adjust the network weights and biases, and construct the MHA-CNN-SLSTM combined prediction model.

[0011] Step 6: Use the error between the predicted and actual values ​​as a training set, and input it again into the combined prediction model constructed in Step 5 to generate compensation data to compensate for the predicted values.

[0012] In step 1, the variational mode decomposition (VMD) algorithm is used to decompose the original transformer error data into different high and low frequency components.

[0013] Based on the magnitude of the zero-crossing rate, high and low frequency components are distinguished. The results of VMD decomposition of the transformer error data are as follows: Figure 2 As shown in Table 1, the zero-crossing rates of each component are calculated. Based on practical experience, the threshold for high and low frequency components is set to 0.083. As shown in Table 1, IMF8-IMF3 are high frequency components, and IMF2-IMF1 are low frequency components.

[0014] The formula for calculating the zero-crossing rate is as follows:

[0015]

[0016] In the formula, P is the magnitude of the zero-crossing rate; Z zero The signal passes through zero the most times; N is the interval length.

[0017] Table 1 Zero-crossing rate of each component

[0018]

[0019] In step 2, the traditional Seagull algorithm is improved by introducing an opposition search operator strategy and a nonlinear convergence control factor. An improved Seagull optimization algorithm (ISOA) is proposed, which selects permutation entropy as the fitness function and uses the improved Seagull optimization algorithm ISOA to optimize the key parameters of the variational mode decomposition algorithm VMD, and optimizes the number of decompositions k and the penalty factor α to achieve the goal of optimal decomposition of error data.

[0020] Step 2 is as follows:

[0021] First, we introduce an opposition search operator strategy to optimize the initial population by generating the current individual and its opposite individual on the interval [L,U], defined as:

[0022]

[0023] In equation (1), Indicates the individual that is opposed to the current individual; x i L represents the current individual; U represents the lower value within the population optimization interval; L represents the upper value within the population optimization interval.

[0024] Based on the above definition, the following initialization method is adopted: set the population size N, compare the current individual with its relative individual, and select the individual with the best fitness value as the initial population;

[0025] The initialization formula is:

[0026]

[0027] In equation (2), This represents the value obtained by substituting the current individual into the fitness function; f(x) i () represents the value obtained by substituting the current individual into the fitness function;

[0028] Then, the nonlinear convergence control factor is improved by adjusting the control factor f. c A redesign was carried out to make the control factor f c The control factor f initially increases with the number of iterations, and after reaching a certain value, it decreases. c It rapidly decreases to a small value, and then gradually increases at a slow rate; based on this, the control factor f c The updated formula is shown below:

[0029] f c =(f ci -f cf )+[(1-t / t max ) / (1-η×t / t max )](3);

[0030] In equation (3), f ci For control factor f c Initial value; f cf For control factor f c Final value; η is the adjustment coefficient; t is the current iteration number; t max This represents the maximum number of iterations.

[0031] In step 2, the key parameters of the variational mode decomposition algorithm VMD are optimized, namely the number of decompositions k and the penalty factor α in VMD, to achieve optimal decomposition of error data.

[0032] Optimal decomposition of error data refers to adaptively determining the optimal k and α in VMD based on the actual data conditions, obtaining high- and low-frequency components with high time-frequency resolution and no modal aliasing, such as... Figure 2 As shown, the decomposition results do not exhibit problems such as modal aliasing or low time-frequency.

[0033] In step 3, based on Pearson correlation coefficient analysis, the factors with a very strong correlation to the ratio difference are temperature and humidity; the next weakly correlated factors are load and vibration; and the last weakly correlated factor is magnetic field. There are certain connections between ratio difference and temperature, and humidity and load. The deep relationship between ratio difference and load can be explored through humidity and temperature characteristics. A multi-head attention mechanism (MHA) is used to cross-process the characteristics of each influencing factor to uncover the deep relationships between these characteristics.

[0034] Step 3 is as follows:

[0035] The Multi-Head Attention (MHA) mechanism assigns different weights to each feature through a value matrix, explores multiple relationships between features, and achieves the cross-interaction of explicit features and implicit features with high correlation coefficients by fusing features from multiple different attention heads, forming new features with richer expressive power compared to the original features.

[0036] Error Influencing Factors Data Matrix Z and Query Linear Projection Weights W q Multiplying them together yields the query matrix Q:

[0037] Q = W q Z

[0038] Error Influencing Factors Data Matrix Z and Bond Linear Projection Weights W k Multiplying them together, we get the bond matrix K:

[0039] K = W k Z

[0040] Error influencing factors data matrix Z and value linear projection weight W v Multiplying them together yields the bond matrix V:

[0041] V = W v Z

[0042] In the above formula, matrices Q and V are used to calculate similarity through dot product operation to uncover the correlation between features and capture multiple internal relationships, as shown below:

[0043]

[0044] In equation (6), F is a self-attention network, and σ is the Softmax activation function; F(Q,K,V) represents the scaling factor; F(Q,K,V) represents the output value of the self-attention network after substituting Q, K, and V; K T d represents the transpose of the bond matrix k Indicate the vector dimensions in Q and K;

[0045] The computations of each sub-attention head within a multi-head attention mechanism are independent of each other. This is achieved by concatenating the output matrices of each sub-attention head and multiplying them by the weight matrix W. oThe final output is shown below:

[0046]

[0047] In equation (7), H i For the multi-head attention of the i-th head; W i Q W i K W i V F represents the weight of the i-th attention head. c Here is the matrix concatenation function; H1, H2, ..., H i This represents the i-th head in a multi-head attention mechanism; This represents the final output matrix of the multi-head attention mechanism.

[0048] To ensure information transmission and gradient stability, according to equation (8) Connect it to the Z-jump and perform normalization.

[0049]

[0050] In the formula, M is the normalized result; F LN The normalization function is LayerNorm. Z represents the final multi-head attention mechanism output error matrix; Z represents the original error matrix.

[0051] In step 3, the feature intersection diagram is as follows: Figure 3 As shown, specifically, the multi-head attention mechanism calculates the original features of different error influencing factors through its internal sub-attention heads, and then dynamically weights and merges these features to obtain new error influencing factor features.

[0052] After feature crossover, the new feature x3 contains not only the independent features of x1 and x2, but also the relationship between them. That is, the relationship between x1 and y can indirectly show the deep relationship between x2 and y. Therefore, the combined prediction model can learn deeper nonlinear features through training with the new features, increasing the fitting accuracy.

[0053] In step 4, the convolutional neural network (CNN) is placed at the back end of the multi-head attention mechanism (MHA) to reduce the dimension of the output features of the MHA.

[0054] A Convolutional Neural Network (CNN) mainly consists of two parts: convolutional layers and pooling layers. The convolutional layers are composed of filters, which can be considered as multiple stacked convolutional kernels. The filters slide across the input data to perform convolution operations, extracting features from the input data and reducing data dimensionality. The degree of dimensionality reduction depends on the filter size. Based on the characteristics of the mutual inductor data, this invention employs a one-dimensional CNN, where the principle of the convolutional layer is as follows: Figure 4 As shown.

[0055] In step 5, to ensure the accuracy of the predicted error value of the mutual inductor output by the LSTM neural network model, a squared loss function is used to update the weight and bias matrix parameters:

[0056]

[0057] In equation (10), J represents the loss value; h(t) represents the predicted error value; m(t) is the actual error result; γ is the regularization parameter; and N is the number of samples. The updated matrix; W T arg represents the transpose of matrix W; W,b This represents the minimum squared loss of matrices W and b;

[0058] W = [W f W i W c W o ];W f W i W c W o These represent the weight matrices for the forget gate, input gate, memory unit, and output gate, respectively.

[0059] b = [b f ,b i ,b c ,b o ];b f ,b i ,b c ,b o These represent the bias matrices for the forget gate, input gate, memory unit, and output gate, respectively.

[0060] Cosine similarity is used to compare the reliability of the error between the training sample output value and the test set output value, and W is dynamically adjusted.

[0061]

[0062] In equation (11), R t,η S represents cosine similarity; t,η For R t,η The result of the nonlinear transformation takes values ​​between [0,1].

[0063] x(t) is the input to the training set at time t; z(η) is the input to the test set at time η; x(t) T Let x(t) be the transpose matrix; exp(-R) t,η ) represents the exponential function value of cosine similarity.

[0064] Combined with formulas (10) and (11) for adjustment

[0065]

[0066] In equation (12), Let z(η) be the output of the test set model at time η; h(t) be the output of x(t) at time t; J η This represents the loss value at time η; This represents the weight matrix after the update at time η; This represents the bias matrix after the update at time η; This represents the minimum squared loss of matrices W and b at time η.

[0067] The mathematical equations of the LSTM neural network can be further described as follows:

[0068]

[0069] In equation (13), f η (t) represents the output of the forget gate in the test set; W f,η This represents the forget gate weight matrix at time η in the test set; b represents the output value of the test set model at the previous moment; f,η The forget gate bias matrix σ(i) represents the test set at time η; σ(i) represents the activation function; i(t) represents the input gate output; W i,η b represents the input gate weight matrix at time η. i,η This represents the input gate bias matrix of the forget gate at each time step. Indicates the state of a memory cell; W c,η b represents the weight matrix of the memory unit at time η; c,η The memory cell bias matrix at time η is represented; o(t) represents the initial output; W o,η b represents the output gate weight matrix at time η; o,η Let f(t) represent the output gate bias matrix at time η; c(t) represent the updated memory cell; f(t) represent the forget gate output; and c(t-1) represent the previous memory cell. The operator represents the Hadamard product; h(t) represents the output of the training set at the previous moment; tanh(t) represents the activation function.

[0070] In step 5, the MHA-CNN-SLSTM combined prediction model is formed by combining three units: Multi-Head Attention (MHA), Convolutional Neural Network (CNN), and SLSTM. The network structure is as follows: Figure 5 As shown, MHA is located at the front end of the combined prediction model and is used to perform cross-processing on data features. The results of the cross-processing are input into the CNN to process the data, and then the processed data is input into the SLSTM for prediction.

[0071] In step 5, after the MHA-CNN-SLSTM combined prediction model established in step 5 completes the prediction of the sample data, the internal parameters are all fixed. At the same time, the error of the current transformer prediction value is generated by comparing the predicted value and the actual value output by the test sample. That is, the error should have a high degree of adaptability with the test sample. Therefore, the prediction error can be directly fed into the MHA-CNN-SLSTM combined prediction model for prediction, and a correlation network between the prediction error and the characteristics of the factors affecting the current transformer error can be established to realize the prediction of the prediction error, thereby compensating and correcting the prediction value.

[0072] This invention provides a current transformer prediction method based on MHA-CNN-SLSTM and error compensation, with the following technical advantages:

[0073] 1) In step 1 of the present invention, the VMD method is used to decompose the error data into multiple modal components. The decomposition results can indicate the characteristics of the error data and separate noise interference signals in the data, thereby improving the accuracy of the model input data from the data source.

[0074] 2) In step 2 of the present invention, the traditional Seagull optimization algorithm is improved and the two parameters of the number of decompositions and the penalty factor in VMD are optimized. The introduced opposing search operator strategy and nonlinear convergence control factor can solve the problem that the traditional Seagull optimization algorithm is prone to getting trapped in local optima and early convergence. Therefore, the improved Seagull optimization algorithm is used to optimize the number of decompositions and the penalty factor in VMD. The optimization result has a higher fitness value and is more in line with the actual situation.

[0075] 3) In step 3 of this invention, since the influencing factors of latent errors are not directly reflected in the data during the traditional prediction process, only the influencing factors of explicit errors are often used for model training, ignoring the correlation between the influencing factors of explicit errors and the influencing factors of latent errors. Step 3 uses MHA to cross-process the features of the influencing factors of errors, and indirectly describes the relationship between the influencing factors of latent errors and errors through the explicit influencing factors of errors. This effectively improves the learning ability for nonlinear features and deep latent features, and can improve the prediction accuracy of the model.

[0076] 4) In step 4 of the present invention, since the feature dimension increases after the MHA feature cross-processing, the excessive feature dimension will affect the model calculation speed. The present invention uses the filter structure characteristics of CNN itself to process the cross-processed features, thereby reducing the data dimension and increasing the calculation speed.

[0077] 5) In step 5 of this invention, the internal parameters are usually not adjusted after the model training is completed, and the test set is directly input into the model to obtain the predicted value. However, in actual application, there is a different degree of correlation between the training set and the test set. In order to improve the generalization ability of the model in actual operation, step 5 improves the LSTM. After the improvement, the model can deeply analyze the correlation between the training samples and the test set. On the basis of the completed training model, the model parameters are slightly adjusted to provide the most adaptive prediction network for each test set, and the model has a stronger generalization ability.

[0078] 7) The improved Seagull Optimization Algorithm ISOA in this invention outperforms other optimization algorithms in terms of convergence speed, convergence accuracy, and ability to escape local optima. When the improved Seagull Optimization Algorithm ISOA is used to optimize the key parameters of the Variational Mode Decomposition (VMD) algorithm, no aliasing phenomenon occurs in the decomposed modes, and the optimization time is relatively short.

[0079] 8) In this invention, after cross-processing of the features affecting the transformer error, the correlation between strongly correlated and weakly correlated features can be effectively mined. The SLSTM model improves the adaptability between the training and test sets and dynamically adjusts the network parameters, thereby enabling the MHA-CNN-SLSTM combined prediction model to have high prediction accuracy. Based on this model, the difference between the predicted error value and the actual value is predicted, and the result compensates for the initial prediction value, further improving the model's prediction accuracy. Attached Figure Description

[0080] Figure 1 This is a schematic diagram of the current transformer prediction method based on MHA-CNN-SLSTM and error compensation according to the present invention.

[0081] Figure 2 This is a diagram showing the results of the VMD decomposition in this invention.

[0082] Figure 3 Schematic diagram of feature intersection.

[0083] Figure 4 Schematic diagram of the one-dimensional CNN convolutional computation layer.

[0084] Figure 5 MHA-CNN-SLSTM network structure

[0085] Figure 6 Results of Pearson correlation coefficient analysis.

[0086] Figure 7 This is a comparison chart of the prediction results of the MHA-CNN-SLSTM combined prediction model in this invention. Detailed Implementation

[0087] A current transformer prediction method based on MHA-CNN-SLSTM and error compensation is proposed. To improve the accuracy of current transformer error prediction, an adversarial search operator strategy and a nonlinear convergence control factor are first introduced to improve the traditional Seagull algorithm. A method based on the improved Seagull Optimization algorithm (ISOA) is proposed to optimize the key parameters of Variational Mode Decomposition (VMD) to achieve adaptive decomposition of error data. Then, based on the Multi-Head Attention (MHA) mechanism, the error-affecting features are cross-processed to explore the correlation between various features. By establishing a deep connection between weakly correlated features and errors through the relationship between strongly correlated features and errors, the prediction accuracy is avoided due to data waste. Considering the relationship between the training set and the test set, a Similar Long Short-Term Memory (SLSTM) neural network considering sample similarity is proposed to dynamically adjust the network weights and biases. Finally, an MHA-CNN-SLSTM prediction model is constructed. The error between the predicted value and the actual value is used as the training set and input into the prediction model again to generate compensation data to compensate for the initial prediction value, further improving the prediction value. Finally, the model was verified using actual measured data from a current transformer, and the results showed that the model proposed in this invention has higher prediction accuracy.

[0088] The current transformer prediction method of this invention includes three parts: current transformer error data decomposition, MHA-CNN-SLSTM model construction, and error compensation.

[0089] (I) Decomposition of current transformer error data:

[0090] The transformer error data decomposition section employs Variational Mode Decomposition (VMD) to decompose the original error data into different high- and low-frequency components. Furthermore, an improvement to the traditional Seagull algorithm is introduced by incorporating an opposition search operator strategy and a nonlinear convergence control factor. A new algorithm based on the Improved Seagull Optimization (ISOA) is proposed, selecting permutation entropy as the fitness function and utilizing ISOA to optimize key VMD parameters, seeking the optimal number of decompositions k and the penalty factor α to achieve the goal of optimal error data decomposition.

[0091] The specific improvement process for ISOA is as follows:

[0092] First, we introduce a contrastive search operator strategy to optimize the initial population by generating the current individual and its opposite individual on the interval [L,U], defined as:

[0093]

[0094] In the formula, x i Indicates the current individual; It refers to the individual that is opposed to the current individual.

[0095] Based on the above definition, the following initialization method is adopted: set the population size N, compare the current individual with its relative individual, and select the individual with the best fitness value as the initial population.

[0096] The initialization formula is:

[0097]

[0098] The Seagull optimization algorithm is highly dependent on the control factor. Therefore, this invention improves the nonlinear convergence control factor, specifically the control factor f. c To redesign, the expectation is f c The value of f initially increases with the number of iterations; after reaching a certain value, f... c It is rapidly reduced to a small value, and then gradually increased at a slow rate.

[0099] Based on this, the control factor f c The updated formula is shown below:

[0100] f c =(f ci -f cf )+[(1-t / t max ) / (1-η×t / t max )](3);

[0101] In the formula, f ci For control factor f c Initial value; f cf For control factor f c Final value; η is the adjustment coefficient; t is the current iteration number; t max This represents the maximum number of iterations.

[0102] The specific process of the ISOA optimization algorithm is as follows: Figure 1 The ISOA optimization flowchart is shown on the left.

[0103] (II) Prediction model based on MHA-CNN-SLSTM combination:

[0104] External factors such as load, temperature, and humidity can affect the error of instrument transformers. Therefore, the prediction model needs to input not only the high and low frequency components of the decomposed instrument transformer error data, but also the characteristic parameters of each influencing factor. The Pearson correlation coefficient, ranging from -1 to 1, is widely used to measure the correlation between variables; a larger absolute value indicates a stronger correlation between the instrument transformer error value and the characteristic of that influencing factor. This invention performs correlation analysis on one month's monitoring data of a specific instrument transformer, using the ratio difference in the error as an example.

[0105] Correlation analysis was performed on the ratio difference using the Pearson correlation coefficient method, focusing on factors such as temperature, humidity, load, vibration, and magnetic field. The results are as follows: Figure 6 As shown.

[0106] It can be seen that the factors with a very strong correlation to the specific difference are temperature (-0.854) and humidity (0.815), followed by the factors with a weak correlation, namely load (0.252) and vibration (-0.225), and finally the factor with a very weak correlation, namely magnetic field (-0.169).

[0107] Further analysis reveals that there are not only direct relationships between the ratio difference and the characteristics of different influencing factors, but also many hidden indirect and deeper relationships. For example, although the ratio difference is strongly correlated with humidity and temperature, and weakly correlated with load, humidity and temperature are moderately correlated with load. That is, there is a certain connection between the ratio difference and temperature, and between humidity and load. The deeper relationship between the ratio difference and load can be explored through the characteristics of humidity and temperature.

[0108] Therefore, if, based on past experience, we directly select strongly correlated influencing factors as input parameters into the model according to the correlation coefficient ranking results, while ignoring weakly correlated factors such as load, vibration, and magnetic field, we will be unable to accurately establish the relationship between transformer error and influencing factor characteristics, which may lead to a decrease in prediction accuracy.

[0109] Therefore, this invention employs a multi-head attention mechanism to cross-process the features of various influencing factors and uncover the deep-seated relationships between these features.

[0110] (1) Multi-Head Attention (MHA):

[0111] Suppose that the predicted value and the feature value have the following relationship:

[0112] y = ω1x1 + ω2x2....ω n x n +b(4);

[0113] In the formula, y is the predicted value; ω n For the corresponding feature x n Weighting coefficients; b is a constant term.

[0114] If only two influencing factors are selected for analysis, y shows a strong correlation with x1 and a weak correlation with x2, while x1 and x2 show a moderate correlation. Based on the above analysis results, it can be seen that directly inputting x1 into the prediction model for training will not be able to learn the deep relationship between features. Therefore, the features need to be cross-processed.

[0115] Multi-head attention mechanisms assign different weights to each feature based on the model's own characteristics, uncover multiple relationships between features, and achieve cross-linking of explicit features with large correlation coefficients by fusing features from multiple different attention heads, forming new features with richer expressive power compared to the original features.

[0116] Matrix Z and weight matrix W q W k W v Multiplying them yields the query / key / value matrix (Q / K / V):

[0117] Q = W q Z, K = W k Z, V = W v Z(5);

[0118] Matrix Q and V are similar to each other by dot product operation to uncover the correlation between features and capture multiple internal relationships, as shown below:

[0119]

[0120] In the formula, F is a self-attention network, and σ is the Softmax activation function; This is the scaling factor.

[0121] The computations of each sub-attention head within a multi-head attention mechanism are independent of each other. This is achieved by concatenating the output matrices of each sub-attention head and multiplying them by the weight matrix W. o The final output is shown below:

[0122]

[0123] In the formula, H i For the multi-head attention of the i-th head; W i Q W i K W i V F represents the weight of the i-th attention head. c This is a matrix concatenation function.

[0124] To ensure information transmission and gradient stability, according to equation (8) Connect it to the Z-jump and perform normalization.

[0125]

[0126] In the formula, M is the normalized result; F LN This is the normalization function (LayerNorm).

[0127] After feature crossing, the new feature x3 contains not only the independent features of x1 and x2, but also the relationship between them. That is, the relationship between x1 and y can indirectly show the deep relationship between x2 and y. Therefore, the prediction model learns deeper nonlinear features through training with the new feature, increasing the fitting accuracy.

[0128] (2) Convolutional Neural Networks (CNN):

[0129] The new features formed after feature crossover based on the multi-head attention mechanism have an increased feature dimensionality compared to the original features. Excessive feature dimensionality can negatively impact model computation speed. Therefore, this invention employs CNNs to reduce feature dimensionality, decrease training parameters, and improve model computation speed.

[0130] A CNN mainly consists of two parts: convolutional layers and pooling layers. The convolutional layers are composed of filters, which can be considered as multiple stacked convolutional kernels. The filters slide across the input data to perform convolution operations, extracting features from the input data and reducing data dimensionality. The degree of dimensionality reduction depends on the filter size. Based on the characteristics of the mutual inductor data, this invention uses a one-dimensional CNN, where the principle of the convolutional layers is as follows: Figure 4 As shown, a filter with a dimension smaller than the input data moves within the input data along a fixed step size (the step size is usually designed according to the actual situation), performs convolution operations on the data within the range of the filter's movement, and uses the result as the output for downscaling.

[0131] (3) Similar Long Short-Term Memory (SLSTM) Neural Network for Sample Similarity:

[0132] Long Short-Term Memory (LSTM) neural networks, as a special type of recurrent neural network, can effectively transmit and represent information in time series and achieve predictions about time series. However, traditional LSTM...

[0133] The inability to dynamically adjust weights and bias matrices in the model, coupled with the lack of consideration for the correlation between the training and test sets, reduces the accuracy of the transformer error prediction to some extent. Therefore, this invention improves upon the traditional LSTM.

[0134] The mathematical expression for the LSTM model is:

[0135]

[0136] In the formula, x(t) is the input value at the current time; f(t), i(t), and o(t) are the outputs of the forget gate, input gate, and output gate, respectively; W f W i W c W o b f b i b c b o These represent their respective weights and bias matrices; c(t) represents the network memory unit; c t-1 (t) represents the network memory unit before the update; h(t) represents the state of the memory cell; h(t) represents the output value at the current time. t-1 (t) represents the model output value at the previous time step; σ and tanh are activation functions.

[0137] To ensure the accuracy of the model's output transformer error prediction values, a squared loss function is used to update the weight and bias matrix parameters:

[0138]

[0139] In the formula, m(t) is the actual error result; γ is the regularization parameter; N is the number of samples; W = [W f W i W c W o ];b=[b f ,b i ,b c ,b o ]; This is the updated matrix.

[0140] Typically, once a model is trained, its internal parameters are not adjusted further, and the test set is directly input into the model to obtain predicted values. However, in practical applications, there are varying degrees of correlation between the training and test sets. Therefore, this invention uses cosine similarity to compare the reliability of the error between the training sample output values ​​and the test set output values, and dynamically adjusts the model accordingly.

[0141]

[0142] In the formula, x(t) is the input of the training set at time t; z(η) is the input of the test set at time η; S t,η For R t,η The result of the nonlinear transformation takes values ​​between [0,1].

[0143] Combined with formulas (10) and (11) for adjustment

[0144]

[0145] In the formula, z(η) is the output of the test set model at time η; h(t) is the output of the training set model at time t.

[0146] The mathematical equations of the LSTM model can then be further rewritten as follows:

[0147]

[0148] The SLSTM calculation process is as follows: Figure 1 The SLSTM flowchart is shown on the right.

[0149] This model is combined with the models described in (1) and (2) to form the MHA-CNN-SLSTM combined prediction model, which is used to predict the error of the mutual inductor. The prediction model framework is as follows: Figure 1 The prediction process based on MHA-CNN-SLSTM is shown in the figure.

[0150] (III) Error Compensation:

[0151] The basic idea of ​​error compensation is to generate compensation data based on the prediction results and influencing factors to offset the error in the prediction value, thereby further improving the prediction accuracy.

[0152] After the prediction model established above completes the prediction of the sample data, the internal parameters are all fixed. At the same time, the error of the current transformer prediction value is generated by comparing the predicted value and the actual value output by the test sample. That is, the error should have a high degree of adaptability with the test sample. Therefore, the prediction error can be directly fed into the model for prediction, and a correlation network between the prediction error and the characteristics of the factors affecting the current transformer error can be established to realize the prediction of the prediction error, thereby compensating for and correcting the prediction value.

[0153] Verification Example:

[0154] This invention uses actual data on the ratio difference of a current transformer in a substation over 30 days, as well as data on environmental factors such as temperature, humidity, load, vibration, and magnetic field, totaling 4320 sampling points. The first 27 days are used as training sample data, and the last 3 days are used as test sample data. The data are normalized to the range [-1, 1]. The proposed method is verified by using a single-step prediction of the current transformer ratio difference in the near future.

[0155] To intuitively represent the model's predictive performance, this invention uses Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Squared Error (MSE) as evaluation metrics for the prediction results.

[0156]

[0157] In the formula, N is the number of samples; y i , These represent the actual error value and the predicted error value, respectively.

[0158] Validation based on the MHA-CNN-SLSTM combined prediction model

[0159] The results of VMD decomposition of the difference data are as follows Figure 2 As shown, from Figure 2 It can be seen that after ISOA optimization, the number of VMD decomposition layers k=8 was determined, and no modal aliasing or low time-frequency resolution issues were found in each component.

[0160] To verify the accuracy of the prediction results of the proposed combined model, it was compared with other combined models, and the results are as follows: Figure 7 As shown. By Figure 7 It can be seen that the model proposed in this invention is closer to the true difference than the convolutional neural network-self-attention encoder-decoder-residual optimization (CNN-SAEDN-RES) and convolutional neural network-long short-term memory neural network (CNN-LSTM). At the same time, the MHA-CNN-SLSTM after error compensation deviates less from the true value and the prediction accuracy is significantly improved, as can also be confirmed by Table 2.

[0161] Table 2 Comparison of Evaluation Indicators for Different Combination Prediction Models

[0162]

[0163] As can be seen from the comparison, the computation time of the model proposed in this invention is longer than that of other combined models. The reason for this is that this invention performs feature fusion on the factors affecting the mutual inductor error. Although CNN is used to reduce the dimensionality of the fused features output by the multi-head self-attention mechanism, the amount of data computation is still large compared to other models. Fortunately, the computation speed is still within an acceptable range.

[0164] In terms of prediction accuracy, the model proposed in this invention has high prediction accuracy, as shown by the evaluation index e. MAEFor example, the MHA-CNN-SLSTM model reduced the accuracy by 7.87% and 5.9% compared to CNN-LSTM and CNN-SAEDN-RES, respectively. After error compensation, the model accuracy was further improved, and the evaluation index decreased by 15.12% and 13.3%, which fully demonstrates the superiority of the combined model and error compensation proposed in this invention and helps to improve the accuracy of mutual inductor ratio difference prediction.

Claims

1. A current transformer prediction method based on MHA-CNN-SLSTM and error compensation, characterized in that... Includes the following steps: Step 1: Use the Variational Mode Decomposition (VMD) algorithm to decompose the current transformer error data into different high and low frequency components; Step 2: Optimize the key parameters of the variational mode decomposition algorithm (VMD) based on the improved Seagull Optimization Algorithm (ISOA) to achieve optimal decomposition of error data; Step 3: Based on the multi-head attention mechanism (MHA), cross-process the error impact features to explore the correlation between each feature, and establish a deep connection between weakly correlated features and the error through the relationship between strongly correlated features and the error; Step 4: Place the Convolutional Neural Network (CNN) at the back end of the Multi-Head Attention (MHA) mechanism to reduce the dimensionality of the MHA output features; Step 5: Consider the correlation between the training set and the test set, improve the LSTM neural network, dynamically adjust the network weights and biases, and construct the MHA-CNN-SLSTM combined prediction model. Step 6: Use the error between the predicted value and the actual value as a training set, input it into the combined prediction model constructed in Step 5, and generate compensation data to compensate for the predicted value; In step 5, to ensure the accuracy of the predicted error value of the mutual inductor output by the LSTM neural network model, a squared loss function is used to update the weight and bias matrix parameters: （10）； In equation (10), Indicates the loss value; This represents the predicted error value; This represents the actual result of the error. For regularization parameters; The number of samples; , This is the updated matrix; Representation matrix transpose; Representation matrix Minimum of the squared loss of b; ; These represent the weight matrices for the forget gate, input gate, memory unit, and output gate, respectively. ; These represent the bias matrices for the forget gate, input gate, memory unit, and output gate, respectively. Cosine similarity is used to compare the reliability of the error between the output values ​​of the training samples and the output values ​​of the test set, and the results are dynamically adjusted. , ; （11）； In equation (11), Indicates cosine similarity; To The result of the nonlinear change takes the value located in between; for Time-based training set input; for Time-test set input; express The transpose of the matrix; The exponential function value representing cosine similarity; Combined with formulas (10) and (11) for adjustment , : （12）； In equation (12), for exist The model output results at each time step of the test set; express Time-based loss value; express The weight matrix is ​​updated at each time step; express The bias matrix is ​​updated at each step; Representation matrix , b exist The minimum squared loss at time t.

2. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In step 1, the variational mode decomposition algorithm (VMD) is used to decompose the transformer error data into different high and low frequency components. Based on the magnitude of the zero-crossing rate, high and low frequency components are distinguished. The formula for calculating the zero-crossing rate is as follows: ； In the formula, The magnitude of the zero-crossing rate; The number of times the signal passes through zero; is the interval length.

3. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In step 2, the traditional Seagull Algorithm is improved by introducing an opposition search operator strategy and a nonlinear convergence control factor, proposing an improved Seagull optimization algorithm, ISOA. Permutation entropy is chosen as the fitness function, and the key parameters of the Variational Mode Decomposition (VMD) algorithm are optimized using the improved ISOA algorithm, specifically the number of decompositions. and penalty factor The goal is to find the optimal solution for the error data.

4. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 3, characterized in that: Step 2 is as follows: First, an opposing search operator strategy is introduced to optimize the initial population within the interval. The current individual and its opposite individual are generated above, defined as: （1）； In equation (1), This refers to the individual that is opposed to the current individual; Indicates the current individual; Indicates the lower value within the population optimization interval; Indicates the upper value within the population optimization interval; Based on the above definition, the following initialization method is adopted: set the population size. Each individual is compared with its relative individual, and the individual with the best fitness value is selected as the initial population. The initialization formula is: （2）； In equation (2), This represents the value obtained by substituting the current individual into the fitness function; This represents the value obtained by substituting the current individual into the fitness function; Then, the nonlinear convergence control factor was improved, and the control factor was adjusted. Redesign to make the control factor The control factor initially increases with the number of iterations, and after reaching a certain value, it decreases. The control factor is rapidly reduced to a small value and then gradually increased at a slow rate. The updated formula is shown below: （3）； In equation (3), Control factor Initial value; Control factor Final value; This is the adjustment coefficient; This represents the current iteration number. This represents the maximum number of iterations.

5. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 4, characterized in that: In step 2, the key parameter of the variational mode decomposition algorithm (VMD) is optimized, namely, the number of decompositions in VMD. k and penalty factor This achieves optimal decomposition of error data.

6. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In step 3, based on the Pearson correlation coefficient analysis, the factors with a very strong correlation to the ratio difference are temperature and humidity; the next weakly correlated factors are load and vibration; and the last weakly correlated factor is magnetic field. There are certain connections between ratio difference and temperature, and humidity and load. The deep relationship between ratio difference and load can be explored through the characteristics of humidity and temperature. The multi-head attention mechanism (MHA) is used to cross-process the characteristics of each influencing factor to explore the deep relationship between each characteristic.

7. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 6, characterized in that: Step 3 is as follows: The Multi-Head Attention (MHA) mechanism assigns different weights to each feature through a matrix, explores multiple relationships between features, and achieves the cross-interaction of explicit features and implicit features with high correlation coefficients by fusing features from multiple different attention heads, forming new features with richer expressive power compared to the original features. Error Influencing Factors Data Matrix With query linear projection weights Multiplying them together yields the query matrix Q: ； Error Influencing Factors Data Matrix With bond linear projection weights Multiplying yields the key matrix. K : ； Error Influencing Factors Data Matrix AND-valued linear projection weights Multiplying yields the key matrix. V : ； In the above formula, the matrix , Similarity is calculated using dot product operations to uncover correlations between features and capture multiple internal relationships, as shown below: （6）； In equation (6), For self-attention networks, Use the Softmax activation function; This is the scaling factor; This represents the output values ​​of the self-attention network when Q, K, and V are substituted into it. Represents the transpose of the key matrix; Indicate the vector dimensions in Q and K; The computations of each sub-attention head within a multi-head attention mechanism are independent. This is achieved by concatenating the output matrices of each sub-attention head and multiplying them by a weight matrix. The final output is shown below: （7）； In equation (7), For the attention of the bulls Size; , , For the first Each attention head weight; This is a matrix concatenation function; This represents the i-th head in a multi-head attention mechanism; This represents the final output matrix of the multi-head attention mechanism; To ensure information transmission and gradient stability, according to equation (8) and Skip connections and perform normalization; （8）； In the formula, This is the result of normalization; This is the normalization function.

8. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In step 3, feature crossing specifically involves: calculating the original features of different error influencing factors through the sub-attention heads within the multi-head attention mechanism, and dynamically weighting and merging them to obtain new error influencing factor features; After feature intersection, the new feature is obtained. Include , While possessing independent characteristics, it also includes the connection between the two, that is, through... and The relationship between them can be indirectly shown. and Deep connections between them; This is a predicted value; Therefore, the combined prediction model can learn deeper nonlinear features through training with new features, thereby increasing the fitting accuracy.

9. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In step 5, the mathematical equations of the LSTM neural network are further described as follows: （13）； In equation (13), This indicates the output of the forget gate in the test set. Indicates the test set Forget gate weight matrix at different times; This represents the output value of the test set model at the previous moment; Indicates the test set The offset matrix of the time-forget gate; Indicates the activation function; Indicates the input gate output; express Input gate weight matrix at each time step; This represents the input gate bias matrix of the forget gate at each time step. Indicates the state of the memory unit; express Time-mapping unit weight matrix; express Time-of-flight memory unit bias matrix; Indicates the initial output; express Output the gate weight matrix at each time step; express Output gate bias matrix at each time step; Indicates the updated memory unit; Indicates the output of the forget gate; This represents the memory unit from the previous moment; This represents the operator, the Hadamard product; This represents the activation function.

10. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In step 5, after the MHA-CNN-SLSTM combined prediction model established in step 5 completes the prediction of the sample data, the internal parameters are all fixed. At the same time, the error of the current transformer prediction value is generated by comparing the predicted value and the actual value output by the test sample. That is, the error should have a high degree of adaptability with the test sample. Therefore, the prediction error can be directly fed into the MHA-CNN-SLSTM combined prediction model for prediction, and a correlation network between the prediction error and the characteristics of the factors affecting the current transformer error can be established to realize the prediction of the prediction error, thereby compensating and correcting the prediction value.

Images