Mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation
Through the combined prediction model based on MHA-CNN-SLSTM and combined with error compensation technology, the problems of low error prediction accuracy and verification safety hazards are solved, and high-precision transformer error prediction and verification guidance are realized, which improves the power grid Safe and stable operation.
Patent Information
- Application Number
- CN202410872555.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-01
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-01
AI Technical Summary
In the prior art, the prediction accuracy of the transformer error prediction method is low, and the traditional verification method has safety hazards and transportation inconvenient problems, making it difficult to effectively guide the verification work of the transformer.
Using transformer prediction method based on MHA-CNN-SLSTM and error compensation, the combined prediction model is constructed to realize the error state of transformer through variational modal decomposition algorithm, improved Haiou optimization algorithm, multi-head attention mechanism and LSTM neural network. Accurate prediction and compensation.
The accuracy and generalization ability of transformer error prediction are improved, the calculation speed is reduced, the adaptability and prediction accuracy of the model are enhanced, and the manpower and material investment in the verification process is reduced.
Smart Images

Figure CN118965945A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of mutual inductor calibration, and in particular to a mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation. Background Art
[0002] With the rapid development of the power system, the value of power data in the fields of power trading settlement, power grid planning and safety protection is becoming increasingly prominent. As a commonly used metering device, the measurement accuracy of the transformer is crucial to the safe and stable operation of the power grid. According to my country's periodic calibration standards for measuring equipment, transformers need to be regularly calibrated for accuracy to ensure that their measurement errors meet the corresponding accuracy requirements. The traditional calibration method requires the standard instrument and the device under test to be connected to the same circuit, and the calibration requires a lot of manpower and material resources. Especially for high-voltage lines, there are many problems such as power outages and inconvenient transportation of calibration equipment. Although live calibration can effectively solve the above problems, this calibration method has great safety hazards. Therefore, it is urgent to study a method for predicting the future error state of the transformer to guide the calibration of the transformer. Summary of the invention
[0003] Aiming at the problem of low prediction accuracy in the mutual inductor error prediction method in the prior art, the present invention provides a mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation, which can accurately predict the change of mutual inductor error state in the short term in the future, and has the characteristics of high prediction accuracy and strong generalization.
[0004] The technical solution adopted by the present invention is:
[0005] The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation includes the following steps:
[0006] Step 1: Use the variational mode decomposition algorithm VMD to decompose the original transformer error data into different high and low frequency components;
[0007] Step 2: Optimize the key parameters of the variational mode decomposition algorithm VMD based on the improved Seagull optimization algorithm ISOA to achieve the optimal decomposition of error data;
[0008] Step 3: Based on the multi-head attention mechanism MHA, the error-affecting features are cross-processed to explore the correlation between the features, and the deep connection between the weakly correlated features and the error is established through the relationship between the strongly correlated features and the error, so as to avoid the reduction of prediction accuracy due to data waste;
[0009] Step 4: Place the convolutional neural network CNN at the back end of the multi-head attention mechanism MHA to reduce the output feature dimension of the multi-head attention mechanism MHA, reduce training parameters, and increase the model calculation speed;
[0010] Step 5: Considering the correlation between the training set and the test set, improve the LSTM neural network, dynamically adjust the network weights and biases, and build the MHA-CNN-SLSTM combined prediction model;
[0011] Step 6: Use the error between the predicted value and the actual value as the training set, input it again into the combined prediction model constructed in step 5, and generate compensation data to compensate the predicted value.
[0012] In the step 1, the original transformer error data is decomposed into different high and low frequency components by using a variational mode decomposition algorithm (VMD);
[0013] The high and low frequency components are distinguished according to the zero crossing rate. The result of the transformer error data after VMD decomposition is as follows: Figure 2 As shown in Table 1, the zero-crossing rate of each component is calculated respectively. According to practical experience, the threshold of high and low frequency components is set to 0.083. It can be seen from Table 1 that IMF8-IMF3 are high frequency components, and IMF2-IMF1 are low frequency components.
[0014] The formula for calculating the zero-crossing rate is as follows:
[0015]
[0016] Where P is the zero-crossing rate; Z zero is the number of times the signal passes through zero; N is the interval length.
[0017] Table 1 Zero-crossing rate of each component
[0018]
[0019] In the step 2, an adversarial search operator strategy and a nonlinear convergence control factor are introduced to improve the traditional Seagull algorithm, and an improved Seagull Optimization algorithm (ISOA) is proposed. The permutation entropy is selected as the fitness function, and the improved Seagull optimization algorithm ISOA is used to optimize the key parameters of the variational mode decomposition algorithm VMD, and the decomposition number k and the penalty factor α are optimized to achieve the goal of optimal decomposition of error data.
[0020] Step 2 is as follows:
[0021] First, the opposition search operator strategy is introduced to optimize the initial population to generate the current individual and the opposite individual in the interval [L,U], which is defined as:
[0022]
[0023] In formula (1), Indicates the individual opposite to the current individual; x i Represents the current individual; L represents the lower value within the population optimization interval; U represents the upper value within the population optimization interval;
[0024] According to the above definition, the following initialization method is adopted: set the population size N, compare the current individual with its relative individual, and select the individual with the best fitness value as the initial population;
[0025] The initialization formula is:
[0026]
[0027] In formula (2), represents the value obtained by substituting the individual opposite to the current individual into the fitness function; f(x i ) represents the value obtained by substituting the current individual into the fitness function;
[0028] Then, the nonlinear convergence control factor is improved and the control factor f c Redesign so that the control factor f c At the beginning of the algorithm, it increases with the number of iterations. When it reaches a certain value, the control factor f c Rapidly decreases to a smaller value, and then gradually increases at a slow speed; based on this, the control factor f c The update formula is as follows:
[0029] f c =(f ci -f cf )+[(1-t / t max ) / (1-η×t / t max )](3);
[0030] In formula (3), f ci is the control factor f c Starting value; f cf is the control factor f c final value; η is the adjustment coefficient; t is the current number of iterations; t max is the maximum number of iterations.
[0031] In step 2, the key parameters of the variational mode decomposition algorithm VMD are optimized, that is, the number of decompositions k and the penalty factor α in VMD are optimized to achieve the optimal decomposition of the error data.
[0032] The optimal decomposition of error data refers to adaptively determining the best k and α in VMD according to the actual data situation, and obtaining high and low frequency components with high time-frequency resolution and no mode aliasing, such as Figure 2 As shown, the decomposition results do not have the problems of modal aliasing and low time-frequency.
[0033] In step 3, according to the Pearson correlation coefficient analysis, it can be obtained that the factors with the strongest correlation with the ratio difference are temperature and humidity; the second weakly correlated factors are load and vibration; and the last weakly correlated factor is the magnetic field. There is a certain connection between the ratio difference-temperature and humidity-load. The deep relationship between the ratio difference and the load can be explored through the humidity and temperature features; the multi-head attention mechanism MHA is used to cross-process the features of each influencing factor to explore the deep relationship between the features.
[0034] Step 3 is as follows:
[0035] The multi-head attention mechanism MHA assigns different weights to each feature through the value matrix, explores the multiple relationships between features, and realizes the intersection of explicit features and implicit features with large correlation coefficients by fusing features of multiple different attention heads, forming new features with richer expressiveness than the original features;
[0036] Error influencing factor data matrix Z and query linear projection weight W q Multiply them together to get the query matrix Q:
[0037] Q=W q Z
[0038] Error influencing factor data matrix Z and key linear projection weight W k Multiply them together to get the key matrix K:
[0039] K=W k Z
[0040] Error influencing factors data matrix Z and value linear projection weight W v Multiply them together to get the key matrix V:
[0041] V=W v Z
[0042] In the above formula, matrices Q and V calculate similarity through dot product operations, explore the correlation between features, and capture internal multiple relationships, as shown below:
[0043]
[0044] In formula (6), F is the self-attention network, σ is the Softmax activation function; is the scaling factor; F(Q,K,V) represents the output value of Q, K, and V when substituted into the self-attention network; K T represents the transpose of the key matrix d k represents the dimension of vectors in Q and K;
[0045] The calculations between the sub-attention heads in the multi-head attention mechanism are independent of each other. The output matrices of each sub-attention head are concatenated and multiplied by the weight matrix W. o, and the final output is as follows:
[0046]
[0047] In formula (7), H i is the i-th head of multi-head attention; W i Q , W i K , W i V is the weight of the i-th attention head; F c is the matrix concatenation function; H1,H2,........,H i Represents the i-th head of the multi-head attention mechanism; Represents the final multi-head attention mechanism output matrix.
[0048] In order to ensure information transmission and gradient stability, according to formula (8) Concatenate with Z jump and normalize.
[0049]
[0050] Where, M is the normalized result; F LN is the normalization function (LayerNorm); represents the final multi-head attention mechanism output error matrix; Z represents the original error matrix;
[0051] In step 3, the feature cross diagram is as follows Figure 3 As shown, specifically: the original different error influencing factor features are calculated through the sub-attention heads within the multi-head attention mechanism, and the new error influencing factor features are obtained by dynamic weighted calculation and merging.
[0052] After the features are crossed, the new feature x3 contains the independent features of x1 and x2, and also contains the connection between the two. That is, the deep connection between x2 and y can be indirectly displayed through the relationship between x1 and y. Therefore, the combined prediction model can learn deeper nonlinear features through new feature training and increase fitting accuracy.
[0053] In the step 4, the convolutional neural network CNN is placed at the back end of the multi-head attention mechanism MHA to reduce the output feature dimension of the multi-head attention mechanism MHA;
[0054] Convolutional neural network CNN is mainly composed of two parts: convolution calculation layer and pooling layer. The convolution calculation layer is composed of filters. The filter can be regarded as a stack of multiple convolution kernels. The filter slides on the input data to perform convolution operations and extract features from the input data to achieve the purpose of reducing the data dimension. The effect of reducing the data dimension depends on the size of the filter. According to the characteristics of the mutual inductor data, the present invention adopts a one-dimensional CNN, in which the convolution calculation layer principle is as follows Figure 4 shown.
[0055] In step 5, in order to ensure the accuracy of the mutual inductor error prediction value output by the LSTM neural network model, the square loss function is used to update the weight and bias matrix parameters:
[0056]
[0057] In formula (10), J represents the loss value; h(t) represents the error prediction value; m(t) is the actual error result; γ is the regularization parameter; N is the number of samples; is the updated matrix; W T Represents the transpose of matrix W; arg W,b Represents the minimum square loss of matrix W and b;
[0058] W=[W f ,W i ,W c ,W o ]; W f ,W i ,W c ,W o They represent the weight matrices of forget gate, input gate, memory unit, and output gate respectively;
[0059] b=[b f ,b i ,b c ,b o ]; b f ,b i ,b c ,b o Respectively represent the forget gate, input gate, memory unit, and output gate bias matrix;
[0060] Cosine similarity is used to compare the error credibility between the output values of the training sample and the output values of the test set, and dynamically adjust W,
[0061]
[0062] In formula (11), R t,η represents cosine similarity; S t,η For R t,η Nonlinear change result, the value is between [0,1];
[0063] x(t) is the training set input at time t; z(η) is the test set input at time η; x(t) T represents the transposed matrix of x(t); exp(-R t,η ) represents the exponential function value of cosine similarity.
[0064] Combine (10) and (11) to adjust
[0065]
[0066] In formula (12), is the output result of the test set model at time η for z(η); h(t) is the output result of the training set model at time t for x(t); J η represents the loss value at time η; represents the updated weight matrix at time η; represents the updated bias matrix at time η; Represents the minimum square loss of matrix W and b at time η.
[0067] The mathematical equation of the LSTM neural network is further described as:
[0068]
[0069] In formula (13), f η (t) represents the forget gate output of the test set; W f,η Represents the forget gate weight matrix of the test set at time η; Indicates the output value of the test set model at the last moment; b f,η represents the forget gate bias matrix of the test set η at time σ(i) represents the activation function; i(t) represents the input gate output; W i,η represents the input gate weight matrix b at time η i,η Represents the moment forget gate input gate bias matrix; Indicates the state of the memory unit; W c,η represents the memory unit weight matrix at time η; b c,η represents the memory unit bias matrix at time η; o(t) represents the initial output; W o,η represents the output gate weight matrix at time η; b o,η represents the output gate bias matrix at time η; c(t) represents the updated memory unit; f(t) represents the forget gate output; c(t-1) represents the memory unit at the previous moment; represents the operator, Hadamard product; h(t) represents the output of the training set at the last moment; tanh(t) represents the activation function;
[0070] In step 5, the MHA-CNN-SLSTM combined prediction model is composed of three units: multi-head attention mechanism MHA, convolutional neural network CNN, and SLSTM. The network structure is as follows: Figure 5 As shown in the figure, MHA is at the front end of the combined prediction model and is used to cross-process the data features. The results of the cross-processed data features are input into CNN to process the data, and then the processed data is input into SLSTM for prediction.
[0071] In the step 5, after the MHA-CNN-SLSTM combined prediction model established in step 5 completes the prediction of the sample data, the internal parameters are fixed, and at the same time, the error of the mutual inductor prediction value is generated by the comparison between the predicted value output by the test sample and the actual value, that is, the error should be highly adaptable to the test sample. Therefore, the prediction error can be directly input into the MHA-CNN-SLSTM combined prediction model for prediction, and an association network between the prediction value error and the characteristics of the mutual inductor error influencing factors is established to realize the prediction of the prediction value error, thereby compensating and correcting the prediction value.
[0072] The present invention provides a mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation, and the technical effects are as follows:
[0073] 1) In step 1 of the present invention, the VMD method is used to decompose the error data into multiple modal components. The decomposition result can indicate the characteristics of the error data and can separate the noise interference signal in the data, thereby improving the accuracy of the model input data from the data source.
[0074] 2) In step 2 of the present invention, the traditional Seagull optimization algorithm is improved and the two parameters of the number of decompositions and the penalty factor in VMD are optimized. The introduced opposition search operator strategy and nonlinear convergence control factor can solve the problem that the traditional Seagull optimization algorithm is prone to fall into local optimality and early convergence. Therefore, the improved Seagull optimization algorithm is used to optimize the number of decompositions and the penalty factor in VMD. The optimization result has a higher fitness value and is more in line with the actual situation.
[0075] 3) In step 3 of the present invention, in the traditional prediction process, since the invisible error influencing factors are not directly reflected in the data, only the explicit error influencing factors are often used in model training, ignoring the correlation between the explicit error influencing factors and the invisible error influencing factors. Step 3 uses MHA to cross-process the characteristics of the error influencing factors, and indirectly describes the relationship between the invisible error influencing factors and the error through the explicit error influencing factors, which will effectively improve the learning ability of nonlinear features and deep invisible features, and can improve the prediction accuracy of the model.
[0076] 4) In step 4 of the present invention, since the feature dimension increases compared to the original feature after MHA feature cross processing, too large feature dimension will affect the model calculation speed. The present invention relies on the filter structure characteristics of CNN itself to process the cross-features, so as to achieve the purpose of reducing data dimension and increasing calculation speed.
[0077] 5) In step 5 of the present invention, usually after the model training is completed, the internal parameters will no longer be adjusted, and the test set is directly input into the model to obtain the predicted value. However, in actual application, there are different degrees of correlation between the training set and the test set. In order to improve the generalization ability of the model in actual operation, step 5 improves the LSTM. The improved model can deeply analyze the correlation between the training samples and the test set. On the basis of the completed training model, the model parameters are slightly adjusted to provide the most adaptable prediction network for each test set, and the model generalization ability is stronger.
[0078] 7) The improved Seagull optimization algorithm ISOA in the present invention is superior to other optimization algorithms in terms of algorithm convergence speed, convergence accuracy and ability to escape local optimality. The improved Seagull optimization algorithm ISOA is used to optimize the key parameters of the variational mode decomposition algorithm VMD. The decomposed modes do not have aliasing phenomenon and the optimization time is short.
[0079] 8) After cross-processing of the characteristics of the influencing factors of the mutual inductor error in the present invention, the correlation between the strong correlation characteristics and the weak correlation characteristics can be effectively explored. The SLSTM model improves the adaptability between the training set and the test set, and dynamically adjusts the network parameters, so that the MHA-CNN-SLSTM combined prediction model has a higher prediction accuracy. Based on the model, the difference between the error prediction value and the actual value is predicted, and the obtained result compensates the preliminary prediction value, and the prediction accuracy of the model is further improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0080] Figure 1 It is a schematic diagram of the framework of the mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation of the present invention.
[0081] Figure 2 This is the result diagram after the ratio difference VMD decomposition in the present invention.
[0082] Figure 3 Schematic diagram of feature intersection.
[0083] Figure 4 Schematic diagram of the one-dimensional CNN convolutional computation layer.
[0084] Figure 5 MHA-CNN-SLSTM network structure
[0085] Figure 6 Pearson correlation coefficient analysis results.
[0086] Figure 7 This is a comparison chart of the prediction results of the MHA-CNN-SLSTM combined prediction model in the present invention. DETAILED DESCRIPTION
[0087] A prediction method for transformers based on MHA-CNN-SLSTM and error compensation is proposed. In order to improve the prediction accuracy of transformer errors, the traditional Seagull algorithm is firstly improved by introducing the opposition search operator strategy and nonlinear convergence control factor. A method based on the improved Seagull Optimization algorithm (ISOA) to optimize the key parameters of the variational mode decomposition (VMD) is proposed to achieve adaptive decomposition of error data. Then, the multi-head attention mechanism (MHA) is used to cross-process the error-affecting features, and the correlation between the features is mined. The deep connection between the weakly correlated features and the errors is established through the relationship between the strongly correlated features and the errors, so as to avoid the reduction of prediction accuracy due to data waste. Considering the relationship between the training set and the test set, a long short-term memory neural network (Similar Long Short-Term Memory, SLSTM) considering the sample similarity is proposed to dynamically adjust the network weights and biases. Finally, the MHA-CNN-SLSTM prediction model is constructed, and the error between the predicted value and the actual value is input into the prediction model again as the training set to generate compensation data to compensate the preliminary prediction value, so as to further improve the prediction value. Finally, the actual measured data of a certain mutual inductor is used for verification, and the results show that the model proposed in the present invention has higher prediction accuracy.
[0088] The transformer prediction method of the present invention includes three parts: transformer error data decomposition, MHA-CNN-SLSTM model construction, and error compensation:
[0089] (I) Transformer error data decomposition:
[0090] The error data decomposition part of the transformer uses the variational mode decomposition algorithm (VMD) to decompose the original error data into different high and low frequency components. In addition, the traditional Seagull algorithm is improved by introducing the opposition search operator strategy and nonlinear convergence control factor. An improved Seagull Optimization algorithm (ISOA) is proposed, which selects the permutation entropy as the fitness function, optimizes the key parameters of VMD using ISOA, and optimizes the decomposition number k and the penalty factor α to achieve the goal of optimal decomposition of the error data.
[0091] The specific improvement process of ISOA is as follows:
[0092] First, the opposition search operator strategy is introduced to optimize the initial population to generate the current individual and the opposite individual in the interval [L,U], which is defined as:
[0093]
[0094] In the formula, x i Represents the current individual; Indicates the individual that is opposite to the current individual.
[0095] According to the above definition, the following initialization method is adopted: set the population size N, compare the current individual with its relative individual, and select the individual with the best fitness value as the initial population.
[0096] The initialization formula is:
[0097]
[0098] The Seagull optimization algorithm is highly dependent on the control factor. Therefore, the present invention then improves the nonlinear convergence control factor and adjusts the control factor f c Redesign, expect f c At the beginning of the algorithm, it increases with the number of iterations. When it reaches a certain value, f c It decreases rapidly to a small value, and then increases gradually at a slow speed.
[0099] Based on this, the control factor f c The update formula is as follows:
[0100] f c =(f ci -f cf )+[(1-t / t max ) / (1-η×t / t max )](3);
[0101] In the formula, f ci is the control factor f c Starting value; f cf is the control factor f c final value; η is the adjustment coefficient; t is the current number of iterations; t max is the maximum number of iterations.
[0102] The specific process of ISOA optimization algorithm is as follows Figure 1 The ISOA optimization flow chart on the left is shown.
[0103] (II) Based on MHA-CNN-SLSTM combined prediction model:
[0104] External factors such as load, temperature, and humidity will affect the mutual inductor error. Therefore, the prediction model not only needs to input the high and low frequency components after the mutual inductor error data is decomposed, but also needs to input the characteristic parameters of each influencing factor into the model. The Pearson correlation coefficient is between -1 and 1 and is widely used to measure the correlation between variables. The larger the absolute value of the coefficient, the stronger the correlation between the mutual inductor error value and the characteristics of the influencing factor. The present invention performs a correlation analysis on the monitoring data of a certain mutual inductor for one month, and takes the ratio difference in the error as an example to illustrate.
[0105] According to the Pearson correlation coefficient method, the correlation between temperature, humidity, load, vibration, magnetic field and other factors and the ratio difference is analyzed. The analysis results are as follows: Figure 6 shown.
[0106] It can be seen that the factors with extremely strong correlation with the ratio difference are temperature (-0.854) and humidity (0.815), followed by load (0.252) and vibration (-0.225), and finally the extremely weakly correlated factor is magnetic field (-0.169).
[0107] It can be further seen that there is not only a direct relationship between the ratio difference and the characteristics of different influencing factors, but also many indirect deep relationships. For example, although the ratio difference-humidity and temperature are extremely strongly correlated, and the ratio difference-load is weakly correlated, the humidity and temperature-load are moderately correlated, that is, there is a certain connection between the ratio difference-temperature and humidity-load, and the deep relationship between the ratio difference and the load can be explored through the humidity and temperature characteristics.
[0108] Therefore, if we directly select the strongly correlated influencing factors as the characteristic parameters of the influencing factors according to the correlation coefficient sorting results based on past experience, and ignore the data of weakly correlated factors such as load, vibration, and magnetic field, we will not be able to accurately establish the accurate relationship between the mutual inductor error and the characteristics of the influencing factors, which may result in a decrease in prediction accuracy.
[0109] To this end, the present invention adopts a multi-head attention mechanism to cross-process the features of each influencing factor and explore the deep-level relationship between the features.
[0110] (1) Multi-Head Attention (MHA):
[0111] Assume that there is the following relationship between the predicted value and the feature value:
[0112] y=ω1x1+ω2x2....ω n x n +b(4);
[0113] Where y is the predicted value; ω n is the corresponding feature x n Weight coefficient; b is a constant term.
[0114] If only two influencing factor features are selected for analysis, there is a strong correlation between y and x1, a weak correlation between y and x2, and a moderate correlation between x1 and x2. Combined with the above analysis results, it can be seen that directly putting x1 into the prediction model training will not be able to learn the deep relationship between the features, so the features need to be cross-processed.
[0115] The multi-head attention mechanism assigns different weights to each feature through the characteristics of the model itself, explores the multiple relationships between features, and realizes the intersection of explicit features and implicit features with large correlation coefficients by fusing features of multiple different attention heads, forming new features with richer expressiveness than the original features.
[0116] Matrix Z and weight matrix W q , W k , W v Multiply to get the query / key / value matrix (Q / K / V):
[0117] Q=W q Z, K = W k Z, V = W v Z(5);
[0118] The matrices Q and V calculate similarity through dot product operations, explore the correlation between features, and capture internal multiple relationships, as shown below:
[0119]
[0120] In the formula, F is the self-attention network, σ is the Softmax activation function; is the scaling factor.
[0121] The calculations between the sub-attention heads in the multi-head attention mechanism are independent of each other. The output matrices of each sub-attention head are concatenated and multiplied by the weight matrix W. o The final output result is as follows:
[0122]
[0123] In the formula, H i is the i-th head of multi-head attention; W i Q , W i K , W i V is the weight of the i-th attention head; F c is the matrix concatenation function.
[0124] In order to ensure information transmission and gradient stability, according to formula (8) Concatenate with Z jump and normalize.
[0125]
[0126] Where, M is the normalized result; F LN is the normalization function (LayerNorm).
[0127] After feature crossover, the new feature x3 contains the independent features of x1 and x2, and also contains the connection between the two, that is, the deep connection between x2 and y can be indirectly displayed through the relationship between x1 and y. Therefore, the prediction model learns deeper nonlinear features through new feature training, which increases the fitting accuracy.
[0128] (2) Convolutional Neural Networks (CNN):
[0129] The new features formed after the crossover of the multi-head attention mechanism have an increased feature dimension compared to the original features. Too large feature dimensions will affect the model calculation speed. To this end, the present invention uses CNN to reduce feature dimensions, reduce training parameters, and improve model calculation speed.
[0130] CNN is mainly composed of two parts: convolution calculation layer and pooling layer. The convolution calculation layer is composed of filters. The filter can be regarded as a stack of multiple convolution kernels. The filter slides on the input data to perform convolution operations and extract features from the input data to achieve the purpose of reducing the data dimension. The effect of reducing the data dimension depends on the size of the filter. According to the characteristics of the mutual inductor data, the present invention adopts a one-dimensional CNN, in which the convolution calculation layer principle is as follows Figure 4 As shown, a filter with a dimension smaller than the input data is moved within the input data along a fixed step size (the step size is usually designed according to actual conditions), a convolution operation is performed on the data within the moving range of the filter, and the result is used as the output for reduction processing.
[0131] (3) Sample Similarity Long Short-Term Memory (SLSTM) Neural Network:
[0132] As a special recurrent neural network, Long Short-Term Memory (LSTM) can effectively transmit and express information in time series and realize the prediction of time series. However, traditional LSTM
[0133] The weight and bias matrix in the model cannot be adjusted dynamically, and the correlation between the model training set and the test set is not considered, which reduces the accuracy of the mutual inductor error prediction value to a certain extent. For this reason, the present invention improves the traditional LSTM.
[0134] The mathematical expression of the LSTM model is:
[0135]
[0136] Where x(t) is the input value at the current moment; f(t), i(t), and o(t) are the outputs of the forget gate, input gate, and output gate, respectively. f , W i , W c , W o 、b f 、b i 、b c 、b o are the weight and bias matrices respectively; c(t) is the network memory unit; c t-1 (t) is the network memory unit before updating; is the memory unit state; h(t) is the output value at the current moment; h t-1 (t) is the model output value at the previous moment; σ and tanh are activation functions.
[0137] In order to ensure the accuracy of the model output mutual inductor error prediction value, the square loss function is used to update the weight and bias matrix parameters:
[0138]
[0139] Where m(t) is the actual error result; γ is the regularization parameter; N is the number of samples; W = [W f ,W i ,W c ,W o ]; b = [b f ,b i ,b c ,b o ]; is the updated matrix.
[0140] Usually, after the model training is completed, the internal parameters will not be adjusted anymore, and the test set is directly input into the model to obtain the predicted value. However, in actual application, there are different degrees of correlation between the training set and the test set. Therefore, the present invention uses cosine similarity to compare the error credibility between the output value of the training sample and the output value of the test set, and dynamically adjusts
[0141]
[0142] Where x(t) is the training set input at time t; z(η) is the test set input at time η; S t,η For R t,η The result of nonlinear change is between [0,1].
[0143] Combine (10) and (11) to adjust
[0144]
[0145] In the formula, z(η) is the output result of the test set model at time η; h(t) is the output result of the training set model x(t) at time t.
[0146] The mathematical equation of the LSTM model is further rewritten as:
[0147]
[0148] The SLSTM calculation process is as follows Figure 1 The SLSTM flowchart on the right is shown.
[0149] This model is combined with the models described in (1) and (2) to finally form the MHA-CNN-SLSTM combined prediction model to predict the mutual inductor error. The prediction model framework is as follows: Figure 1 As shown in the MHA-CNN-SLSTM prediction process.
[0150] (III) Error compensation:
[0151] The basic idea of error compensation is to generate compensation data based on the prediction results and influencing factors to offset the errors in the prediction values, so as to further improve the prediction accuracy.
[0152] After the prediction model established above completes the prediction of the sample data, the internal parameters are fixed. At the same time, the error of the transformer prediction value is caused by the comparison between the predicted value and the actual value output by the test sample, that is, the error should be highly adaptable to the test sample. Therefore, the prediction error can be directly input into the model prediction, and a correlation network between the prediction value error and the characteristics of the transformer error influencing factors is established to realize the prediction of the prediction value error, thereby compensating and correcting the prediction value.
[0153] Verification example:
[0154] The present invention selects actual data of the error ratio difference of a certain transformer in a certain substation for 30 days and data of environmental influencing factors such as temperature, humidity, load, vibration, and magnetic field, with a total of 4320 sampling points, of which the first 27 days are used as training sample data and the last 3 days are used as test sample data. The data are normalized to between [-1,1], and a single-step prediction of the transformer ratio difference in the short term in the future is used to verify the method proposed by the present invention.
[0155] In order to intuitively show the prediction performance of the model, the present invention uses the mean absolute error (MAE), root mean square error (RMSE) and mean squared error (MSE) as prediction result evaluation indicators.
[0156]
[0157] Where N is the number of samples; y i , are the true error value and the predicted error value respectively.
[0158] Verification based on MHA-CNN-SLSTM combined prediction model
[0159] The results of VMD decomposition of the difference data are as follows: Figure 2 As shown, from Figure 2 It can be seen that after ISOA optimization, the VMD decomposition level k is determined to be 8, and the components do not have the problems of modal aliasing and low time-frequency resolution.
[0160] In order to verify the accuracy of the prediction results of the combined model proposed in the present invention, we conducted a comparative analysis with other combined models. The results are as follows: Figure 7 As shown. Figure 7 It can be seen that the model proposed in the present invention is closer to the true ratio difference than the convolutional neural network-self-attention encoder-decoder-residual optimization (CNN-SAEDN-RES) and the convolutional neural network-long short-term memory neural network (CNN-LSTM). At the same time, the MHA-CNN-SLSTM after error compensation deviates less from the true value, and the prediction accuracy is significantly improved, which can also be confirmed by Table 2.
[0161] Table 2 Comparison of evaluation indicators of different combined prediction models
[0162]
[0163] By comparison, it can be seen that the operation time of the model proposed in the present invention is longer than that of other combination models. The reason for this is that the present invention performs feature fusion on the factors affecting the mutual inductor error. Although CNN is used to reduce the dimension of the fusion features output by the multi-head self-attention mechanism, the amount of data calculation is still large compared with other models. Fortunately, the operation speed is still within an acceptable range.
[0164] In terms of prediction accuracy, the model proposed in this invention has a high prediction accuracy. MAEFor example, the MHA-CNN-SLSTM model decreased by 7.87% and 5.9% compared with CNN-LSTM and CNN-SAEDN-RES respectively. After error compensation, the model accuracy was further improved, and the evaluation index decreased by 15.12% and 13.3%, which fully demonstrated the superiority of the combined model and error compensation proposed in the present invention, and was helpful to improve the prediction accuracy of the mutual inductor ratio difference.
Claims
1. A transformer prediction method based on MHA-CNN-SLSTM and error compensation, characterized in that The following steps are involved: Step 1: Use the variational mode decomposition algorithm VMD to decompose the transformer error data into different high and low frequency components; Step 2: Optimize the key parameters of the variational mode decomposition algorithm VMD based on the improved Seagull optimization algorithm ISOA to achieve the optimal decomposition of error data; Step 3: Based on the multi-head attention mechanism MHA, cross-process the error-affecting features, explore the correlation between the features, and establish a deep connection between the weakly correlated features and the error through the relationship between the strongly correlated features and the error; Step 4: Place the convolutional neural network CNN at the back end of the multi-head attention mechanism MHA to reduce the output feature dimension of the multi-head attention mechanism MHA; Step 5: Considering the correlation between the training set and the test set, improve the LSTM neural network, dynamically adjust the network weights and biases, and build the MHA-CNN-SLSTM combined prediction model; Step 6: Use the error between the predicted value and the actual value as the training set, input it into the combined prediction model constructed in step 5, and generate compensation data to compensate the predicted value.
2. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In the step 1, the mutual inductor error data is decomposed into different high and low frequency components by using the variational mode decomposition algorithm VMD; The high and low frequency components are distinguished according to the zero crossing rate. The zero crossing rate calculation formula is as follows: Where P is the zero-crossing rate; Z zero is the number of times the signal passes through zero; N is the interval length.
3. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In the step 2, an adversarial search operator strategy and a nonlinear convergence control factor are introduced to improve the traditional Seagull algorithm, an improved Seagull optimization algorithm ISOA is proposed, permutation entropy is selected as the fitness function, and the improved Seagull optimization algorithm ISOA is used to optimize the key parameters of the variational mode decomposition algorithm VMD, and the decomposition number k and the penalty factor α are optimized to achieve the goal of optimal decomposition of error data.
4. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 3 is characterized in that: Step 2 is as follows: First, the opposition search operator strategy is introduced to optimize the initial population to generate the current individual and the opposite individual in the interval [L,U], which is defined as: In formula (1), Indicates the individual opposite to the current individual; x i Represents the current individual; L represents the lower value within the population optimization interval; U represents the upper value within the population optimization interval; According to the above definition, the following initialization method is adopted: set the population size N, compare the current individual with its relative individual, and select the individual with the best fitness value as the initial population; The initialization formula is: In formula (2), represents the value obtained by substituting the individual opposite to the current individual into the fitness function; f(x i ) represents the value obtained by substituting the current individual into the fitness function; Then, the nonlinear convergence control factor is improved and the control factor f c Redesign so that the control factor f c At the beginning of the algorithm, it increases with the number of iterations. When it reaches a certain value, the control factor f c Rapidly decreases to a smaller value, and then gradually increases at a slow speed; based on this, the control factor f c The update formula is as follows: f c =(f ci -f cf )+[(1-t / t max ) / (1-η×t / t max )](3); In formula (3), f ci is the control factor f c Starting value; f cf is the control factor f c final value; η is the adjustment coefficient; t is the current number of iterations; t max is the maximum number of iterations.
5. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 4 is characterized in that: In step 2, the key parameters of the variational mode decomposition algorithm VMD are optimized, that is, the number of decompositions k and the penalty factor α in VMD are optimized to achieve the optimal decomposition of the error data.
6. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In step 3, according to the Pearson correlation coefficient analysis, it can be obtained that the factors with extremely strong correlation with the ratio difference are temperature and humidity; the second weakly correlated factors are load and vibration; the last extremely weakly correlated factor is the magnetic field; there is a certain connection between the ratio difference-temperature and humidity-load, and the deep relationship between the ratio difference and the load can be explored through the humidity and temperature characteristics; the multi-head attention mechanism MHA is used to cross-process the characteristics of each influencing factor to explore the deep relationship between the characteristics.
7. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 6, characterized in that: Step 3 is as follows: The multi-head attention mechanism MHA assigns different weights to each feature through a matrix, explores the multiple relationships between features, and realizes the intersection of explicit features and implicit features with large correlation coefficients by fusing features of multiple different attention heads, forming new features with richer expressiveness than the original features. Error influencing factor data matrix Z and query linear projection weight W q Multiply them together to get the query matrix Q: Q=W q Z Error influencing factor data matrix Z and key linear projection weight W k Multiply them together to get the key matrix K: W=W k WITH Error influencing factors data matrix Z and value linear projection weight W v Multiply them together to get the key matrix V: V=W v Z In the above formula, matrices Q and V calculate similarity through dot product operations, explore the correlation between features, and capture internal multiple relationships, as shown below: In formula (6), F is the self-attention network, σ is the Softmax activation function; is the scaling factor; F(Q,K,V) represents the output value of Q, K, and V when they are substituted into the self-attention network; K T represents the transpose of the key matrix d k represents the dimension of vectors in Q and K; The calculations between the sub-attention heads in the multi-head attention mechanism are independent of each other. The output matrices of each sub-attention head are concatenated and multiplied by the weight matrix W. o , and the final output is as follows: In formula (7), H i is the i-th head of multi-head attention; W i Q , W i K , W i V is the weight of the i-th attention head; F c is the matrix concatenation function; H1,H2,........,H i Represents the i-th head of the multi-head attention mechanism; Represents the final multi-head attention mechanism output matrix; To ensure information transmission and gradient stability, Z and Z are jump-connected and normalized according to formula (8); Where, M is the normalized result; F LN is the normalization function (LayerNorm); represents the final multi-head attention mechanism output error matrix; Z represents the original error matrix.
8. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In step 3, feature crossover specifically involves: calculating the original features of different error influencing factors through the sub-attention heads within the multi-head attention mechanism, and dynamically weighting and merging them to obtain new features of error influencing factors; After the features are crossed, the new feature x3 contains the independent features of x1 and x2, and also contains the connection between the two. That is, the deep connection between x2 and y can be indirectly displayed through the relationship between x1 and y. Therefore, the combined prediction model can learn deeper nonlinear features through new feature training and increase the fitting accuracy.
9. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In step 5, in order to ensure the accuracy of the mutual inductor error prediction value output by the LSTM neural network model, the square loss function is used to update the weight and bias matrix parameters: In formula (10), J represents the loss value; h(t) represents the error prediction value; m(t) is the actual error result; γ is the regularization parameter; N is the number of samples; is the updated matrix; W T Represents the transpose of matrix W; arg W,b Represents the minimum square loss of matrix W and b; W=[W f ,W i ,W c ,W o ]; W f ,W i ,W c ,W o They represent the weight matrices of forget gate, input gate, memory unit, and output gate respectively; b=[b f ,b i ,b c ,b o ]; b f ,b i ,b c ,b o Respectively represent the forget gate, input gate, memory unit, and output gate bias matrix; Cosine similarity is used to compare the error credibility between the output values of the training sample and the output values of the test set, and dynamically adjust In formula (11), R t,η represents cosine similarity; S t,η For R t,η Nonlinear change result, the value is between [0,1]; x(t) is the training set input at time t; z(η) is the test set input at time η; x(t) T represents the transposed matrix of x(t); exp(-R t,η ) represents the exponential function value of cosine similarity; Combine (10) and (11) to adjust In formula (12), is the output result of the test set model at time η for z(η); h(t) is the output result of the training set model at time t for x(t); J η represents the loss value at time η; represents the updated weight matrix at time η; represents the updated bias matrix at time η; represents the minimum square loss of matrix W and b at time η; The mathematical equation of the LSTM neural network is further described as: In formula (13), f η (t) represents the forget gate output of the test set; W f,η Represents the forget gate weight matrix of the test set at time η; Indicates the output value of the test set model at the last moment; b f,η represents the forget gate bias matrix of the test set η at time σ(i) represents the activation function; i(t) represents the input gate output; W i,η represents the input gate weight matrix b at time η i,η Represents the moment forget gate input gate bias matrix; Indicates the state of the memory unit; W c,η represents the memory unit weight matrix at time η; b c,η represents the memory unit bias matrix at time η; o(t) represents the initial output; W o,η represents the output gate weight matrix at time η; b o,η represents the output gate bias matrix at time η; c(t) represents the updated memory unit; f(t) represents the forget gate output; c(t-1) represents the memory unit at the previous moment; represents the operator, Hadamard product; h(t) represents the output of the training set at the previous moment; tanh(t) represents the activation function.
10. The mutual inductor prediction method based on MHA-CNN-SLSTM and error compensation according to claim 1, characterized in that: In the step 5, after the MHA-CNN-SLSTM combined prediction model established in step 5 completes the prediction of the sample data, the internal parameters are fixed, and at the same time, the error of the mutual inductor prediction value is generated by the comparison between the predicted value output by the test sample and the actual value, that is, the error should be highly adaptable to the test sample. Therefore, the prediction error can be directly input into the MHA-CNN-SLSTM combined prediction model for prediction, and an association network between the prediction value error and the characteristics of the mutual inductor error influencing factors is established to realize the prediction of the prediction value error, thereby compensating and correcting the prediction value.
Citation Information
Patent Citations
Controller optimization method based on improved seagull algorithm
CN115469532A
Two-stage short-term power load combined prediction method considering error compensation
CN116526450A
Mechanical fault diagnosis method based on multi-scale image attention fusion network
CN116821762A
Ultra-short-term photovoltaic output prediction method with hyper-parameter self-optimization and structure self-adjustment
CN117454934A
Human body sitting track prediction method
CN117671791A
Cited By
Online measurement and compensation method for geometric error of numerical control machine tool
CN121680277A
Risk assessment system based on myocardial ischemia and storage medium
CN122498857A
