Ultra-short-term wind power combined prediction method based on TCN-SENet-Transformer

Abstract

The invention discloses an ultra-short-term wind power combined prediction method based on TCN-SENet-Transformer, and the method comprises the steps: firstly adjusting the channel weight through SENet, and extracting the spatial features of data through TCN; meanwhile, adopting Transform to mine the time sequence characteristics of the data, and capturing the context characteristic relation; and finally, data space and time sequence features are fully fused by using CA. According to the method, the characteristic that the TCN and the Transform can process all data of a sequence in a multi-thread mode is fully utilized, meanwhile, the TCN is used for analyzing spatial characteristics, the Transform is used for extracting time sequence characteristics, and dual parallelism is achieved. According to the method, the prediction precision is effectively improved, and the technical problems of low prediction precision, complex process calculation and low prediction efficiency in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power prediction, and more specifically, to an ultra-short-term wind power combination prediction method based on TCN-SENet-Transformer. Background Art

[0002] With the continuous advancement of the "dual carbon" policy, my country's energy structure reform has gradually deepened, and wind power as a clean renewable energy has become a research hotspot. Among them, the phenomenon of "wind abandonment and power rationing" occurs frequently. Due to the volatility of wind power generation and the insufficient absorption capacity of the power grid, the power quality and the stability of the power grid operation are affected. Therefore, improving the accuracy of wind power prediction is of great significance to power dispatching.

[0003] At present, there are various classification standards for wind power prediction methods. According to the prediction model, it can be divided into physical methods and statistical methods, among which physical methods include numerical weather prediction (NWP). The literature uses NWP wind speed to hierarchically cluster the prediction errors of historical data and uses the empirical distribution model to fit the probability distribution. NWP has a low update frequency, high data collection requirements, and a complex model. It is often used for medium- and long-term predictions, but not for ultra-short-term wind power predictions. Statistical methods include time series prediction methods and artificial intelligence methods. The main time series prediction models include autoregressive models (AR), autoregressive integrated moving average models (ARIMA), quantile regression (QR), etc.

[0004] In CN202310774650.0, an ultra-short-term wind power prediction method based on time series representation and multi-level attention is introduced on the basis of deep learning to effectively extract the multi-scale time information of wind power, and consider the dynamic coupling effect of meteorological characteristics on wind power, so as to improve the accuracy and reliability of ultra-short-term wind power prediction. However, this patent only analyzes the data characteristics from the time series itself, and rarely considers the connection between time series and space. In CN202311286873.9, a wind power prediction method based on time series attention and spatial channel reconstruction convolution, the original data of the wind farm is first normalized to generate a red, green and blue RGB image, which is used to train the time series attention unit to obtain prediction data 1, and the historical wind power output data is subjected to complete set empirical mode decomposition and then spatial channel reconstruction convolution is performed, and the feature information of the extracted wind farm data is input into the acceptance weighted key value model to obtain prediction data 2, and finally the prediction data 1 and the prediction data 2 are fully connected layer processed to obtain the wind power prediction result of this area. This patent uses a parallelized timing module with strong nonlinear fitting capabilities. The spatial channel reconstruction convolution can reduce redundant features, process and predict complex wind power output changes, and improve the speed and accuracy of wind power prediction. However, the patent first converts the data itself into RGB images, and then uses the temporal attention to analyze the image features, which destroys the continuity of the original data, ignores the dependency between data, and affects the prediction accuracy. In addition, the process is complex to calculate and has a large number of parameters, which affects the prediction efficiency. Summary of the invention

[0005] The main technical problem to be solved by the present invention is that most traditional wind power prediction methods only analyze the unilateral characteristics of the data, and pay little attention to the close connection between the time series and the global characteristics of the space and parallel processing, and the prediction reliability is limited. The existing wind power prediction based on time series attention and spatial channels destroys the continuity characteristics of the original data, ignores the dependency between the data, and affects the prediction accuracy. In addition, the process is complex in calculation and has low prediction efficiency. The present invention provides an ultra-short-term wind power combination prediction method based on TCN-SENet-Transformer.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] An ultra-short-term wind power combination forecasting method based on TCN-SENet-Transformer, comprising the following steps:

[0008] S1. Standardize and preprocess the original data set;

[0009] S2. Capture the spatial features of the new dataset through a temporal convolutional network and adjust the feature weights using a compression-excitation network;

[0010] S21. Input the preprocessed data into the dilated convolutional network, and then use padding to fill the sequence with zeros on the left and right so that the input data and output data have the same dimension;

[0011] S22. Perform weight normalization, use the ReLU activation function, introduce nonlinear factors into the network, and obtain F(x) after regularization;

[0012] S23. Input the preprocessed data into the dilated convolutional network to ensure that the number of input and output channels of each layer remains consistent, and obtain the output H(x);

[0013] S24. Process H(x) to obtain F(x), and then perform global average pooling through the compression channel;

[0014] S25. Through the excitation channel, use the fully connected layer to reduce the dimension, add the ReLU activation function to enhance the nonlinear fitting ability of the model, send it to the fully connected layer to upgrade the dimension to the original dimension, and use the Sigmoid activation function to speed up the network convergence;

[0015] S26. Use the compressed excitation channel to process s and input f to weight the final output x m , assign feature weights according to the loss function and output the feature matrix Q;

[0016] S3. Encode the original sequence through multi-head attention mechanism, residual normalization and feedforward neural network processing, and then calculate the output matrix Z to obtain matrix K and matrix V;

[0017] S31. Position encoding the preprocessed data using sine and cosine functions of different frequencies;

[0018] S32. Send the encoded data to the multi-head attention mechanism and then use residual normalization;

[0019] S33. Use the fully connected layer to enhance the nonlinear fitting ability of the model, and then use the residual standardization to obtain the output matrix K and matrix V;

[0020] S4. Input matrix Q, matrix K and matrix V into the CA model, calculate the attention weight distribution, and output the result.

[0021] Furthermore, the standardization formula is:

[0022]

[0023] Where: z is the standardized result, x is the input observation value, μ is the mean of the total data, and σ is the standard deviation of the total data.

[0024] Furthermore, the dilated convolutional network is expressed as:

[0025]

[0026] Where: G(t) is the output of the t-th element under the dilated convolution; k is the convolution kernel; n is the element ordinal number; g(n) is the n-th element filter value; Move the t element forward by d n The input value of .

[0027] Further, the filling calculation is expressed as:

[0028] padding = (k-1)*d, d = 2 i

[0029] Where: k is the convolution kernel; d is the dilation factor; i is the number of levels.

[0030] Furthermore, the expression of the compression channel is:

[0031]

[0032] Where: f m is the compressed output; H×W is the spatial dimension; g m is the m-th two-dimensional input matrix; i, j are g m The number of rows and columns.

[0033] Furthermore, the sigmoid activation function is expressed as:

[0034] s=σ(W2δ(W1z))

[0035] Where: s is the excitation output; σ is the sigmoid function; W1, W2 are the weight matrices of the fully connected layer; δ is the relu activation function; z is the input vector.

[0036] Furthermore, the final output x m It is expressed as:

[0037] x m =s m *g m

[0038] Where: x m is the final output; s m is the weight value.

[0039] Furthermore, the sine and cosine functions are expressed as:

[0040]

[0041] Where: pos is the position, i is the dimension, d is model is the embedding dimension.

[0042] Furthermore, the self-attention expression of the multi-head attention mechanism is:

[0043]

[0044] Where: softmax is the normalization function; Q is the query matrix; K is the key matrix; V is the value matrix; d k is the vector dimension.

[0045] Furthermore, the residual normalization process is expressed as:

[0046] LayerNorm(X+MultiHeadAttention(X))

[0047] LayerNorm(X+FeedForward(X))

[0048] Where: X represents the input of the multi-head attention or feedforward neural network, MultiHeadAttention(X) and FeedForward(X) represent the output.

[0049] Furthermore, the fully connected layer enhancement model expression is:

[0050] max(0,XW1+b1)W2+b2

[0051] Where: X is the input; W1, W2 are the input matrices; b1, b2 are the deviations.

[0052] Furthermore, the self-attention expression of the cross-attention mechanism in the CA model is:

[0053]

[0054] Where: softmax is the normalization function; Q is the query matrix; K is the key matrix; V is the value matrix; d k is the vector dimension.

[0055] Compared with the prior art, the beneficial effects are:

[0056] The method of the present invention first uses SENet to adjust the channel weights and extracts the spatial features of the data through TCN; at the same time, Transformer is used to mine the temporal features of the data and capture the contextual feature connections; finally, CA is used to fully integrate the spatial and temporal features of the data. The present invention makes full use of the characteristics of TCN and Transformer that each can process all the data of the sequence in multiple threads, and further proposes to use TCN to analyze spatial features and Transformer to extract temporal features at the same time to achieve dual parallelism.

[0057] This method starts directly from the data itself. TCN identifies the spatial features of the data, and Transformer identifies the temporal features of the data, deeply mining the intrinsic connections between the data. TCN uses a causal convolutional network to ensure that there will be no omissions in the processing of data from the past to the future, ensuring the integrity of the analyzed data; by expanding the convolutional network, the sampling range is expanded, the receptive field is widened, and each layer can jump to extract the upper layer data and identify the spatial features of the data; the residual convolutional network is used to solve the gradient disappearance or explosion problem that may be caused by the expansion convolution, avoid the simple superposition between TCN layers, ensure the consistency of data information flow, improve the generalization ability of the network, and fully identify spatial features. SENet assigns feature weights, enhances effective features, reduces inefficient or invalid features, and simplifies the process of TCN identifying spatial features. Transformer enhances the ability to pay attention to the data as a whole through the multi-head attention mechanism, better captures the long-distance dependency between data, and identifies the temporal features of data. In addition, TCN and Transformer have the ability to process data in parallel, effectively reducing the prediction time, enhancing the prediction ability, and achieving dual parallelism. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is the flow chart of ultra-short-term wind power combination forecasting based on TCN-SENet-Transformer.

[0059] Figure 2 This is the mean square error loss curve for the training set and the test set.

[0060] Figure 3 Prediction curves for four models: TCN, Transformer, TTC and TSTC.

[0061] Figure 4 Comparison of local peak fluctuation curves of four models: TCN, Transformer, TTC and TSTC. DETAILED DESCRIPTION

[0062] The present invention will be further explained and illustrated below in conjunction with the embodiments, but the specific embodiments do not limit the present invention in any form.

[0063] Example 1

[0064] This embodiment provides an ultra-short-term wind power combination prediction model based on TCN-SENet-Transformer, including:

[0065] The TCN-SENet prediction model includes a TCN feature extraction module and a SENet channel attention module. TCN feature extraction is mainly composed of three parts: causal convolution, dilated convolution, and residual connections. Among them, causal convolution and dilated convolution are the ways in which TCN processes data, and the residual connection structure is the main body of the TCN structure. The residual connection network of this method is a two-layer structure. The input data is sent to the dilated convolution network, and then processed by weight normalization and ReLU activation function. Finally, the data is regularized and the above process is repeated. At the same time, the input data is sent to a 1*1 convolutional network for processing and superimposed with the regularized data.

[0066] The SENet channel attention module first performs global average pooling on the input data, reduces the dimension through the fully connected layer, processes the ReLU activation function, and then uses the fully connected layer to increase the dimension to the dimension before dimensionality reduction, processes it through the Sigmoid activation function, and weights it with the original data.

[0067] The Transformer prediction model consists of an encoder, which consists of three parts: multi-head attention, residual normalization (add&norm) and feedforward neural network (FFN). In the encoder structure, the input data is first processed by position encoding, one part is processed by multi-head attention mechanism and residual normalization, and the other part is directly subjected to residual normalization. Part of the processed data is sent to the feedforward neural network and residual normalization, and the other part is directly subjected to residual normalization, and finally the K and V matrices are obtained.

[0068] The CA prediction model consists of two parts. One part inputs the data processed by SENET to obtain the Q matrix, and the other part inputs the data processed by Transformer to obtain the K and V matrices, and outputs the data according to the attention weight relationship.

[0069] Example 2

[0070] This embodiment provides a super short-term wind power combination prediction method based on TCN-SENet-Transformer, such as Figure 1 As shown, the steps include:

[0071] S1. Perform z-score normalization on the original data set to convert each feature of the data to the same level, ensure data comparability, and enhance training accuracy. The z-score normalization formula is expressed as:

[0072]

[0073] Where: z is the standardized result, x is the input observation value, μ is the mean of the total data, and σ is the standard deviation of the total data.

[0074] S2. Capture the spatial features of the new dataset through a temporal convolutional network and adjust the feature weights using a compression-excitation network;

[0075] S21. Send the input data to the dilated convolutional network, the expression is:

[0076]

[0077] Where: G(t) is the output of the t-th element under the dilated convolution; k is the convolution kernel; n is the element ordinal number; g(n) is the n-th element filter value; Move the t element forward by d n The input value of .

[0078] The above process needs to meet the conditions of the causal convolutional network, so that the output at each moment is obtained only by the convolution of the current moment and the previous input.

[0079] Use padding to fill the sequence with zeros on the left and right so that the input data and output data have the same dimension. The padding calculation is as follows:

[0080] padding = (k-1)*d, d = 2 i

[0081] Where: k is the convolution kernel; d is the dilation factor; i is the number of levels.

[0082] S22. Perform weight normalization, change the weight of the deep network to speed up the processing process; use the ReLU activation function to introduce nonlinear factors into the network, enhance the ability to learn complex structures, and reduce gradient disappearance; regularize the data to prevent overfitting. This method has a two-layer structure, and F(x) is obtained by repeating the above process;

[0083] S23. Send the input data to the 1*1 convolutional network to ensure that the number of input and output channels of each layer remains consistent, and obtain the output H(x);

[0084] S24. The F(x) processed by H(x), i.e., f, is globally averaged pooled through a compression channel; the expression of the compression channel is:

[0085]

[0086] Where: f m is the compressed output; H×W is the spatial dimension; g m is the m-th two-dimensional input matrix; i, j are gm The number of rows and columns.

[0087] Through the excitation channel, the fully connected layer is used to reduce the dimension, and the ReLU activation function is added to enhance the nonlinear fitting ability of the model. It is sent to the fully connected layer to increase the dimension to the original dimension, and the Sigmoid activation function is used to speed up the network convergence. The sigmoid activation gating mechanism is expressed as:

[0088] s=σ(W2δ(W1z))

[0089] Where: s is the excitation output; σ is the sigmoid function; W1, W2 are the weight matrices of the fully connected layer; δ is the relu activation function; z is the input vector.

[0090] The compressed excitation channel is used to process s and the input f to weight them, that is, the activation s rescales g m Get the final output x m :

[0091] x m =s m *g m

[0092] Where: x m is the final output; s m is the weight value.

[0093] Assign feature weights according to the loss function and output the feature matrix Q.

[0094] S3. Encode the original sequence through multi-head attention mechanism, residual normalization and feedforward neural network processing, and then calculate the output matrix Z to obtain matrix K and matrix V;

[0095] The input data is positionally encoded using sine and cosine functions of different frequencies. The expression is:

[0096]

[0097] Where: pos is the position, i is the dimension, d is model is the embedding dimension.

[0098] The encoded data is fed into the multi-head attention mechanism. The self-attention expression of the multi-head attention mechanism is:

[0099]

[0100] Where: softmax is the normalization function; Q is the query matrix; K is the key matrix; V is the value matrix; d k is the vector dimension.

[0101] Residual normalization is used to enhance the learning ability of deep networks and avoid gradient disappearance. The expression is:

[0102] LayerNorm(X+MultiHeadAttention(X))

[0103] LayerNorm(X+FeedForward(X))

[0104] Where: X represents the input of the multi-head attention or feedforward neural network, MultiHeadAttention(X) and FeedForward(X) represent the output.

[0105] The fully connected layer is used to enhance the nonlinear fitting ability of the model. The expression is:

[0106] max(0,XW1+b1)W2+b2

[0107] Where: X is the input; W1, W2 are the input matrices; b1, b2 are the deviations.

[0108] Then use the residual normalization to get the output matrix K, V.

[0109] S4. Input the matrix Q, matrix K and matrix V into the CA model, calculate the attention weight distribution, and output the result. The self-attention expression of the cross attention mechanism adopted by the CA model is:

[0110]

[0111] Where: softmax is the normalization function; Q is the query matrix; K is the key matrix; V is the value matrix; d k is the vector dimension.

[0112] Example 3

[0113] This embodiment provides a simulation experiment to test the effectiveness of the combined prediction model of TCN-SENet-Transformer-CA (TSTC for short) that combines cross-attention and spatiotemporal features, and selects three models, TCN, Transformer, and TCN-Transformer-CA (TTC for short), for comparative analysis with the proposed combined prediction model.

[0114] This embodiment takes data_processed (processed data) of a wind farm in China (wind farm site6) as an example. The sampling period is 15 minutes, and the data from 2020 / 1 / 1 0:00 to 2020 / 12 / 31 23:45 are taken, with a total of 35039 valid sampling points. The z-score of the data set used is first standardized so that the features of the original data are at the same order of magnitude. The sliding window method is used to traverse the data, and the sliding window length is 16 to construct a new data set for easy training of the model. The original data is divided into training set and test set according to a ratio of 9:1. The training set has a total of 31609 sampling points and the test set has a total of 3498 sampling points. The computing platform is Win10 system, Intel i7-1165G7@2.80GHz, and RAM 16.0GB. The combined model is programmed in Python3.9.

[0115] Depend on Figure 2 From the mean square error loss training set and test set curves of the four model data shown, it can be seen that Transformer has the worst training effect on mean square error loss, and the test value is roughly distributed at 0.008, which differs from the training value by about 0.0075 at most, and the test set curve does not have an obvious trend of the training set curve; TCN has a poor training effect when the number of iterations is 0-30, with a maximum difference of about 0.002, and the best training effect is 30-50. The test set curve tends to overlap with the training set curve, and the overall curve trend is consistent with the training set curve trend; TTC and TSTC test set curves are closest to the training set curve, among which TTC has a test value less than the training value when the number of iterations is 0-10 and a maximum difference of about 0.0002, and the test set curve of 10-50 almost overlaps with the training set curve; TSTC has a test value higher than the training value when the number of iterations is 0-3 and a difference of about 0.0005, and the test set curve of 3-50 iterations almost completely overlaps with the training set curve.

[0116] TCN prediction curve is shown in Figure 3 a), we can see that TCN is not good enough to predict the floating of data between local peaks at sampling points 300-400, 500-600, 700-950, 1150-1200, 1800-2100, 2600-2650, 2850-2900, and 3200-3250. For other parts, the predicted curves almost coincide with the original data curves.

[0117] Transformer prediction curve Figure 3b), it can be seen that compared with the original data curve, the Transformer prediction curve is obviously ahead of the original data curve by a certain time step, and the overall prediction trend is similar to the original curve. Between the sampling points 100-600, 1100-1300, 2200-2300, and 3100-3300, the prediction of fluctuating data is poor, and the prediction of mutation data is missing.

[0118] TTC prediction curve Figure 3 c), it can be seen that like TCN, TTC also has the problem of insufficient prediction, but in the sampling point interval mentioned by TCN above, TTC's prediction degree is slightly deeper than TCN, but TTC's prediction effect on mutation data is slightly worse than TCN, such as sampling points 450, 700, 1250, 3450, etc.

[0119] TSTC prediction curve is shown in Figure 3 d), we can see that TSTC solves the problem of insufficient prediction of sampling points 300-400, 500-600, and 700-950 by TCN and TTC, and solves it through the Transformer after attention allocation, and has good prediction effect for fluctuating data near local peaks. And the prediction effect for mutation data is slightly better than TTC, such as the sampling points mentioned above, but slightly worse than TCN.

[0120] Depend on Figure 4 It can be seen that compared with the other three prediction models, Transformer has an earlier prediction time and a deeper prediction depth; TCN has a deeper prediction for mutation data, but insufficient prediction for fluctuation data near the local peak; TTC has a deeper prediction for fluctuation data near the local peak, but insufficient prediction for mutation data; TSTC uses SENet and CA to well balance the prediction degree of Transformer and TCN for fluctuation data near the local peak and mutation data, and the prediction curve is closer to the original curve.

[0121] The prediction error comparison of the model is shown in Table 1:

[0122] Table 1

[0123]

[0124] As can be seen from Table 1, the combined prediction model τ R2 , τ rmse , τ mae , τ mape Compared with the other three models, the indicators are the best, the error value is the smallest, and the prediction accuracy is high. Compared with the TCN, Transformer, and TTC models, this model Closest to 1, τ rmse They were reduced by 3.08%, 81.93%, and 5.50% respectively; τ maeThey were reduced by 8.29%, 82.73%, and 10.68% respectively; τ mape The reductions were 36.94%, 94.96% and 13.53% respectively. It can be seen that the prediction model of the present invention has higher prediction accuracy and better comprehensive data.

[0125] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for ultra-short-term wind power combination forecasting based on TCN-SENet-Transformer, characterized in that the steps include: S1. Standardize and preprocess the original data set; S2. Capture the spatial features of the new dataset through a temporal convolutional network and adjust the feature weights using a compression-excitation network; S21. Input the preprocessed data into the dilated convolutional network, and then use padding to fill the sequence with zeros on the left and right so that the input data and output data have the same dimension; S22. Perform weight normalization, use the ReLU activation function, introduce nonlinear factors into the network, and obtain F(x) after regularization; S23. Input the preprocessed data into the dilated convolutional network to ensure that the number of input and output channels of each layer remains consistent, and obtain the output H(x); S24. Process H(x) to obtain F(x), and then perform global average pooling through the compression channel; S25. Through the excitation channel, use the fully connected layer to reduce the dimension, add the ReLU activation function to enhance the nonlinear fitting ability of the model, send it to the fully connected layer to upgrade the dimension to the original dimension, and use the Sigmoid activation function to speed up the network convergence; S26. Use the compressed excitation channel to process s and input f to weight the final output x m , assign feature weights according to the loss function and output the feature matrix Q; S3. Encode the original sequence through multi-head attention mechanism, residual normalization and feedforward neural network processing, and then calculate the output matrix Z to obtain matrix K and matrix V; S31. Position encoding the preprocessed data using sine and cosine functions of different frequencies; S32. Send the encoded data to the multi-head attention mechanism and then use residual normalization; S33. Use the fully connected layer to enhance the nonlinear fitting ability of the model, and then use the residual standardization to obtain the output matrix K and matrix V; S4. Input matrix Q, matrix K and matrix V into the CA model, calculate the attention weight distribution, and output the result.

2. The ultra-short-term wind power combined forecasting method based on TCN-SENet-Transformer according to claim 1 is characterized in that: The dilated convolutional network is expressed as: Where: G(t) is the output of the t-th element under the dilated convolution; k is the convolution kernel; n is the element ordinal number; g(n) is the n-th element filter value; Move the t element forward by d n The input value of .

3. The ultra-short-term wind power combined forecasting method based on TCN-SENet-Transformer according to claim 1 is characterized in that: The expression of the compression channel is: Where: f m is the compressed output; H×W is the spatial dimension; g m is the m-th two-dimensional input matrix; i, j are g m The number of rows and columns.

4. The ultra-short-term wind power combined forecasting method based on TCN-SENet-Transformer according to claim 1 is characterized in that: The sigmoid activation function is expressed as: s=σ(W2δ(W1z)) Where: s is the excitation output; σ is the sigmoid function; W1, W2 are the weight matrices of the fully connected layer; δ is the relu activation function; z is the input vector.

5. The ultra-short-term wind power combined forecasting method based on TCN-SENet-Transformer according to claim 1 is characterized in that: The final output x m It is expressed as: x m =s m *g m Where: x m is the final output; s m is the weight value.

6. The ultra-short-term wind power combined forecasting method based on TCN-SENet-Transformer according to claim 1 is characterized in that: The sine and cosine functions are expressed as: Where: pos is the position, i is the dimension, d is model is the embedding dimension.

7. The ultra-short-term wind power combined forecasting method based on TCN-SENet-Transformer according to claim 1 is characterized in that: The self-attention expression of the multi-head attention mechanism is: Where: softmax is the normalization function; Q is the query matrix; K is the key matrix; V is the value matrix; d k is the vector dimension.

8. The ultra-short-term wind power combined forecasting method based on TCN-SENet-Transformer according to claim 1 is characterized in that: The residual standardization process is expressed as: LayerNorm(X+MultiHeadAttention(X)) LayerNorm(X+FeedForward(X)) Where: X represents the input of the multi-head attention or feedforward neural network, MultiHeadAttention(X) and FeedForward(X) represent the output.

9. The ultra-short-term wind power combined forecasting method based on TCN-SENet-Transformer according to claim 1 is characterized in that: The expression of the fully connected layer enhancement model is: max(0,XW1+b1)W2+b2 Where: X is the input; W1, W2 are the input matrices; b1, b2 are the deviations.

10. The ultra-short-term wind power combined forecasting method based on TCN-SENet-Transformer according to claim 1 is characterized in that: The self-attention expression of the cross-attention mechanism in the CA model is: Where: softmax is the normalization function; Q is the query matrix; K is the key matrix; V is the value matrix; d k is the vector dimension.

Images