An energy short-term load prediction method and system based on SE-Block improved transformer

Abstract

This invention relates to the field of energy forecasting technology, and in particular to a method and system for short-term energy load forecasting based on an improved Transformer using SE-Block. The method includes reversible normalization preprocessing of acquired multivariate load sequence data; feature extraction and fusion of the preprocessed data using improved cross-scale interactive patching, including multi-scale feature extraction, cross-scale interactive alignment, residual correction, and dynamic fusion; and feature filtering of the fused features based on a channel attention mechanism, including feature response based on improved SE-Block and nonlinear interaction of context vectors. This invention addresses the non-stationarity of actual load caused by meteorological conditions and user behavior. By automatically eliminating noise interference among multiple variables, the model accurately depicts the fluctuation details of the load curve, demonstrating its robustness in multivariate load forecasting for integrated energy systems.

Description

Technical Field

[0001] This invention relates to the field of multi-scale feature fusion technology, and in particular to a method and system for short-term energy load forecasting based on SE-Block improved Transformer. Background Technology

[0002] With the adjustment of the global energy structure and the rapid development of the scale and complexity of integrated energy systems (IES), the importance of high-precision multi-source load forecasting has significantly increased. Integrated energy systems not only include electrical systems but also integrate various energy media, such as heating, cooling, gas, energy storage, and renewable energy. High-precision load forecasting is crucial for optimizing energy dispatch, improving energy efficiency, and ensuring system stability and economic feasibility.

[0003] In recent years, deep learning has been widely used in load forecasting of integrated energy systems. Existing deep learning prediction methods can be divided into two categories: traditional deep learning methods based on recurrent and convolutional layers, and methods based on Transformers. Traditional deep learning methods based on recurrent and convolutional layers utilize recurrent units or convolutional kernels to capture local features and temporal characteristics of time-series data. For example, RNNs recursively pass information along the time axis through hidden states, and CNNs extract local features through convolutional kernels and expand the field of view using dilated convolutions. These traditional deep learning prediction methods can achieve fast convergence and excellent performance on small sample data. However, due to the vanishing gradient problem, they struggle to capture long-distance dependencies, and the limited receptive field leads to a surge in model parameters, making training difficult if longer historical data needs to be captured. Transformer-based prediction methods, on the other hand, can obtain a global perspective through self-attention mechanisms, thus directly establishing long-distance temporal correlations. Furthermore, these methods can simultaneously compute all time steps, greatly utilizing GPU computing power and improving training speed.

[0004] While the traditional Transformer architecture excels at capturing long-range temporal dependencies thanks to its self-attention mechanism, the point-to-point attention computation method employed by Transformer not only leads to increased computational complexity... This increases the complexity of the data and limits the model's ability to handle longer historical windows. Furthermore, because traditional Transformers focus on point-to-point correlations, while time-series data characteristics are often embedded in a waveform rather than a single numerical value, traditional Transformers often lose local semantic features without special processing. Summary of the Invention

[0005] To address the challenges of simultaneously handling long-distance dependencies and transient abrupt changes, as well as multivariate coupling noise and high computational complexity in comprehensive energy load forecasting, this invention provides a method and system for short-term energy load forecasting based on an improved Transformer using SE-Block. It utilizes a channel-independent strategy to eliminate noise interference between different loads, combines cross-scale interactive patching with a frequency-domain-aware SE-Block module to decouple high, medium, and low-frequency domain features, and employs a dual-path attention mechanism to fuse Transformer Encoders. This reduces computational complexity while accurately capturing temporal dependencies across long periods, overcoming the memory limitations of traditional networks.

[0006] In a first aspect, the present invention provides a short-term energy load forecasting method based on an improved Transformer using SE-Block, employing the following technical solution:

[0007] A short-term energy load forecasting method based on SE-Block improved Transformer includes:

[0008] Obtain multivariate load sequence data;

[0009] Reversible normalization preprocessing is performed based on the acquired multivariate load sequence data;

[0010] Improved cross-scale interactive patching is used to extract and fuse features from preprocessed data, including multi-scale feature extraction, cross-scale interactive alignment, residual correction and dynamic fusion.

[0011] Feature selection is performed on the fused features based on the channel attention mechanism, including feature response based on improved SE-Block and non-linear interaction of context vectors;

[0012] Global long-distance temporal dependency modeling is performed on the selected features based on Transformer;

[0013] Output multidimensional prediction results based on channel merging strategy.

[0014] Secondly, an energy short-term load forecasting system based on SE-Block improved Transformer includes:

[0015] The data acquisition module is configured to acquire multivariate load sequence data;

[0016] The preprocessing module is configured to perform reversible normalization preprocessing based on the acquired multivariate load sequence data;

[0017] The feature fusion module is configured to extract and fuse features from preprocessed data using improved cross-scale interactive patching, including multi-scale feature extraction, cross-scale interactive alignment, residual correction and dynamic fusion.

[0018] The feature selection module is configured to perform feature selection on the fused features based on the channel attention mechanism, including feature response based on improved SE-Block and non-linear interaction of context vectors.

[0019] The global modeling module is configured to perform global long-distance temporal dependency modeling on the selected features based on Transformer.

[0020] The prediction module is configured to output multidimensional prediction results based on a channel merging strategy.

[0021] Thirdly, the present invention provides a computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the aforementioned short-term energy load forecasting method based on SE-Block improved Transformer.

[0022] Fourthly, the present invention provides a terminal device, including a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, the instructions being adapted to be loaded and executed by the processor to provide the described method for short-term energy load forecasting based on an improved Transformer using SE-Block.

[0023] In summary, the present invention has the following beneficial technical effects:

[0024] In response to the strong high-frequency fluctuations and non-stationary characteristics of electrical load, the model accurately captures the extreme points of peaks and troughs and the details of rapid transitions, avoiding the lag phenomenon commonly found in traditional Transformers. This demonstrates that cross-scale interactive patching can compensate for the phase drift problem that may exist between high-frequency details extracted from multi-scale features and long-term trends. It also shows that the local attention mechanism of this invention can effectively perceive high-frequency features and adaptively increase the weight of convolutional branches to accurately depict more fluctuation details of the load curve.

[0025] For the cold load forecast, the forecast results achieved precise alignment in both phase and amplitude, indicating that the global linear attention mechanism can efficiently aggregate long historical context information while reducing computational complexity and suppressing the interference of high-frequency noise on the smooth trend.

[0026] For heat load forecasting, the forecast results effectively isolate random noise interference while maintaining the overall load trend smooth and stable. This indicates that the model suppresses noise interference from invalid channels through spectral feature recalibration, and achieves the retention of long-term heat load trend memory and the identification and isolation of local high-frequency noise disturbances.

[0027] In summary, addressing the non-stationarity of actual load due to weather conditions and user behavior, the model accurately depicts the fluctuation details of the load curve by automatically eliminating noise interference among multiple variables, demonstrating its robustness in multivariate load forecasting for integrated energy systems. By deeply mining the temporal patterns of historical loads, the model effectively captures long-term time dependencies, ensuring the continuity of forecast results over long time scales. Furthermore, through the extraction of rich local semantic features, the model achieves accurate capture of short-term load fluctuations, fully demonstrating its effectiveness in integrated energy load forecasting tasks. Attached Figure Description

[0028] Figure 1 This is a framework diagram of the prediction model for Example 1 of the present invention.

[0029] Figure 2 This is a schematic diagram of the actual electrical load curve from 2020 to 2023 for Example 1 of the present invention.

[0030] Figure 3 This is a schematic diagram of the actual cooling load curve from 2020 to 2023 for Example 1 of the present invention.

[0031] Figure 4 This is a schematic diagram of the actual heat load curve from 2020 to 2023 for Example 1 of the present invention.

[0032] Figure 5 This is a schematic diagram of the cross-scale interactive patching structure of Example 1 of the present invention.

[0033] Figure 6 This is a schematic diagram of the prediction flowchart for Example 1 of the present invention.

[0034] Figure 7 This is a schematic diagram of the SE-Block structure in Example 1 of the present invention.

[0035] Figure 8 This is a schematic diagram of the Transformer Encoder structure in Example 1 of the present invention.

[0036] Figure 9 This is a schematic diagram of the electrical load prediction results in Example 1 of the present invention.

[0037] Figure 10 This is a schematic diagram of the cooling load prediction results in Example 1 of the present invention.

[0038] Figure 11 This is a schematic diagram of the heat load prediction results in Example 1 of the present invention. Detailed Implementation

[0039] The present invention will be further described in detail below with reference to the accompanying drawings.

[0040] Example 1

[0041] Reference Figure 1 This embodiment of a method for short-term energy load forecasting based on an improved Transformer using SE-Block includes:

[0042] Multivariate load series data often exhibit non-stationarity. Reversible normalization is used to standardize the data and address this non-stationarity. A channel-independent strategy is employed to decompose multidimensional load variables into independent univariate sequences that do not interfere with each other. This isolates noise interference between different loads while significantly increasing the number of training samples, preventing the model from learning incorrect coupling relationships. Traditional Transformers use point-to-point attention calculations, which not only increases computational complexity but also limits the model's ability to handle longer historical windows. Therefore, this invention introduces cross-scale interactive patching, using convolutional kernels of different sizes to reconstruct continuous time points into patch fragments with features in different frequency domains (high, medium, and low). Furthermore, a cross-attention mechanism is introduced to allow the model to consider overall trend characteristics when calculating detailed features, endowing high-frequency features with the ability to perceive global trends. A residual correction strategy is used to achieve simultaneous and accurate characterization of transient changes in electrical load and long-term inertia of thermal load.

[0043] Building upon this foundation, a Squeeze-and-Excitation module based on frequency domain characteristics is introduced to adaptively recalibrate the weights of feature channels in each frequency band, enhancing dominant frequency domain features and suppressing noise interference from irrelevant frequency bands. To preserve the temporal location information of time slices, learnable positional encodings are superimposed before entering the Transformer. A Transformer encoder module based on a dual-path attention mechanism is constructed, utilizing two parallel paths—global linear attention and local convolutional attention—to achieve real-time matching between computational patterns and load variable characteristics. Since multivariate load data exhibits significant diurnal and weekly periodicity, the memory of traditional LSTM decays with increasing time steps, while the Transformer Encoder, utilizing Self-Attention, can capture long-distance dependencies. Traditional sequence prediction models such as LSTM, GRU, or the standard Transformer Decoder employ a serial iterative approach for multi-step prediction. In contrast, the Transformer model of this invention uses only an Encoder and a Flatten Head, flattening the high-dimensional context of all time slices from the encoder output in a single step. Future predictions are directly generated through linear projection, and all historical information is processed in parallel for direct multi-step prediction, effectively avoiding the error accumulation problem inherent in serial iteration and significantly improving inference speed. The final prediction output is then restored to the original scale of electrical, cold, and thermal multivariate prediction results through channel reorganization and inverse normalization operations. The model framework is as follows: Figure 1 As shown.

[0044] The first step is data standardization and independent channel reconstruction.

[0045] This invention uses data from the University of Arizona's Integrated Energy System from January 1, 2020 to December 31, 2023. The dataset includes electricity, cooling, and heating load data with a time resolution of 1 hour and approximately 35,000 sample points. The electricity, cooling, and heating load curves are shown below. Figure 2 , Figure 3 and Figure 4 As shown.

[0046] From the overall distribution characteristics of the data, electricity and cooling loads exhibit a strong seasonal correlation. Influenced by the local hot climate, both electricity and cooling loads show a trend of high in summer and low in winter, with peak values ​​concentrated between May and September each year. In addition, heating load demand is relatively stable throughout the year, showing a clear characteristic of high in winter and low in summer, mainly concentrated from December to February of the following year, showing a significant complementary relationship with electricity and cooling loads on a seasonal scale. The three types of loads—electricity, cooling, and heating—exhibit strict daily and weekly periodicity. The daily periodicity varies with work and rest schedules, while the weekly periodicity shows a significant difference between high on weekdays and low on weekends. This periodic pattern reflects the impact of user energy consumption habits on load changes.

[0047] Since ASU data originates from real building management systems, outliers are inevitable. Therefore, load data threshold filtering and multi-level interpolation are performed first. For periods of extreme high temperatures, cooling load data may exhibit spikes; therefore, robust normalization based on quantiles is employed.

[0048] ,

[0049] in, These are the normalized values. It is a specific value in the original input sequence. It is the median of the dataset. It is the first quartile. It is the third quartile.

[0050] It is the interquartile range, which acts as a scaling factor here.

[0051] A sliding window is used to slice the original long-sequence data, with a window length of 168 hours (i.e., the past 7 days), to predict the next 24 hours. The window slides along the time axis with a fixed step size, each time extracting a continuous sequence of length 168 as the model input, and using the immediately following 24 time steps as the corresponding ground truth labels. The input data is then reshaped into a standard three-dimensional tensor format [B, L, V], as shown below. Figure 5 As shown, Batch Size represents the batch size, Variables represents the variable dimension, and Length represents the time step dimension, which is fixed at 168.

[0052] To address the issue of data non-stationarity, the input electrical, cooling, and heating load data is reversibly normalized before independent channel operation, as shown in the following formula:

[0053] ,

[0054] in, The output vector is normalized and rescaled. It is the input vector of this layer (i.e., the output of the previous layer). Input vector The mean, It is the input vector The standard deviation. It is a very small constant to prevent the denominator from being zero and to ensure numerical stability. It is a learnable scaling parameter These are learnable translation parameters.

[0055] One channel in a multivariate load sequence may contain a significant amount of noise, which, in channel mixing mode, can be projected onto other channels through the embedding space. To enable the Transformer to focus more on capturing temporal features and avoid interference from noise between variables, a channel-independent strategy is employed to process the data.

[0056] Traditional models treat electrical, cooling, and heating loads as a single vector input. Channel-independent strategies, through tensor reshaping, effectively isolate noise interference between different load types, allowing each variable to focus more on its own temporal characteristics. Simultaneously, this significantly expands the amount of training data, substantially improving the model's generalization ability and robustness with limited data. The formula is as follows:

[0057] ,

[0058] Wherein, the original input tensor is ,in For batch size, For window length, For the number of variables, For a channel-independent tensor, Permute & Reshape are the permutation and reshaping operations of the tensor dimensions.

[0059] like Figure 5 As shown, the batch of samples is converted into batch*variable samples, thereby increasing the number of training samples and avoiding the model learning incorrect coupling relationships.

[0060] The second step is cross-scale interactive patching.

[0061] The cross-scale interactive patching module consists of three steps: multi-scale feature extraction, cross-scale interaction, residual correction, and dynamic fusion.

[0062] Multi-scale feature extraction is a temporal slice feature extraction method that addresses the traditional point-to-point attention computation method of Transformer by reconstructing continuous time points into patch fragments containing local feature semantics. This solves the problem of lack of local semantics in point-to-point input, significantly reduces the computational and memory overhead of Transformer, and enables it to efficiently process long-term historical data.

[0063] Traditional patching methods use a single-window static processing approach to uniformly extract single-scale features from the three load variables. This often leads to a problem: if the window is too small, it can accurately capture the trend characteristics of electrical load, but because the information within the window is too limited, it cannot identify the long-term trends of cold and hot loads. If the window is too large, it can capture the smooth trends of cold and hot loads, but it easily overlooks the instantaneous changes in electrical load, resulting in large prediction errors. Multi-scale patching feature extraction, on the other hand, uses three one-dimensional convolutional layers with different kernel sizes to capture high-frequency instantaneous details, mid-frequency diurnal periodic patterns, and low-frequency trends, respectively.

[0064] Using one-dimensional convolution operation on dimension The original payload sequence was reconstructed in spatiotemporal dimensions. By setting different convolutional kernel sizes and strides (8, 16, and 32 respectively), high-frequency details, mid-frequency transitions, and low-frequency trend features were extracted. The high-frequency branch aggregated 168 time points into 21 patches, while the mid-frequency and low-frequency branches, due to their larger strides, naturally generated feature sequences shorter than 21.

[0065] ,

[0066] ,

[0067] ,

[0068] in, For the input time series data, This is a one-dimensional convolution operation. To represent the size of the convolution kernel, the values ​​are 8, 16, and 32, respectively. These are high-frequency, mid-frequency, and low-frequency feature tensors, with dimensions of: .

[0069] To address the issue of inconsistent feature sequence lengths caused by multi-scale convolution, a forced alignment operation is performed. Using the feature length of 21 for the high-frequency branch as a baseline, a linear interpolation algorithm is used to temporally upsample the shorter feature sequences of the mid-frequency and low-frequency branches, restoring their length to 21. After this operation, the features from all frequency branches are ultimately generated into tensors of consistent dimension. .

[0070] ,

[0071] in, For the aligned feature tensor, For alignment operations, To set the alignment size, .

[0072] Finally, a cross-scale feature fusion operation is performed. A channel-stitching strategy is used to stack the feature tensors of the three independent branches along the feature channel dimension. This operation expands the feature dimension of the tensor from the original single-branch dimension of 128 to three times, i.e., 384. At this point, the overall dimension of the data increases from... Become The model introduces a learnable linear projection layer, which maps the 384-dimensional mixed features back to a 128-dimensional latent space through fully connected operations. The model automatically learns the weight matrix during training, determining whether high-frequency mutation information or low-frequency trend information is more important in the current time slice. This generates a comprehensive feature vector that considers both local details and global contours, restoring the tensor dimension to [missing information]. .

[0073] ,

[0074] in, It is a learnable fusion projection matrix. For bias terms, It is the fused multi-scale feature tensor. Cross-scale attention alignment.

[0075] In time-series data, isolated high-frequency details are easily misclassified as noise, and these details may exhibit slight phase shifts relative to the overall trend. To compensate for the potential phase drift issue of high-frequency details extracted from multi-scale features relative to the long-term trend, a cross-attention mechanism is introduced.

[0076] By using high-frequency features as query vectors and low-frequency trend features as key-value vectors, the model considers the overall trend characteristics when calculating detailed features, compensating for the lack of global perspective in local features. While keeping the data dimensions unchanged, it endows high-frequency features with the ability to perceive global trends.

[0077] ,

[0078] ,

[0079] ,

[0080] in, These are the query, key, and value vectors, respectively. and These are low-frequency characteristics and high-frequency characteristics, respectively. are learnable linear projection matrices used to map features to the attention space.

[0081] The core operation for performing cross-attention utilizes the query vector. With key vector The transpose of the expression is used to perform a dot product operation to calculate the correlation score between high-frequency details and low-frequency trends, and then divided by the scaling factor. This is to prevent gradient vanishing. The weights are then processed by the Softmax function to generate a normalized attention weight map.

[0082] ,

[0083] in, The attention score matrix, Scaling factor For normalization function, For attention output.

[0084] Finally, to preserve the integrity of the original high-frequency information and prevent network degradation, a residual connection structure is employed, directly adding the attention-enhanced features to the original high-frequency input features. The result of this addition is processed through layer normalization, ultimately generating an aligned feature tensor that fuses cross-scale information. .

[0085] ,

[0086] in, It is the original high-frequency feature input, that is, the high-frequency branch features that have not been processed by attention. It is attention output. To achieve layer normalization, the features are standardized. This is the final alignment feature.

[0087] Residual gated correction and dynamic fusion

[0088] By leveraging the difference between the aligned high-frequency features and the mid-frequency baseline, a sigmoid gating mechanism is used to adaptively correct the mid-frequency features, suppressing random noise while preserving effective abrupt changes. Global pooling and an MLP network are used to dynamically calculate the importance weights for the three scales, and the high-frequency details, mid-frequency baseline, and low-frequency trend branches are weighted and summed. Finally, this module outputs a high-dimensional semantic vector with dimensions [32*3, 21, 128].

[0089] First, the difference between the high-frequency features after cross-attention alignment and the original mid-frequency features is calculated. This is done using the formula... The residual tensor is obtained.

[0090] To determine the validity of the residual information, a learnable gating network is introduced. The residual is input into a linear layer and processed by a sigmoid activation function to generate gating coefficients.

[0091] ,

[0092] in, The difference between the aligned high-frequency characteristics and the intermediate-frequency reference. The learnable weights and biases of the gated network. For activation function, A numerical tensor between 0 and 1.

[0093] Dynamic correction of intermediate frequency characteristics utilizes generated gating coefficients Nonlinear corrections are applied to the intermediate frequency (IF) characteristics. While preserving the stability of the IF reference characteristics, selected high-frequency details are incorporated.

[0094] ,

[0095] in, For element-wise multiplication, Corrected mid-frequency characteristics The difference between the aligned high-frequency characteristics and the intermediate-frequency reference. The gating coefficient, This is a mid-frequency reference characteristic.

[0096] Finally, the model performs global weighted fusion, applying global average pooling to each of the three branches to obtain three global context vectors. .

[0097] ,

[0098] in, For global feature vectors, The aligned high-frequency feature tensor is a high-frequency feature that incorporates the low-frequency trend context. The corrected intermediate frequency feature tensor is an intermediate frequency feature that integrates high-frequency mutation information through a gated residual network. This is the original low-frequency feature tensor. This is a global average pooling operation.

[0099] These three vectors are concatenated and passed through a multilayer perceptron (MLP) and a softmax function to calculate three dynamic weights. The final output is the weighted sum of the three branches.

[0100] ,

[0101] ,

[0102] in, It is a multilayer perceptron. For weight fusion. For normalization function, This is the final output tensor.

[0103] The output data is input into the prediction framework described above, and the calculation flowchart is as follows: Figure 6 As shown.

[0104] The third step is feature selection based on the attention mechanism.

[0105] The Squeeze-and-Excitation Block is a lightweight channel attention mechanism module. Its core idea is to adaptively readjust channel feature weights by explicitly modeling the interdependencies between channel features. Traditional SE-Block modules typically use global average pooling to compress spatial or temporal information. This aggregation method, when processing IES data, smooths out the differences in physical characteristics of different loads across time scales, limiting the model's ability to model complex coupling relationships among multiple loads. For example, global pooling can over-smooth the spike features of electrical loads, leading to the loss of abrupt changes. This module aims to build a channel attention mechanism that can perceive the physical characteristics of load variables, setting specific feature extraction perspectives for different types of loads, enabling them to extract features at their most accurate time scales, thereby more sensitively capturing transient changes in electrical loads and inertial trends in thermal loads. The entire processing can be divided into the following four key stages:

[0106] The Squeeze operation compresses the information of each channel across all patches into a single value, characterizing the feature response of that channel by capturing global information for each channel. This module modifies the Squeeze stage of the traditional SE module based on the frequency domain characteristics of the load variable, employing dual-path feature extraction. Using a frequency domain adaptive convolution strategy, it first performs view reconstruction and feature separation, reconstructing the input feature tensor from a single batch dimension into independent dimensions containing both batch and variable information, allowing the model to clearly distinguish which type of load is being processed.

[0107] For input data To reshape the dimensions and organize them into The standard form.

[0108] ,

[0109] in, These are the original input features. These are the reconstructed input features; Reshape is the only reshaping operation. Batch Size Variables refers to the number of load variables such as electricity, cooling, and heating. The number of patches. It is the feature dimension, the feature vector dimension of each patch.

[0110] Execute the first-way Squeeze operation, in the time dimension Global average pooling is performed on the input features.

[0111] ,

[0112] ,

[0113] in, This is the global statistical baseline feature vector. This is a global pooling operation, which sums and averages along the time dimension. These are the reconstructed input features. This indicates that the operation is performed on all batches, all variables, the i-th time step, and all feature dimensions.

[0114] The model abandons the general global pooling approach, instead applying convolutional kernels of different sizes to aggregate features for different variables. For electrical load, a very small kernel (Kernel=1) is used to preserve the features at the current moment, ensuring that instantaneous spikes and high-frequency fluctuations in electrical load are not diluted by historical averages. For cooling load, a medium-sized kernel (Kernel=9) is used, and for heating load, a large kernel (Kernel=13) is used to simulate thermal inertia and transmission delays in the heating system. By observing extremely long time windows, the model can capture deep-seated heat accumulation effects and slowly changing trends, ensuring that the prediction results conform to the gradual laws of thermodynamics. Furthermore, the model continues the traditional SE-Block module approach, performing global average pooling in the time dimension, compressing the features of each variable within the time window into a single variable, representing the global baseline level of the variable, facilitating subsequent operations.

[0115] In parallel, the module performs a second frequency domain feature sensing operation. This is achieved using a one-dimensional convolutional layer. Local filtering of input features, convolution kernel Capture local dependencies and lag effects in time series data to generate lag feature tensors. .

[0116] ,

[0117] in, The lag feature tensor is the output of the convolution operation and contains local dynamic information. It is a one-dimensional convolution operation. It is the convolution kernel. It is a slice of the v-th variable in the input tensor.

[0118] Subsequently, the same global average pooling is performed on the hysteresis feature to obtain the frequency domain feature context vector, ensuring that the model not only focuses on the overall mean, but also perceives the physical frequency domain features hidden in the local dynamics.

[0119] ,

[0120] ,

[0121] in, This is a local dynamic context feature vector. This is a global pooling operation, which sums and averages along the time dimension. For the hysteresis characteristic tensor, This indicates that the operation is performed on all batches, the i-th time step, and all feature dimensions.

[0122] The pre-excitation fusion operation employs a multiplicative residual modulation strategy, organically integrating global baseline features with frequency-domain adaptive convolutional features to construct a dynamic gain structure. The frequency-domain adaptive convolutional features serve as the dynamic gain coefficients for the global baseline. When the convolutional kernel captures significant thermal inertia accumulation or high-frequency electrical load details, it can adaptively amplify the corresponding global feature amplitude. This operation occurs before the weights are generated in the fully connected layer. Its purpose is to overcome the limitation of traditional SE modules relying solely on global pooling statistical averages, ensuring that subsequent excitation operations, when calculating channel importance, consider not only numerical values ​​but also the physical dynamic characteristics of the waveform.

[0123] To combine global trends with local physical properties, the model performs a feature fusion operation. The global context vector and the physical context vector are non-linearly interacted. Using the physical hysteresis feature as a gain coefficient, the global trend feature is adaptively enhanced or suppressed, resulting in a fused vector. .

[0124] ,

[0125] in, Local dynamic context-oriented feature vectors, Features after fusion This is the global statistical baseline feature vector. This is the Hadamard product, performed element-wise multiplication. 1 is the unit base value. When it approaches 0, Approaching 1, at this point This ensures that, in the worst case, the model can at least retain the original global features and will not lose basic information due to modulation operations.

[0126] After obtaining the enhanced feature vectors from the pre-fusion operation, the model performs excitation and scaling operations to further capture the non-linear dependencies between channels and complete the final feature selection. In the excitation stage, the enhanced features are fed into a layer consisting of two fully connected layers. The bottleneck structure is formed by dimensionality reduction layers and activation functions. Nonlinearity is introduced and features are compressed to capture the dependencies between channels. Then, the dimensionality is restored through an upscaling layer, and the Sigmoid activation function is utilized. The output is mapped to the interval between 0 and 1. This ultimately generates the channel attention weight vector. .

[0127] ,

[0128] in, These are the weight matrices for dimensionality reduction and dimensionality increase, respectively. These are the ReLU activation function and the sigmoid activation function, respectively. This is the feature vector fused from the previous step. This represents the channel attention weight.

[0129] During the scaling phase, the model performs element-wise multiplication, applying the generated channel attention weights back to the original input feature vector. At this point, the model no longer simply learns correlations based on traditional statistical values, but rather employs a recalibration mechanism that fuses statistical and frequency domain features. The formula is as follows:

[0130] ,

[0131] in, For channel attention weights, For Hadamard product, perform element-wise multiplication. This is the raw data before entering SE-Block. This is the final output feature.

[0132] The structure of the SE Block is as follows: Figure 7 As shown.

[0133] The fourth step is to model global long-distance temporal dependencies based on Transformer.

[0134] This paper uses a Transformer encoder as the core feature extraction network. The backbone network consists of... It consists of stacked identical encoder layers, each containing two main sub-layers: a multi-head self-attention mechanism and a feedforward neural network. While traditional transformer attention mechanisms excel at capturing global dependencies, their... The computational complexity of convolutional networks limits their ability to model long sequences and easily overlooks high-frequency abrupt changes in local signals. While simple convolutional networks are sensitive to local features, they lack the ability to capture long-term characteristics. Therefore, this paper abandons the standard Transformer single attention mechanism and constructs two parallel attention strategies: global linear attention and local convolutional attention. The adaptive fusion of the two is achieved through a gating mechanism.

[0135] Before the features are fed into the Transformer encoder, learnable position embeddings are introduced to map the position indices into dense vectors with the same dimension as the features, injecting temporal sequence information into the model, thereby correctly capturing the temporal evolution pattern in subsequent attention calculations.

[0136] ,

[0137] in, The feature tensor output by the SE module. The learnable position embedding matrix is ​​a parameter matrix that is randomly initialized and continuously updated during backpropagation. The input tensor for the Transformer encoder after adding position encoding.

[0138] First, input features Mapped to different subspaces. Through three learnable linear projection matrices. Generate query, key, and value vectors.

[0139] ,

[0140] in, Input tensors to the encoder, This is the weight matrix. For querying the matrix, The key matrix, It is a value matrix.

[0141] This approach employs a linear attention mechanism to capture long-range dependencies between heat and cooling loads. Unlike the high computational cost of standard Softmax attention, this method utilizes kernel functions to reduce computational complexity. From reduced to The model prioritizes calculating the global context matrix. This matrix provides a high-level summary of the global correlation information for the entire time series. Secondly, the query vector... Information can be retrieved directly from this global matrix. The linear attention mechanism not only significantly reduces memory usage but also accurately extracts slow trends and periodic patterns hidden in long sequences.

[0142] ,

[0143] in, For global attention features, Let be a linear attention kernel function, taking... . This is the global context matrix, representing the global summary information of the entire sequence.

[0144] To capture abrupt changes in electrical load signals, a local convolutional attention strategy is employed. This strategy uses depthwise separable one-dimensional convolution with a small kernel size of Kernel=3 to capture high-frequency details and edge features in the sequence, ensuring that key inflection points and peak information in electrical load prediction are not diluted by global averaging effects.

[0145] ,

[0146] in, It is a feature of local attention. Depthwise separable convolution is a lightweight convolution operation that performs convolution on each channel independently, effectively extracting features from local adjacent locations, while having far fewer parameters than standard convolution. For Gaussian error linear units, nonlinearity is introduced.

[0147] For the two attention mechanisms mentioned above, the model designs an adaptive gating mechanism to achieve adaptive fusion of dual-stream features through learnable weight parameters. It uses a lightweight linear network to perceive the dynamic features of the input data in real time and calculates an adjustment coefficient between 0 and 1.

[0148] ,

[0149] in, Learnable weights and biases for adaptive gating mechanisms The gating coefficient, It is the Sigmoid activation function, which compresses the output value to between 0 and 1. Input tensors to the encoder.

[0150] When the model detects that the input signal exhibits significant long-term memory characteristics, the coefficient tends to 1, and the model primarily utilizes global linear attention to focus on extrapolating long-term evolution trends. When the model detects that the signal has strong transient change characteristics, the coefficient tends to 0, instructing the model to preferentially activate local convolutional attention. Ultimately, the model achieves real-time matching between the computational pattern and the physical characteristics of the load through the dynamic weighted fusion of these two channels.

[0151] ,

[0152] in, It is a multi-scale attention fusion feature. For Hadamard product, perform element-wise multiplication. The gating coefficient, It is a feature of local attention. This is a global attention feature.

[0153] The fused multi-scale attention features After Dropout processing, the residuals are connected to the original input. Add them together and then perform layer normalization. This step generates... This ensures the stability of feature propagation and alleviates the gradient vanishing problem.

[0154] ,

[0155] in, This is the output result of the multi-head self-attention step. This is the intermediate output tensor after sub-layer processing. Dropout is a random deactivation operation, a regularization technique to prevent overfitting. This is a layer normalization operation.

[0156] Feedforward neural networks employ a strategy of first increasing the dimensionality of the feature representation within each time slice, then activating it, and finally reducing the dimensionality, through fully connected layers and nonlinear activation functions, in order to enhance the nonlinear expressive power of the model.

[0157] ,

[0158] in, As the first fully connected layer of the upgraded dimension, It is the second dimension-reduced fully connected layer, and GELU is a Gaussian error linear unit. It is the output of the feedforward network.

[0159] The output of FFN is also processed by residual connections and layer normalization modules to ensure the training stability of deep networks, ultimately generating the output tensor of the encoder layer. .

[0160] ,

[0161] in, It is the output of the feedforward network. This is a layer normalization operation. This refers to the intermediate output tensor after sub-layer processing; Dropout is a random deactivation operation. It is the final output of the current encoder layer.

[0162] Transformer Encoder structure as follows: Figure 8 As shown.

[0163] Step 5: Channel merging and prediction output

[0164] Channel merging is the inverse operation of channel independence. Since the model effectively isolates noise interference between different load variables using a channel independence strategy in conjunction with SE-Block, and achieves decoupling and dynamic fusion of long-term inertial trends and instantaneous features, it is necessary to map the independent univariate sequences generated by the parallel prediction of Flatten Head back to the original multidimensional physical space through structured recombination. The channel merging module, through structured recombination, merges the discrete prediction components into a unified multivariate tensor, eliminating the structural isolation caused by channel independence. This allows the model to maintain its advantage in capturing the temporal features of single variables while outputting multidimensional prediction results corresponding to the original structure, thus supporting subsequent loss calculation and performance evaluation based on the joint distribution of multivariates.

[0165] ,

[0166] in, This is the final output of the Transformer encoder. Flatten the last two dimensions This is a linear projection operation. The obtained prediction results.

[0167] ,

[0168] in, The output prediction results for Flatten Head. For reshaping and displacement operations, the predicted length is... , This is the final multivariate prediction tensor for reconstruction.

[0169] Because robust normalization was used in the data preprocessing stage, the model predicted the following results. In fact, it's still a normalized value. To obtain the true load value, an inverse normalization operation must be performed.

[0170] ,

[0171] in, Model output under normalized scale. It is the median of the dataset. It is the first quartile. It is the third quartile.

[0172] Experimental verification:

[0173] This experimental dataset is strictly divided into training, validation, and test sets in chronological order to avoid future data leakage. The dataset is divided according to time nodes: the training set (January 1, 2020 to October 1, 2022) is used for model parameter learning and feature extraction; the validation set (October 1, 2022 to January 1, 2023) is used for hyperparameter fine-tuning and early stopping during training to prevent overfitting; and the test set (January 1, 2023 to December 30, 2023) is used solely for final performance testing.

[0174] The model outputs the predicted electricity, cooling, and heating loads as follows: Figure 9 , Figure 10 and Figure 11 As shown, the model exhibits excellent prediction accuracy, with an accuracy rate of 94.02% for electrical load prediction, 95.41% for heat load prediction, and 93.87% for cooling load prediction, with the overall error remaining at a low level.

[0175] Example 2

[0176] This embodiment provides a short-term energy load forecasting system based on an improved Transformer using SE-Block, including:

[0177] The data acquisition module is configured to acquire multivariate load sequence data;

[0178] The preprocessing module is configured to perform reversible normalization preprocessing based on the acquired multivariate load sequence data;

[0179] The feature fusion module is configured to extract and fuse features from preprocessed data using improved cross-scale interactive patching, including multi-scale feature extraction, cross-scale interactive alignment, residual correction and dynamic fusion.

[0180] The feature selection module is configured to perform feature selection on the fused features based on the channel attention mechanism, including feature response based on improved SE-Block and non-linear interaction of context vectors.

[0181] The global modeling module is configured to perform global long-distance temporal dependency modeling on the selected features based on Transformer.

[0182] The prediction module is configured to output multidimensional prediction results based on a channel merging strategy.

[0183] A computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the aforementioned short-term energy load forecasting method based on an SE-Block improved Transformer.

[0184] A terminal device includes a processor and a computer-readable storage medium, the processor being used to implement various instructions; the computer-readable storage medium being used to store multiple instructions adapted for loading and execution by the processor of the aforementioned short-term energy load forecasting method based on SE-Block improved Transformer.

[0185] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for short-term energy load forecasting based on an improved Transformer using SE-Block, characterized in that, include: Obtain multivariate load sequence data; Reversible normalization preprocessing is performed based on the acquired multivariate load sequence data; Improved cross-scale interactive patching is used to extract and fuse features from preprocessed data, including multi-scale feature extraction, cross-scale interactive alignment, residual correction and dynamic fusion. Feature selection is performed on the fused features based on the channel attention mechanism, including feature response based on improved SE-Block and non-linear interaction of context vectors; Global long-distance temporal dependency modeling is performed on the selected features based on Transformer; Output multidimensional prediction results based on channel merging strategy; The multi-scale feature extraction includes addressing the problem of static processing within a single window in the patching method, utilizing one-dimensional convolution operations on features of dimension [missing information]. The original load sequence is reconstructed in spatiotemporal dimensions. By setting convolutional kernels of different sizes and strides, high-frequency details, mid-frequency transitions, and low-frequency trend features are extracted, as shown below: , , ,in, For the input time series data, This is a one-dimensional convolution operation. To represent the size of the convolution kernel These are high-frequency, mid-frequency, and low-frequency feature tensors, respectively. To address the issue of inconsistent feature sequence lengths caused by multi-scale convolution, a linear interpolation algorithm is used to upsample the shorter feature sequences of the mid-frequency and low-frequency branches in the temporal domain, based on the feature length of the high-frequency branch. This ensures that the features of all frequency branches ultimately generate tensors with consistent dimensions. , is represented as: in, For the aligned feature tensor, For alignment operations, To set the alignment size, Finally, a cross-scale feature fusion operation is performed, employing a channel concatenation strategy to stack the feature tensors of the three independent branches along the feature channel dimension. A learnable linear projection layer is introduced, and through fully connected operations, the mixed features are mapped back to the latent space, thereby generating a comprehensive feature vector that takes into account both local details and global contours. The tensor dimension is restored to its original size. , is represented as: ,in, It is a learnable fusion projection matrix. For bias terms, It is the fused multi-scale feature tensor; The cross-scale interactive alignment includes addressing the misclassification of isolated high-frequency data as noise in time-series data and compensating for the phase drift problem of high-frequency details extracted from multi-scale features relative to the long-term trend. Based on an improved cross-attention mechanism, high-frequency features are used as query vectors, and low-frequency trend features are used as key-value vectors. When calculating detailed features, the overall trend characteristics are considered to compensate for the lack of global perspective in local features. This is expressed as follows: , , ,in, These are query, key, and value vectors, respectively. and These are the aligned low-frequency features and the aligned high-frequency features, respectively. These are learnable linear projection matrices used to map features to the attention space; then, the core operation of cross-attention is performed, utilizing the query vector. With key vector The transpose of the expression is used to perform a dot product operation to calculate the correlation score between high-frequency details and low-frequency trends, and then divided by the scaling factor. To prevent gradient vanishing, the gradient is then processed by the Softmax function to generate a normalized attention weight map, represented as follows: in, The attention score matrix, Scaling factor For normalization function, The attention output is used for the network. Finally, to preserve the integrity of the original high-frequency information and prevent network degradation, a residual connection structure is adopted, which directly adds the attention-enhanced features to the original high-frequency input features. The result of the addition is processed by layer normalization, ultimately generating cross-scale fused and aligned high-frequency features. , is represented as: in, It is the aligned high-frequency feature input. It is attention output. For layer normalization, The high-frequency features that are finally generated through cross-scale fusion alignment; The residual correction and dynamic fusion include first calculating the difference between the high-frequency features after cross-attention cross-scale fusion alignment and the aligned mid-frequency features, using the formula... The residual tensor is obtained. To determine the validity of the residual information, a learnable gating network is introduced. The residual is input into a linear layer and processed by a sigmoid activation function to generate gating coefficients. in, This represents the difference between the high-frequency features after cross-scale fusion and alignment and the aligned intermediate-frequency reference. For the learnable weights and biases of the gating network, For activation function, It is a numerical tensor; then the generated gating coefficients are used. Nonlinear correction is applied to the intermediate frequency characteristics, preserving the stability of the intermediate frequency reference characteristics while incorporating selected high-frequency details, as shown below: in, For element-wise multiplication, Corrected mid-frequency characteristics The gating coefficient, The aligned mid-frequency baseline features are then used; finally, global weighted fusion is performed, and global average pooling is applied to each of the three branches to obtain three global context vectors. , , in, For global feature vectors, For the aligned high-frequency feature tensor, This is the corrected mid-frequency feature tensor. For global average pooling, the three vectors are concatenated and passed through a multilayer perceptron (MLP) and a softmax function to calculate three dynamic weights. The final output is the weighted sum of the three branches: , in, It is a multilayer perceptron. To integrate weights, For normalization function, This is the final output tensor.

2. The energy short-term load forecasting method based on SE-Block improved Transformer according to claim 1, characterized in that, The reversible normalization preprocessing based on the acquired multivariate load sequence data includes robust normalization based on quantiles for peak segments of cooling load data during extreme high-temperature periods. ,in, These are the normalized values. It is a specific value in the original input sequence. It is the median of the dataset. It is the first quartile. It is the third or fourth quartile. Then, a sliding window is used to slice the original long sequence data. The window length and prediction duration are set, and the window slides on the time axis with a fixed step size. By truncating continuous sequences as model input, the next adjacent time step is used as the corresponding true prediction label. The input data is then reshaped into a standard three-dimensional tensor format [B, L, V]. To solve the problem of data non-stationarity, the input electrical, cooling, and heating load data is reversibly normalized before channel-independent operation. To enable the Transformer to focus more on capturing temporal features, tensor reshaping is performed based on a channel-independent strategy, represented as: Wherein, the original input tensor is ,in For batch size, For window length, For the number of variables, For channel-independent tensors, Permute & Reshape are the permutation and reshaping operations of tensor dimensions; finally, batch samples are converted into batch*variable samples to increase the number of training samples and avoid the model learning incorrect coupling relationships.

3. The energy short-term load forecasting method based on SE-Block improved Transformer according to claim 2, characterized in that, The improved SE-Block-based feature response includes modifying the Squeeze stage of the traditional SE module based on the frequency domain characteristics of the load variable, using a frequency domain adaptive convolution strategy for view reconstruction and feature separation, reconstructing the input feature tensor from a single batch dimension into an independent dimension containing both batches and variables, and processing the input data. To reshape the dimensions and organize them into The standard form in, These are the original input features. These are the reconstructed input features; Reshape is the dimension reshaping operation. Batch Size For the number of variables, The number of patches. It's the feature dimension; then the first-way Squeeze operation is performed, in the time dimension. Global average pooling is performed on the input features. , in, This is the global statistical baseline feature vector. This is a global pooling operation. These are the reconstructed input features. This indicates that operations are performed on all batches, all variables, the i-th time step, and all feature dimensions; in parallel, the second-path frequency domain feature-aware operation is executed using a one-dimensional convolutional layer. Local filtering of input features, convolution kernel Capture local dependencies and lag effects in time series data to generate lag feature tensors. , is represented as: in, For the hysteresis characteristic tensor, It is a one-dimensional convolution operation. For convolution kernel, This is a slice of the v-th variable in the input tensor; finally, global average pooling is performed on the hysteresis features to obtain the frequency domain characteristic context vector, represented as: , ,in, This is a local dynamic context feature vector. This is a global pooling operation. For the hysteresis characteristic tensor, This indicates that the operation is performed on all batches, the i-th time step, and all feature dimensions.

4. The energy short-term load forecasting method based on SE-Block improved Transformer according to claim 3, characterized in that, The nonlinear interaction of the context vectors includes performing a nonlinear interaction between the global context vector and the physical context vector to combine global trends and local physical characteristics, using physical hysteresis features as a gain coefficient to adaptively enhance or suppress global trend features, resulting in a fused vector. , is represented as: in, Local dynamic context-oriented feature vectors, Features after fusion This is the global statistical baseline feature vector. The Hadamard product is used, with 1 as the unit baseline. Based on the enhanced feature vector obtained from the pre-fusion operation, Excitation and Scale operations are used to further capture the non-linear dependencies between channels and complete the final feature selection. In the Excitation stage, the enhanced features are fed into a matrix consisting of two fully connected layers. The bottleneck structure is formed by dimensionality reduction layers and activation functions. Nonlinearity is introduced and features are compressed to capture the dependencies between channels. Then, the dimensionality is restored through an upscaling layer, and the sigmoid activation function is utilized. The output is mapped to the interval between 0 and 1, ultimately generating the channel attention weight vector. , is represented as: in, These are the weight matrices for dimensionality reduction and dimensionality increase, respectively. These are the ReLU activation function and the sigmoid activation function, respectively. This is the feature vector fused from the previous step. Here are the channel attention weights; during the scaling phase, element-wise multiplication is performed, and the generated channel attention weights are applied back to the original input feature vector, as follows: in, For channel attention weights, For Hadama accumulation, This is the raw data before entering SE-Block. This is the final output feature.

5. The energy short-term load forecasting method based on SE-Block improved Transformer according to claim 4, characterized in that, The method of performing global long-distance temporal dependency modeling on the selected features based on Transformer includes using a Transformer encoder as the core feature extraction network, with the backbone network consisting of... The model consists of stacked identical encoder layers, each containing two main sub-layers: a multi-head self-attention mechanism and a feedforward neural network. It abandons the single attention mechanism of the Transformer and constructs two parallel attention strategies: global linear attention and local convolutional attention. An adaptive fusion of these two strategies is achieved through a gating mechanism. Before features are fed into the Transformer encoder, learnable position embeddings are introduced, mapping position indices to dense vectors with the same dimension as the features. This injects temporal sequence information into the model, enabling it to correctly capture temporal evolution patterns in subsequent attention calculations. This can be represented as: ,in, The feature tensor output by the SE module. For learnable position embedding matrix, The input tensor of the Transformer encoder after adding position encoding, and then the input features Mapped to different subspaces, through a learnable linear projection matrix. The process generates query, key, and value vectors. Finally, a linear attention mechanism is used to capture long-range dependencies between hot and cold loads. Unlike the high computational cost of standard Softmax attention, this approach utilizes kernel functions to reduce computational complexity. From reduced to And prioritize the calculation of the global context matrix. Secondly, query vector Information is retrieved directly from this global matrix, and then a linear attention mechanism is used to accurately extract the slow changing trends and periodic patterns hidden in the long sequence, represented as: in, For global attention features, Let be a linear attention kernel function, taking... , This is the global context matrix.

6. The energy short-term load forecasting method based on SE-Block improved Transformer according to claim 5, characterized in that, The Transformer-based global long-range temporal dependency modeling of the selected features also includes a local convolutional attention strategy to capture abrupt changes in electrical load signals. This strategy uses depthwise separable one-dimensional convolutions with small-sized kernels to capture high-frequency details and edge features in the sequence, ensuring that key inflection points and peak information in electrical load prediction are not diluted by global averaging effects. This is expressed as follows: in, It is a feature of local attention. It is a depthwise separable convolution. The Gaussian error linear unit is used; then an adaptive gating mechanism is constructed to achieve adaptive fusion of dual-stream features through learnable weight parameters. A lightweight linear network is used to perceive the dynamic features of the input data in real time and calculate an adjustment coefficient between 0 and 1, expressed as: in, Learnable weights and biases for adaptive gating mechanisms The gating coefficient, It's the sigmoid activation function, which compresses the output value to between 0 and 1. The encoder is input with a tensor; then, through dynamic weighted fusion of these two channels, real-time matching of the calculation mode and the physical characteristics of the load is achieved, as shown below: ,in, It is a multi-scale attention fusion feature. For the Hadamard product, perform element-wise multiplication. The gating coefficient, It is a feature of local attention. The global attention features are used as the basis for the final fusion of multi-scale attention features. After Dropout processing, the residuals are connected to the original input. Add them together and perform layer normalization to generate... To ensure the stability of feature propagation and alleviate the gradient vanishing problem, it is represented as: in, This is the output result of the multi-head self-attention step. This is the intermediate output tensor after sub-layer processing; Dropout is the random deactivation operation. This is a layer normalization operation; furthermore, the feedforward neural network, through fully connected layers and nonlinear activation functions, employs a strategy of first increasing the dimensionality, then activating, and finally reducing the dimensionality of the feature representation within each time slice to enhance the model's nonlinear expressive power, expressed as: in, As the first fully connected layer of the upgraded dimension, This is the second dimension-reducing fully connected layer; GELU stands for Gaussian error linear unit. This is the output of the feedforward network. The output of the FFN is processed by the residual connection and layer normalization module to ensure the training stability of the deep network, and finally generates the output tensor of the encoder layer. , is represented as: in, It is the output of the feedforward network. For layer normalization operation, This is the intermediate output tensor after sub-layer processing; Dropout is the random deactivation operation. It is the final output of the current encoder layer.

7. A short-term energy load forecasting system based on SE-Block improved Transformer, executing the short-term energy load forecasting method based on SE-Block improved Transformer as described in claim 1, characterized in that, include: The data acquisition module is configured to acquire multivariate load sequence data; The preprocessing module is configured to perform reversible normalization preprocessing based on the acquired multivariate load sequence data; The feature fusion module is configured to extract and fuse features from preprocessed data using improved cross-scale interactive patching, including multi-scale feature extraction, cross-scale interactive alignment, residual correction and dynamic fusion. The feature selection module is configured to perform feature selection on the fused features based on the channel attention mechanism, including feature response based on improved SE-Block and non-linear interaction of context vectors. The global modeling module is configured to perform global long-distance temporal dependency modeling on the selected features based on Transformer. The prediction module is configured to output multidimensional prediction results based on a channel merging strategy.

Images