Power load prediction method and system based on dual attention mechanism WPmean

Abstract

The invention provides a power load prediction method and system based on a dual attention mechanism WPmean, and relates to the technical field of power load prediction. The preprocessed power load time sequence data are input into a trained dual attention mechanism WPMixer model, wavelet decomposition is carried out on the input data, and branch features comprising a low-frequency branch and a plurality of high-frequency branches are obtained; carrying out Patch processing on each branch feature, inputting the processed branch feature into an improved Mixer module group, and carrying out slicing dimension and embedding dimension processing based on an attention mechanism to obtain a deep feature; based on a cross-resolution attention interaction module, taking low-frequency branch features in the deep features as shared context information, and performing cross attention calculation on the low-frequency branch features and the high-frequency branch features to obtain modulation features; and performing multi-resolution wavelet reconstruction based on the modulation characteristics to obtain a power load prediction value. And on the premise of not obviously increasing the calculation overhead, more accurate and more robust power load prediction is realized.

Description

Technical Field

[0001] This invention relates to the field of power load forecasting technology, and in particular to a power load forecasting method and system based on the dual attention mechanism WPMixer. Background Technology

[0002] Electricity load forecasting is a core support for ensuring the safe and economical operation of the power system, and is crucial for power generation planning, grid dispatching, and supply-demand balance. With the advancement of new power system construction, load data exhibits strong nonlinearity, non-stationarity, and multi-timescale characteristics, making traditional forecasting methods insufficient to meet the requirements for accuracy and robustness. In recent years, Transformer variants and MLP-like models have become research hotspots. Among them, the WPMixer model, through multi-level wavelet decomposition and an efficient MLP architecture, demonstrates significant computational efficiency advantages in long-sequence forecasting, becoming an important choice for efficient forecasting solutions.

[0003] However, in complex power load scenarios, the performance bottlenecks of existing models are becoming increasingly apparent. While WPMixer achieves multi-resolution feature extraction, each resolution branch is processed independently, lacking an effective interaction mechanism and failing to capture the potential synergistic relationship between trend and fluctuation components, resulting in insufficient information utilization. Furthermore, its core Mixer module uses static linear transformations and fixed weights after training, making it unable to adaptively adjust to dynamic load changes caused by factors such as holidays and extreme weather, thus limiting the improvement of prediction accuracy. In addition, while Transformer-type models have high accuracy, their self-attention mechanisms are computationally complex, making it difficult to meet real-time requirements. Traditional MLP-type models, on the other hand, suffer from insufficient dynamic feature capture capabilities; neither can balance computational efficiency with prediction accuracy. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a power load forecasting method and system based on the dual attention mechanism WPMixer, aiming to achieve more accurate and robust power load forecasting without significantly increasing computational overhead.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a power load forecasting method based on the dual attention mechanism WPMixer, comprising: Acquire time-series power load data and perform preprocessing; The preprocessed power load time series data is input into the trained dual attention mechanism WPMixer model to predict power load. The process involves wavelet decomposition of the input data to obtain multiple resolution branch features, including one low-frequency branch and multiple high-frequency branches. For each branch feature, patching is performed before inputting into an improved Mixer module group. Based on the attention mechanism, slice dimension and embedding dimension processing are performed to obtain deep features. Based on the cross-resolution attention interaction module, the low-frequency branch features in the deep features are used as shared context information and cross-attention calculation is performed with each high-frequency branch feature to obtain modulation features. Based on the modulation features, multi-resolution wavelet reconstruction is performed to obtain the predicted power load value.

[0006] Secondly, the present invention provides a power load forecasting system based on the dual attention mechanism WPMixer, comprising: The data processing module is configured to acquire power load time-series data and perform preprocessing. The load forecasting module is configured to input the preprocessed power load time series data into the trained dual attention mechanism WPMixer model to perform power load forecasting. The process involves wavelet decomposition of the input data to obtain multiple resolution branch features, including one low-frequency branch and multiple high-frequency branches. For each branch feature, patching is performed before inputting into an improved Mixer module group. Based on the attention mechanism, slice dimension and embedding dimension processing are performed to obtain deep features. Based on the cross-resolution attention interaction module, the low-frequency branch features in the deep features are used as shared context information and cross-attention calculation is performed with each high-frequency branch feature to obtain modulation features. Based on the modulation features, multi-resolution wavelet reconstruction is performed to obtain the predicted power load value.

[0007] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the power load forecasting method based on the dual attention mechanism WPMixer described in the first aspect.

[0008] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the power load forecasting method based on the dual attention mechanism WPMixer described in the first aspect.

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention accurately splits the trend and fluctuation characteristics of load data through wavelet decomposition, and combines the dual attention mechanism of the improved Mixer module to dynamically capture the key dependencies of slice and embedding dimensions, breaking through the limitations of traditional static mixing. At the same time, cross-resolution attention interaction enables low-frequency trends and high-frequency fluctuations to work together efficiently, fully explore the correlation of multi-scale information, and improve the model's adaptability to dynamic scenarios such as holidays and extreme weather. In addition, it inherits the high efficiency characteristics of the MLP architecture, avoids high computational overhead, and improves prediction accuracy and robustness while ensuring the real-time performance of short-term and ultra-short-term predictions, providing reliable support for the safe and economical operation of the power system.

[0010] (2) This invention uses a lightweight attention layer to replace the static linear mixer in the original WPMixer. This design enables the model to dynamically adjust the information fusion weights between different variable features (channel attention) or between different time segments (time series attention) according to the specific content of the input load sequence. Compared with the fixed linear transformation of the original model, this dynamic mixing mechanism has a stronger adaptability to complex nonlinear changes in power load caused by factors such as holidays, promotional activities or extreme weather, and can more accurately characterize the impact of key features at specific moments.

[0011] (3) This invention effectively solves the problem of insufficient information coordination caused by the isolated signal processing of each resolution branch in the existing WPMixer model by introducing a cross-resolution attention interaction module. This module can dynamically capture the intrinsic correlation between low-frequency trend components and high-frequency fluctuation components, rather than simply processing them independently. In power load forecasting, this enables the model to more accurately coordinate the relationship between long-term power consumption trends and short-term random fluctuations (such as sudden weather changes and emergencies), thereby improving the fitting ability and prediction accuracy of complex load sequences.

[0012] (4) This invention achieves an excellent balance between model efficiency and performance by integrating a dual attention mechanism on the MLP architecture. On the one hand, the model inherits the efficient computational characteristics of WPMixer based on MLP and modular design, avoiding the huge computational overhead of complex Transformer architecture. On the other hand, through targeted attention enhancement, the model overcomes the limitations of the original MLP architecture in capturing long-term dependencies and complex dynamic features. This makes this invention particularly suitable for short-term and ultra-short-term power load forecasting scenarios with high real-time requirements, providing more reliable forecast results while ensuring operational efficiency.

[0013] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0014] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0015] Figure 1 A main flowchart of a power load forecasting method based on the dual attention mechanism WPMixer provided in this embodiment of the invention; Figure 2 A flowchart illustrating a power load forecasting method based on the dual attention mechanism WPMixer, provided in an embodiment of the present invention; Figure 3 A schematic diagram of the improved Mixer module group provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of cross-resolution attention interaction provided in an embodiment of the present invention; Figure 5 This is a graph illustrating the predicted power load for the next 48 hours, provided as an embodiment of the present invention. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Existing load forecasting technologies have evolved over many years, from traditional methods based on mathematical statistics (such as time series analysis and regression analysis) to advanced methods based on artificial intelligence (such as neural networks and deep learning models). In recent years, the Transformer model and its variants (such as Informer and Crossformer) have received widespread attention in the field of load forecasting due to their powerful ability to capture long-range dependencies.

[0018] However, the core self-attention mechanism of such models has high computational complexity, posing a significant challenge to computational efficiency when processing long sequences. In response, models based on multilayer perceptrons (MLPs) have shown potential as an efficient alternative. Among them, Murad et al. proposed the WPMixer (Wavelet Patch Mixer) model in their paper "WPMixer: Efficient Multi-Resolution Mixing for Long-TermTime Series Forecasting," a newly proposed MLP-like model specifically designed for long-term time series forecasting.

[0019] The core technical solution of this model is as follows: First, multi-level wavelet decomposition is used to decompose the original load sequence into approximate coefficient sequences (representing trends) and detailed coefficient sequences (representing fluctuations) at multiple resolutions, thereby explicitly extracting signal features in the time-frequency domain. Second, each decomposed coefficient sequence is processed by an independent resolution branch. This branch divides the sequence into segments using patching technology, and then mixes the information through Patch Mixer and Embedding Mixer (both MLP structures) to capture local correlation and global dependency, respectively. Finally, the prediction outputs of each branch are reconstructed using wavelet to obtain the final load prediction value. WPMixer demonstrates a significant computational efficiency advantage when processing long sequences by avoiding complex attention calculations. However, while pursuing efficiency, this model also has limitations that cannot be ignored.

[0020] While the WPMixer model offers advantages in efficiency and multi-resolution analysis, its core Mixer module is essentially a static linear transformation. Its mixing weights are fixed after model training and cannot be dynamically adjusted based on the specific content of the input sequence. This static characteristic makes the model ill-suited to the complex dynamic changes in power load caused by holidays, unexpected events, etc., resulting in insufficient adaptability and limiting further improvements in prediction accuracy. Furthermore, the independent processing of each resolution branch within WPMixer, lacking necessary interaction mechanisms, ignores the potential synergistic relationships between signals at different scales, such as trends and details, causing the model to fail to fully utilize the information from the decomposed signals. Therefore, this invention proposes a power load forecasting method, system, medium, and device based on the dual attention mechanism WPMixer. While inheriting the advantages of WPMixer's multi-resolution analysis and efficient MLP architecture, it improves upon the shortcomings of its hybrid mechanism in terms of staticity and inter-branch isolation, so as to improve the ability to capture the complex dynamic characteristics of power load and the forecasting accuracy while maintaining high efficiency.

[0021] Example 1 like Figure 1 As shown, this embodiment discloses a power load forecasting method based on the dual attention mechanism WPMixer, including the following steps: S1: Acquire power load time-series data and perform preprocessing; S2: Input the preprocessed power load time series data into the trained dual attention mechanism WPMixer model to predict power load; The process involves wavelet decomposition of the input data to obtain multiple resolution branch features, including one low-frequency branch and multiple high-frequency branches. For each branch feature, patching is performed before inputting into an improved Mixer module group. Based on the attention mechanism, slice dimension and embedding dimension processing are performed to obtain deep features. Based on the cross-resolution attention interaction module, the low-frequency branch features in the deep features are used as shared context information and cross-attention calculation is performed with each high-frequency branch feature to obtain modulation features. Based on the modulation features, multi-resolution wavelet reconstruction is performed to obtain the predicted power load value.

[0022] Next, combined Figure 2 This embodiment provides a detailed description of a power load forecasting method based on the dual attention mechanism WPMixer.

[0023] (I) Data preparation and sample division 1. Dataset Preparation Historical electricity load data is acquired as the foundational dataset. Depending on the specific needs of the forecasting task, external variable data associated with load changes (e.g., historical weather data, date type, etc.) can also be acquired simultaneously. This provides raw data support for model training and forecasting.

[0024] To conduct supervised model training and evaluation, the historical dataset needs to be divided into training, validation, and test sets according to a predetermined ratio (e.g., 0.7:0.2:0.1). This division process must strictly adhere to the chronological order to prevent future information leakage and ensure the reliability of the model evaluation results.

[0025] As one implementation method, historical electricity load data from a region in northern China, after being anonymized by dimensions, was acquired as a sample dataset. Simultaneously, external variable data related to load changes were also obtained. Specifically, the data spanned nearly 14 months, from May 1, 2024 to June 23, 2025, with 15-minute intervals, totaling 96 data points per day.

[0026] The selected power load forecasting operational model has 14 measurement points (including the date column). In addition to the target measurement point load, there are 9 time-related characteristic measurement points and 4 meteorological data measurement points. The specific details of the measurement points are shown in Table 1 below: Table 1. Measurement Points for the Power Load Forecasting Business Model;

[0027] Obtain the complete historical dataset H according to the above requirements. The dataset size is 40224 rows and 14 columns. The historical dataset H is divided into training set TR ([28158,14]), validation set V ([8044,14]), and test set TS ([4022,14]) by default with a ratio of 0.8:0.1:0.1. All measurement data of the training set TR are standardized according to formula (1) to a distribution with a mean of 0 and a variance of 1. (1) in, It is the mean of each measuring point. It is the standard deviation of each measuring point.

[0028] 2. Sample Division To adapt to deep learning-based prediction models, training data needs to be transformed into sample units that the model can process.

[0029] Specifically, a fixed-length sliding window is used to divide the training data into a series of continuous multivariate time-series sample slices. Each sample slice contains data from a historical window (model input data) and data from a corresponding future prediction window (target prediction data). This transforms the continuous prediction problem into a supervised learning problem, providing the model with standardized input-output pairs.

[0030] As one implementation method, in order to process the samples into a form acceptable to deep learning-based prediction models, this embodiment uses a sliding window of fixed length L along the time axis to process the training dataset. By sequentially extracting segments with a step size S, a series of continuous multivariate time series slices are generated. Here, parameter L is set to 384 (duration is 4 days), and parameter S is set to 1 (slice movement step size is 15 minutes).

[0031] Will Each slice of sample data in the set is further divided into two equal parts: 1) Model input data The preceding 192 consecutive multivariate sequences are represented as follows: ,in ; 2) Target prediction data Immediately following the model input data The subsequent 192 consecutive multivariate sequences are represented as follows: ,in .

[0032] This step transforms the continuous prediction problem into a supervised learning problem, providing the model with standardized input-output pairs.

[0033] (ii) Multi-resolution wavelet decomposition Multivariate time series sample slices are input into the dual attention mechanism WPMixer model.

[0034] First, the sample slices are preprocessed, performing row normalization and row / column transpose sequentially. Then, the core operation is performed: Discrete Wavelet Transform (DWT) is applied to each measurement point of the sample slice. DWT is essentially a multi-stage filtering and downsampling process using low-pass and high-pass filter banks.

[0035] Specifically, the decomposition process involves DWT decomposing each input measurement point sequence level by level. Each level of decomposition generates two sets of coefficients: 1. Low-frequency approximation coefficient sequence: generated by a low-pass filter, capturing the main contours and long-term trends of the signal. After decomposition, the length of this sequence is halved (downsampling).

[0036] 2. High-frequency detail coefficient sequence: generated by a high-pass filter, capturing local details and short-term fluctuations of the signal at this decomposition scale. It is also downsampled.

[0037] The low-frequency coefficient sequence obtained from the first-level decomposition can be further decomposed into a second-level decomposition. This process can be repeated until, after m levels of decomposition, one low-frequency approximate coefficient sequence (from the m-th level decomposition) and m high-frequency detail coefficient sequences (from the first to the m-th level decompositions) are obtained. After all the measurement points of the multivariate sample slice undergo discrete wavelet transform according to the above process, m+1 multivariate sequence samples are obtained. These m+1 multivariate sequences together constitute the complete representation of the original sample slice from a multi-resolution perspective.

[0038] As one implementation method, a multivariate time series sample slice set is used. As input, the following is... centralized Taking the DB4 wavelet basis function as an example, the three-level decomposition process is illustrated. Here, L is the sequence length, and C is the number of measurement points (load and meteorological related measurement points).

[0039] Sample slices Preprocessing is performed, and column normalization is completed sequentially. Row and column transpose Two processes.

[0040] Then, the parameters of its discrete wavelet decomposition function are configured as follows: 1. Wavelet basis selection: The DB wavelet is selected as the mother wavelet in this embodiment. The DB4 wavelet has tight support and a certain degree of smoothness, making it very suitable for analyzing non-stationary signals such as power load.

[0041] 2. Decomposition levels: Perform 3-level (m=3) discrete wavelet decomposition to separate high-frequency and low-frequency signals to a limited extent.

[0042] Next, single-point wavelet decomposition is performed. For the sample slice... Each single measurement point sequence in Perform the following operations: Level 1 decomposition: [The sequence is broken down into smaller parts.] Using the low-pass filter corresponding to the DB4 wavelet and high-pass filter Perform convolution and downsampling (with a stride of 2), and obtain the first-order approximation coefficients (low frequency) according to the following formulas (2) and (3). And detail factor (high frequency) .

[0043] (2) (3) Where n represents the time point of the decomposed sequence; : Represents the original input signal of the c-th variable; and : These are the coefficients of the low-pass and high-pass filters corresponding to the selected wavelet basis (DB4), respectively; The low-frequency approximation coefficients obtained from the first-level decomposition characterize the long-term trend of the signal. The high-frequency detail coefficients obtained from the first-level decomposition characterize the short-term fluctuations of the signal. : indicates a convolution summation operation, where k is the index of the filter coefficient.

[0044] effect: It captured the overall profile and trends in the sequence at L / 2 resolution; This captures the most subtle fluctuations and noise at that resolution.

[0045] Level 2 decomposition: The low-frequency coefficients obtained from Level 1 decomposition... As input, repeat the filtering and downsampling process of formulas (2) and (3) above to obtain the second-level coefficients. and .

[0046] effect: It characterizes the medium- to long-term trends at a coarser scale (L / 4 resolution); It characterizes the medium-frequency fluctuations at this resolution.

[0047] Level 3 decomposition: This will be further decomposed to obtain the third-level coefficients. and

[0048] effect: It represents the long-term trends and basic forms at the coarsest scale (L / 8 resolution); It characterizes the low-frequency fluctuation components at this scale.

[0049] After three levels of decomposition, the original sequence is output. Decomposed into Four wavelet coefficient components. This process is repeated for all variables, ultimately yielding four multivariate sequence matrices. These correspond to wavelet coefficient features related to load at different resolutions. Through this three-level decomposition, the original multivariate sequence... They are adaptively decoupled into sub-signals of different physical meanings and lengths.

[0050] In this embodiment, the discrete wavelet transform serves as a key data processing step, decoupling the non-stationary, nonlinear original multivariate sequence into sub-signals (trends and fluctuations at various levels) with different physical meanings in the time-frequency domain. This significantly reduces the complexity of subsequent modeling and provides reliable input for subsequent multi-resolution analysis.

[0051] (III) First-level attention mechanism The resulting 1 low-frequency branch and m high-frequency branches, totaling m+1 branches, are input in parallel into their respective independent branch feature processing flows, such as... Figure 3 As shown.

[0052] In one implementation, the decomposed low-frequency branch and three high-frequency branches, totaling four branches, are input in parallel into their respective independent branch feature processing flows.

[0053] Each branch's feature processing flow includes the following sub-steps: 1. Data processing and patching The input branch sequence (such as a low-frequency coefficient sequence) is normalized and transposed twice, and then divided into multiple non-overlapping patches. Each patch is mapped to a high-dimensional embedding vector through a linear embedding layer. Its purpose is to normalize and structure the data, preparing it for deep feature extraction.

[0054] Specifically, the sequence (characterizing the long-term trend) is obtained from any resolution branch through wavelet decomposition. For example, firstly, for this multivariate sequence A dimension transpose is performed to facilitate subsequent layer normalization. After layer normalization, a second dimension transpose is performed to restore the original dimension arrangement, resulting in the final product. The purpose of this step is to normalize the data and prepare it for subsequent patching.

[0055] Transposed sequence The time dimension is divided into P non-overlapping patches. Each patch contains k consecutive time points, resulting in... (in Then, a linear projection layer (embedding layer) is used to map each patch to a D-dimensional embedding vector, thus obtaining the embedding representation of that branch. This step transforms the sequence into a series of patches, enhancing its feature representation capabilities.

[0056] 2. Improved Mixer module processing: such as Figure 3 As shown, the sequence representation after patching and embedding will be passed through two concatenated improved Mixer modules in sequence.

[0057] Each Mixer module consists of two sub-modules: a slice dimension processing sub-module and an embedding dimension processing sub-module.

[0058] (1) Slice Dimension Processing Submodule: Responsible for Pactch dimension mixing. In this embodiment, a lightweight attention layer is added in front of the first static linear layer in the original WPMixer.

[0059] This lightweight attention layer (Patch Attention) dynamically calculates the relevance weights between different patches based on the input data, rather than performing a fixed linear transformation. Its function is to achieve adaptive and self-aware feature mixing, enabling the model to dynamically capture complex and changing dependencies in load-related data, significantly improving the model's expressive power and adaptability. The attention results are then further processed through two linear layers and the GELU activation function to fuse and extract features from the patch data again, achieving the capture of local information features.

[0060] Specifically, the embedding representation matrix of the input Perform layer normalization, and then adjust the dimensions by placing the embedding dimension first, to obtain... .

[0061] Input Projected onto lower dimensions through three independent linear mapping layers. ( (to achieve lightweighting), to obtain the query ,key Sum All three dimensions are .

[0062] Next, the scaling dot product attention calculation formula is: (4) This operation dynamically calculates the relevance weights between different time segments (patches) based on the content, rather than performing a fixed linear transformation. This enables adaptive and self-aware mixing of information between patches, allowing the model to dynamically capture complex temporal dependencies in the data sequence.

[0063] Finally, attention output Projected back through a linear layer Dimension, then sequentially through two fully connected layers and The activation function performs nonlinear transformations and feature extraction, ultimately yielding the feature matrix. .

[0064] (2) Embedded Dimension Processing Submodule: This module is responsible for embedding dimension mixing. Its structure and processing flow are similar to those of the sliced ​​dimension processing submodule, but its core operation is used to embed feature dimensions. It also uses a lightweight attention layer (such as a channel attention layer) to replace the static linear layer, dynamically fusing information between different feature channels to achieve global feature fusion and extraction.

[0065] Specifically, first, the output of the Patch dimension mixing function module... By performing layer normalization and dimension adjustment, we obtain .

[0066] A lightweight attention layer is applied to fuse embedded dimensional information. Similarly, first... Obtained through linear layer projection Shape and size are ,in .

[0067] Calculate the attention along the embedding dimension using the following formula: (5) in, By dynamically fusing information from different feature channels, the model can adaptively learn the importance of variable features.

[0068] Then, the dimensions are reduced through linear projection. Return After passing through two fully connected layers and the GELU activation function, the output result is... Then, a residual join is performed with the input, i.e., the output of the slice dimension processing submodule, to obtain the result. .

[0069] Through the two improvements to the Mixer module mentioned above, efficient and adaptive deep feature extraction is achieved within the four high- and low-resolution branches, resulting in the final feature map. ,in , , , L is 192, k is the patch length, which defaults to 4.

[0070] The output of the second Mixer module is residually concatenated with its input, which is the output of the first Mixer module, to obtain deep features. The result is then fed into the next module or layer. The purpose of the residual connection is to promote the flow of gradients in deep networks, alleviate the gradient vanishing problem, and thus ensure stable training of the model.

[0071] In this embodiment, internal residual connections enable efficient fusion of information from slice dimension processing and embedding dimension processing, alleviating the gradient vanishing problem and helping the model learn more complex features. Inter-module residual connections allow the outputs of the first Mixer module and the second Mixer module to complement each other, preserving previous features while promoting deep feature propagation, improving the model's ability to model sequential data such as power load, enhancing the model's stability and performance, and making predictions more accurate.

[0072] Compared to the static linear mixing of the original WPMixer, the lightweight attention mechanism introduced in this embodiment effectively enhances the ability to capture dynamic change patterns in non-stationary power load signals, providing high-quality feature representations for subsequent cross-resolution interactions.

[0073] (iv) The second attention mechanism After each branch completes feature extraction independently, the data is input into the cross-resolution attention interaction module. See the detailed structure of this module below. Figure 4 Its purpose is to break the limitation of isolated processing of each branch of the original WPMixer model and realize dynamic and adaptive fusion of information at multiple scales.

[0074] 1. Preparation of Interaction Context and Query This step prepares the input for the interactive process. The processed output feature map of the low-frequency branch (representing the long-term trend) is used as shared context information. This feature map is passed through a linear projection layer to generate the keys (K) and values ​​(V) required for the attention mechanism. Simultaneously, the processed output feature map of each high-frequency branch (representing short-term fluctuations at different scales) is used as an independent query source. The feature map of each high-frequency branch is passed through its dedicated linear projection layer to generate the corresponding query (…). , where i = 1, 2, ..., m).

[0075] Specifically, it receives feature maps from four branches. .include: Low-frequency branches (context K, V): Feature maps representing long-term trends , size and shape .

[0076] High-frequency branches (query source Q): Three feature maps representing short-term fluctuations at different scales, denoted as follows: , , Their shapes and sizes are respectively .

[0077] Generate Key (K) and Value (V): This involves generating the low-frequency branch feature map. As a shared global context, it is passed through a linear projection layer to generate the keys (K) and values ​​(V) in the attention mechanism: (6) (7) in, , This refers to the projected dimension. Its function is to encode trend information into a queryable "information database".

[0078] Generate query ( ): Feature map of each high-frequency branch , , Each branch serves as an independent query source. Each branch generates a corresponding query through its dedicated linear projection layer. ): (8) Each of them This ensures that it matches the dimensions of K and V. Its purpose is to preserve the uniqueness of each high-frequency detail, preparing for subsequent personalized attention interactions.

[0079] 2. Parallel Cross-Attention Calculation Interaction logic executed: This embodiment designs multiple parallel cross-attention layers. Each high-frequency branch corresponds to an independent cross-attention calculation unit. For the i-th high-frequency branch, the calculation process is as follows: Queries using this branch The shared key K and value V generated from the low-frequency branch features are then subjected to standard scaled dot product attention computation. This computation outputs a modulated new feature representation that incorporates global trend information.

[0080] Specifically, for the i-th high-frequency branch, the calculation process is as follows: (9) in, and The matrix multiplication represents the correlation score between the i-th high-frequency detail and the global trend information. The function converts relevance scores into weights, and then combines them with trend information. Weighted summation.

[0081] The output of this operation It is a new feature representation that has been modulated and incorporates global trend information. For example, the fluctuation of a detail such as "peak electricity consumption at noon" can be adjusted in magnitude according to the trend of "overall increase in electricity consumption in summer".

[0082] 3. Feature Modulation and Output After cross-attention computation, each high-frequency branch acquires enhanced features associated with its own detailed information and modulated by the global trend. These enhanced features are then passed through a linear mapping layer to adjust their dimensions, obtaining a prediction step size dimension that matches the requirements of subsequent reconstruction steps. Simultaneously, the feature maps of low-frequency branches undergo the corresponding linear mapping.

[0083] Specifically, each enhanced high-frequency feature obtained after cross-attention calculation... Its dimensions are adjusted through a dedicated linear mapping layer. Dimension ,in Then, the reshape function is used to adjust the dimension to match the final prediction step size L. Simultaneously, the features of the low-frequency branch, Flow, are also linearly mapped accordingly, with the output dimension also being [missing value]. .

[0084] in, .

[0085] All branches yielded four branch prediction results that had undergone cross-resolution interactive modulation. These were then fed into the next step for wavelet reconstruction.

[0086] In this embodiment, the cross-resolution attention interaction module breaks the isolation between the resolution branches of the original WPMixer model, dynamically establishing the modulation relationship between low-frequency trend information and various high-frequency detail information. This allows for adjustments to the prediction of local fluctuation amplitudes based on the overall target data trend, achieving synergy and fusion between multi-scale information and fully utilizing all information from the decomposed signal.

[0087] (V) Multi-resolution wavelet reconstruction and result output The enhanced high-frequency branch prediction coefficient features are then combined with the low-frequency branch prediction coefficient features for multi-resolution wavelet reconstruction. This reconstruction process is the inverse of wavelet decomposition, and its function is to resynthesize the processed resolution coefficients into the final prediction sequence. Subsequently, the reconstruction results are subjected to data processing such as inverse normalization to obtain the final power load prediction value.

[0088] The shape and physical meaning of the prediction results of each branch output by the cross-resolution attention interaction module, after cross-resolution attention modulation, are as follows: Low-frequency branch output After modulation, the approximate coefficient representing the long-term trend of the future sequence at the coarsest scale (level 3) is used. High-frequency branch output :and The detail coefficients at the same scale represent the long-term fluctuation components of level 3; High-frequency branch output : Level 2 detail coefficients, representing the medium-term volatility component.

[0089] High-frequency branch output Level 1 detail coefficients, representing short-term and instantaneous fluctuation components.

[0090] Reconstruction process (taking a three-level decomposition with m=3 as an example): 1. The coarsest level of third-level reconstruction Input: Approximation coefficients for Level 3 and detail coefficient .

[0091] Core operation: To and Perform one-dimensional inverse discrete wavelet transform .

[0092] Specific calculations: This transformation includes upsampling (inserting zero values ​​into the sequence) and a reconstruction filter (low-pass filter). Qualcomm Convolution is performed. The formula can be expressed as: (10) Where n represents the discrete-time index of the output sequence (reconstructed signal), and k represents the index of the input coefficient sequence.

[0093] Output: Obtain the approximation coefficients for Level 2. .

[0094] Function: Combines the coarsest-scale trends and fluctuations into a finer-level (Level 2) trend forecast.

[0095] 2. Second-level restructuring Input: Result of the previous level reconstruction And Level 2 detail coefficient .

[0096] Core operation: To and Perform one-dimensional inverse discrete wavelet transform .

[0097] The calculation formula is as follows: (11) Where n represents the discrete-time index of the output sequence (reconstructed signal), and k represents the index of the input coefficient sequence.

[0098] Output: Obtain the approximation coefficients for Level 1. .

[0099] Function: Combines the trend and intermediate fluctuations of Level 2 into a more detailed (Level 1) trend prediction.

[0100] 3. First-level reconstruction (finest scale) Input: The result of the previous level reconstruction And level 1 detail factor

[0101] Core operation: To and Perform one-dimensional inverse discrete wavelet transform .

[0102] The calculation formula is as follows: (12) Where n represents the discrete-time index of the output sequence (reconstructed signal), and k represents the index of the input coefficient sequence.

[0103] Output: The final reconstructed sequence. This is the original resolution prediction result generated by the model, which is the same length as the future prediction window.

[0104] 4. Post-processing and final output Inverse normalization: the reconstructed prediction sequence The model is denormalized to convert the load from its standard distribution within the model back to its original physical dimensions of electrical load (e.g., megawatts, MW).

[0105] Transpose output: The inverse normalized sequence is transposed by row and column to obtain... This is the final predicted power load value generated by the method of the present invention.

[0106] Through this bottom-up, step-by-step wavelet reconstruction, this embodiment accurately and smoothly fuses the prediction components (long-term trend, medium-term fluctuation, and short-term details) at different time scales, optimized by a dual attention mechanism, into a complete, high-resolution future load curve. This method strictly adheres to signal processing principles, ensuring the fidelity of information during decomposition and reconstruction, and is a crucial step in achieving high-precision prediction.

[0107] (vi) Model training and optimization The predicted results obtained from wavelet reconstruction are compared with the actual target data (labels), and the prediction error (loss function, such as MSE) is calculated. Then, the gradient of the loss function with respect to all model parameters is calculated using the backpropagation algorithm, and the model parameters are updated using a gradient descent optimization algorithm (such as Adam). The purpose of this step is to continuously adjust the model parameters, minimize the prediction error, and enable the model to gradually learn the changing patterns of electricity load. Specifically: 1. Training Preparation and Loss Function Definition enter: Training data: Slices of training set samples It contains a large amount of historical data. and forecast data The sample pairs formed.

[0108] Prediction Model: A WPMixer power load prediction model based on a dual attention mechanism was built.

[0109] Loss function definition: Mean squared error (MSE) is used as the primary loss function during the training phase. For a batch of samples, the calculation formula is: (13) Formula explanation: Where B is the number of data entries in the batch, C is the number of variables, and L is the number of prediction sequence steps. and These are the actual input value and the predicted output value of the model, respectively. Function: The MSE loss function can sharply punish large prediction errors, driving the model's prediction curve to approximate the true value in both overall shape and value, making it the standard choice in prediction tasks.

[0110] 2. Iterative optimization and backpropagation Core operation: Stochastic gradient descent using the Adam optimizer (Adaptive Moment Estimation). Adam combines the advantages of momentum and adaptive learning rate, and can typically achieve fast and stable convergence of model parameters in deep model training.

[0111] Training iterations (Epochs): The default setting is 10, iterating through the entire training set 10 times. Each Epoch contains the following steps: a. Forward propagation: The training dataset is then... Input the model, process it, and obtain the output predicted value. .

[0112] b. Loss calculation: Calculate the MSE loss value of all batches of data according to formula (13).

[0113] c. Backpropagation: Using the chain rule, automatically calculate the gradient of the loss function with respect to all trainable parameters of the model (including the weights w and biases b of all linear mapping layers and attention layers). This process reveals the direction for adjusting parameters to reduce losses.

[0114] d. Parameter update: The Adam optimizer updates the model parameters according to the gradient obtained from backpropagation and the rule formula (14). : (14) Where η is the initial learning rate. and This is the bias correction estimate of the first and second moments of the gradient. Its purpose: By repeatedly performing this process, the model parameters... It is gradually adjusted to minimize the loss of the model's predictions.

[0115] 3. Verify monitoring and early shutdown mechanism Validation set monitoring: After each training epoch, the current model is used to perform forward propagation on the validation set V that was not used in training, and the loss of the validation set V is calculated.

[0116] Early Stopping Mechanism: An early stopping mechanism is introduced to prevent overfitting. Specifically, it is set to continuously monitor the loss of the validation set V. If the loss does not decrease to a new minimum point within P consecutive periods (P is the patience value, which is set to 3 by default), the model is determined to be fully trained and has begun to overfit, and training is terminated immediately.

[0117] Model selection: Throughout the training process, the model with the lowest loss on the validation set V is always retained as the optimal model. Purpose: This mechanism ensures that the final model has the strongest generalization ability, avoiding overfitting to the training data.

[0118] 4. Final Testing and Comprehensive Performance Evaluation After training and validation, the best-performing model is evaluated using a test set TS that has not participated in any training or tuning process. The evaluation employs a comprehensive set of metrics, rather than a single metric, to reflect predictive performance from multiple perspectives: Root Mean Square Error (RMSE): (15) It can be used to measure the absolute magnitude of the prediction error.

[0119] Mean Absolute Error (MAE): (16) It can be used to reflect the average error level of the prediction.

[0120] Mean Absolute Percentage Error (MAPE): (17) It can be used to reflect the relative magnitude of the prediction error.

[0121] Accuracy (AC): (18) This indicator is commonly used in the power industry and can comprehensively measure the degree of agreement between the predicted curve and the actual curve.

[0122] in, and These are the i-th actual value and the i-th predicted value, respectively, and n is the number of steps for future predictions, which is 192.

[0123] The validation set is continuously monitored and tuned after each training round to ensure a model with strong generalization ability and prevent overfitting.

[0124] Iterative validation and early stopping mechanism: After each (or several) training iteration, the current model is forward-propagated onto the validation dataset to calculate its loss function value and other performance metrics (such as mean squared error, MSE) on the validation set. This process is used to monitor the model's performance on unseen data. Typically, an early stopping mechanism is introduced: if the validation set loss no longer decreases or even begins to increase over several consecutive iterations, the model is considered overfitted, training is terminated early, and the model parameters are rolled back to the snapshot where the validation set performance was optimal.

[0125] After the model training and tuning are completed, the final "best model" is evaluated fairly and objectively using a test set that has not participated in any training or tuning process.

[0126] Comprehensive performance evaluation: Load forecasting is performed on the test set using the best model, and the forecast results are compared with the actual values. Evaluation should not rely solely on a single accuracy metric, but rather employ a comprehensive evaluation system, typically including: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and Accuracy Calculation (AC): for power load forecasting, an industry-specific accuracy formula.

[0127] As one implementation method, to ensure the effectiveness of this embodiment, four conventional prediction models—Informer, Crossformer, Autoformer, and WPMixer—were used to perform prediction tests on the power load dataset against the prediction model of this invention. The four evaluation metrics were obtained. The dataset was divided using two different methods, and the training and validation tests used two different division ratios: 0.7:0.15:0.15 and 0.8:0.1:0.1. The test results are shown in Table 2. Table 2. Test results of all prediction models;

[0128] As shown in Table 2, this embodiment outperforms the four models mentioned above in all four indicators, demonstrating excellent prediction performance with high accuracy and stability. This embodiment achieves an AC accuracy of 99.25% on the power load dataset II, a 2.22% improvement over the original WPMixer model. The prediction effect is as follows: Figure 5 As shown.

[0129] In summary, this invention, through the organic combination of multiple steps, constitutes a complete "dual attention mechanism," effectively solving the problems pointed out in the background art. It demonstrates greater practical value in addressing the challenges of strong nonlinearity and high randomness in load sequences under the background of new power systems.

[0130] Example 2 This embodiment provides a power load forecasting system based on the dual attention mechanism WPMixer, including: The data processing module is configured to acquire power load time-series data and perform preprocessing. The load forecasting module is configured to input the preprocessed power load time series data into the trained dual attention mechanism WPMixer model to perform power load forecasting. The process involves wavelet decomposition of the input data to obtain multiple resolution branch features, including one low-frequency branch and multiple high-frequency branches. For each branch feature, patching is performed before inputting into an improved Mixer module group. Based on the attention mechanism, slice dimension and embedding dimension processing are performed to obtain deep features. Based on the cross-resolution attention interaction module, the low-frequency branch features in the deep features are used as shared context information and cross-attention calculation is performed with each high-frequency branch feature to obtain modulation features. Based on the modulation features, multi-resolution wavelet reconstruction is performed to obtain the predicted power load value.

[0131] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the power load forecasting method based on the dual attention mechanism WPMixer as described in Embodiment 1 above.

[0132] Example 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the power load forecasting method based on the dual attention mechanism WPMixer as described in Embodiment 1 above.

[0133] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0134] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A power load forecasting method based on the dual attention mechanism WPMixer, characterized in that, include: Acquire time-series power load data and perform preprocessing; The preprocessed power load time series data is input into the trained dual attention mechanism WPMixer model to predict power load. The process involves wavelet decomposition of the input data to obtain multiple resolution branch features, including one low-frequency branch and multiple high-frequency branches. For each branch feature, patching is performed before inputting into an improved Mixer module group. Based on the attention mechanism, slice dimension and embedding dimension processing are performed to obtain deep features. Based on the cross-resolution attention interaction module, the low-frequency branch features in the deep features are used as shared context information and cross-attention calculation is performed with each high-frequency branch feature to obtain modulation features. Based on the modulation features, multi-resolution wavelet reconstruction is performed to obtain the predicted power load value.

2. The power load forecasting method based on the dual attention mechanism WPMixer as described in claim 1, characterized in that, The wavelet decomposition is a three-level discrete wavelet decomposition using the DB4 wavelet basis function.

3. The power load forecasting method based on the dual attention mechanism WPMixer as described in claim 1, characterized in that, The patching process specifically involves dividing each branch feature into non-overlapping patches in the time dimension and mapping them to high-dimensional embedding vectors through a linear embedding layer.

4. The power load forecasting method based on the dual attention mechanism WPMixer as described in claim 1, characterized in that, The improved Mixer module group includes two improved Mixer modules connected in series. The output of the second improved Mixer module is residually connected with the output of the first improved Mixer module to obtain deep features.

5. The power load forecasting method based on the dual attention mechanism WPMixer as described in claim 4, characterized in that, The improved Mixer module includes a cascaded slice dimension processing submodule and an embedding dimension processing submodule; both submodules add a lightweight attention layer before the first static linear layer for dynamic feature fusion. The slice dimension processing submodule is used to perform lightweight attention calculation on the input features based on the Patch dimension, dynamically fuse information between different time segments, and obtain an enhanced temporal feature representation. The embedding dimension processing submodule is used to perform lightweight attention calculation on the input features based on the embedding dimension, dynamically fuse information between different feature channels, and obtain optimized feature channel representations.

6. The power load forecasting method based on the dual attention mechanism WPMixer as described in claim 1, characterized in that, In the cross-resolution attention interaction module, low-frequency branch features generate keys and values ​​through linear projection as shared context information; each high-frequency branch feature generates a query through dedicated linear projection, and low-frequency trend information modulates high-frequency detail information based on parallel cross-attention calculation to obtain modulated features.

7. The power load forecasting method based on the dual attention mechanism WPMixer as described in claim 1, characterized in that, The process of performing multi-resolution wavelet reconstruction based on modulation features to obtain the predicted power load specifically includes: The modulated branch features are used as wavelet coefficients, and the inverse discrete wavelet transform is performed step by step from the coarsest scale to the finest scale. The third-level approximation coefficients and detail coefficients are reconstructed into second-level approximation coefficients; Reconstruct the first-level approximation coefficients by combining the second-level approximation coefficients with the corresponding detail coefficients; The first-level approximation coefficients and detail coefficients are reconstructed into the final prediction sequence; The final predicted sequence is denormalized to restore the physical dimensions of the power load and obtain the predicted value.

8. A power load forecasting system based on the dual attention mechanism WPMixer, characterized in that, include: The data processing module is configured to acquire power load time-series data and perform preprocessing. The load forecasting module is configured to input the preprocessed power load time series data into the trained dual attention mechanism WPMixer model to perform power load forecasting. The process involves wavelet decomposition of the input data to obtain multiple resolution branch features, including one low-frequency branch and multiple high-frequency branches. For each branch feature, patching is performed before inputting into an improved Mixer module group. Based on the attention mechanism, slice dimension and embedding dimension processing are performed to obtain deep features. Based on the cross-resolution attention interaction module, the low-frequency branch features in the deep features are used as shared context information and cross-attention calculation is performed with each high-frequency branch feature to obtain modulation features. Based on the modulation features, multi-resolution wavelet reconstruction is performed to obtain the predicted power load value.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the power load forecasting method based on the dual attention mechanism WPMixer as described in any one of claims 1-7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the power load forecasting method based on the dual attention mechanism WPMixer as described in any one of claims 1-7.

Images