Power load prediction method and system based on time sequence decomposition and attention mechanism

Abstract

The invention relates to the technical field of load prediction, and provides a power load prediction method and system based on time sequence decomposition and an attention mechanism, and the method comprises the steps: carrying out the adaptive time sequence decomposition of an obtained original load sequence, calculating the sample entropy of each decomposed component, and carrying out the clustering; constructing a group of encoder and decoder networks for each piece of clustered data, performing parallel encoding to extract features, performing serial decoding reconstruction on the features from low frequency to high frequency, and outputting prediction data from low frequency to high frequency step by step; the weight is initialized based on the sample entropy, the trained weight is obtained through optimization in the encoder and decoder network training process, and the predicted value of the power load is obtained through weighted fusion. According to the method, adaptive time sequence decomposition, a weight mechanism guided by sample entropy and an attention-enhanced encoder-decoder structure are introduced, so that multi-component collaborative modeling and cross-scale dynamic prediction are realized, and the prediction accuracy and stability in complex load data and small sample scenes are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of power load forecasting technology, specifically to a power load forecasting method and system based on time-series decomposition and attention mechanism. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] The power system is characterized by the simultaneous production, transmission, and consumption, necessitating a constant supply-demand balance to ensure stable, safe, and economical operation. Load forecasting, a crucial element of power grid dispatching and operation management, directly impacts the scientific nature of generation planning, the rationality of dispatching strategies, and the optimal allocation of energy resources. High-precision load forecasting not only effectively improves power system operating efficiency and reduces operating costs but also plays a vital role in ensuring normal societal operation and enhancing the quality of electricity services. Especially in the context of new power systems characterized by large-scale integration of renewable energy and the increasing prevalence of demand response mechanisms, the complexity and importance of load forecasting have further increased, placing higher demands on the intelligence and adaptive capabilities of forecasting models.

[0004] Currently, power load forecasting mainly relies on traditional statistical modeling methods (such as time series analysis and regression models) and some classic machine learning methods. However, with the increasing complexity of power grid structures and the widespread integration of multi-source heterogeneous factors, these methods have exposed a series of technical bottlenecks in practical applications. First, traditional models generally rely on linear or stationary assumptions, making it difficult to adapt to the widespread nonlinear and non-stationary characteristics in load data. Second, when faced with highly uncertain external variables such as meteorological factors, renewable energy generation output, and changes in user behavior, the adaptability and generalization ability of existing models are significantly insufficient. In addition, many methods heavily rely on large-scale, high-quality historical data and are sensitive to data noise, missing values, and outliers, making it difficult to meet the needs of real-time forecasting. Although new deep learning structures, represented by Transformers, have shown good performance in time series modeling in recent years, with models such as Crossformer, iTransformer, and PatchTST optimized for specific structures, they still suffer from unstable prediction accuracy, weak model generalization ability, and insufficient feature fusion mechanisms when facing problems such as the coordinated changes of variables in load sequences and the decoupling of multiple components. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a power load forecasting method and system based on temporal decomposition and attention mechanisms. It introduces adaptive temporal decomposition, a sample entropy-guided weighting mechanism, and an attention-enhanced encoder-decoder structure, thereby achieving multi-component collaborative modeling and cross-scale dynamic forecasting. This effectively improves the forecasting accuracy and stability under complex load data and small sample scenarios.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: One or more embodiments provide a power load forecasting method based on time-series decomposition and attention mechanisms, including the following steps: The original load sequence is adaptively decomposed into time series, and the sample entropy of each component after decomposition is calculated and clustered to obtain low-frequency data, medium-frequency data and high-frequency data. For each of the obtained low-frequency, mid-frequency, and high-frequency data, an encoder and decoder network is constructed. After parallel encoding to extract features, the features are serially decoded and reconstructed from low frequency to high frequency, and the predicted data from low frequency to high frequency is output step by step. The weights are initialized based on sample entropy and optimized during the training of the encoder and decoder networks to obtain the trained weights. The trained weights are then used to weight and fuse the prediction data from low frequency to high frequency to obtain the predicted value of power load.

[0007] A power load forecasting system based on time-series decomposition and attention mechanism includes: The multi-scale time series decomposition module is configured to perform adaptive time series decomposition on the acquired original load sequence, calculate the sample entropy of each decomposed component and perform clustering to obtain low-frequency data, mid-frequency data and high-frequency data. The model prediction module is configured to construct a set of encoder and decoder networks for each of the obtained low-frequency, mid-frequency and high-frequency data, perform parallel encoding to extract features, and then serially decode and reconstruct the features from low frequency to high frequency, outputting the prediction data from low frequency to high frequency step by step. The adaptive weighted fusion module is configured to initialize weights based on sample entropy and optimize the trained weights during the encoder and decoder network training process. The trained weights are then used to weight and fuse the prediction data from low frequency to high frequency to obtain the predicted value of power load.

[0008] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps in the above-described power load forecasting method based on time-series decomposition and attention mechanism.

[0009] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps in the above-described power load forecasting method based on time-series decomposition and attention mechanism.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: The method of this invention can effectively address the nonlinear and time-varying characteristics of complex load data. It improves the accuracy of feature extraction through adaptive temporal decomposition and avoids the limitations of traditional manual feature design. By using sample entropy to guide clustering and weight initialization, the model acquires a certain ability to perceive data complexity, thereby improving prediction robustness in small-sample scenarios. The serial decoding and weighted fusion mechanism introduces a hierarchical strategy in modeling different frequency bands, achieving multi-component collaborative modeling and improving prediction accuracy and stability across time scales. Furthermore, the introduction of an attention mechanism enhances the model's feature recognition ability, further improving its generalization performance.

[0011] The advantages of the present invention, as well as its additional advantages, will be described in detail in the following specific embodiments. Attached Figure Description

[0012] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0013] Figure 1 This is a flowchart illustrating the power load forecasting method according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the preprocessing process for multi-scale temporal decomposition in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the prediction results in an experimental example of Embodiment 1 of the present invention. Detailed Implementation

[0014] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0015] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0016] It should be noted that the terminology used herein is for describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. It should be noted that, without conflict, the various embodiments and features within those embodiments can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0017] Example 1 In one or more of the technical solutions disclosed in the embodiments, such as Figures 1 to 3 As shown, a power load forecasting method based on time-series decomposition and attention mechanism includes the following steps: Step 1: Perform adaptive time-series decomposition on the obtained original load sequence, calculate the sample entropy of each decomposed component and perform clustering to obtain low-frequency data, medium-frequency data and high-frequency data. Step 2: For each of the obtained low-frequency, mid-frequency, and high-frequency data, construct a set of encoder and decoder networks, perform parallel encoding to extract features, and then serially decode and reconstruct the features from low frequency to high frequency, outputting the predicted data from low frequency to high frequency step by step. Step 3: Initialize weights based on sample entropy, and optimize the trained weights during the encoder and decoder network training process. Use the trained weights to perform weighted fusion of low-frequency to high-frequency prediction data to obtain the predicted value of power load. This method takes raw power load time-series data as input. First, it divides the sequence into multiple intrinsic modal components according to scale using an adaptive time-series decomposition method (such as CEEMD or VMD). Then, it evaluates the complexity of each modal component based on sample entropy values ​​and classifies the components using clustering algorithms such as K-means, obtaining data subsets at three levels: low frequency, mid frequency, and high frequency. Subsequently, an independent encoder-decoder structure is constructed for each frequency band. The encoder is used to extract subsequence features, and the decoder is used to reconstruct and predict the features. An attention mechanism is used to dynamically adjust the region of interest in feature extraction, improving the model's responsiveness to important features. Multi-band features are input to the decoding module in order from low frequency to high frequency, reconstructing the predicted values ​​at each level step by step, achieving time-dependent hierarchical modeling. Finally, initial weights are set according to the sample entropy of each frequency band, and the weight parameters are fine-tuned and optimized during model training. A weighted fusion strategy is used to integrate the prediction results of each frequency band into the final power load prediction value, thus forming a load forecasting process that takes into account hierarchical modeling and dynamic weight adjustment.

[0018] This implementation effectively addresses the nonlinear and time-varying characteristics of complex load data. Adaptive temporal decomposition improves feature extraction accuracy and avoids the limitations of traditional manual feature design. Using sample entropy to guide clustering and weight initialization endows the model with a certain degree of data complexity awareness, thereby enhancing prediction robustness in small-sample scenarios. The serial decoding and weighted fusion mechanism introduces a hierarchical strategy in modeling different frequency bands, achieving multi-component collaborative modeling and improving prediction accuracy and stability across time scales. Furthermore, the introduction of an attention mechanism enhances the model's feature recognition capability, further improving its generalization performance.

[0019] Step 1 is the data preprocessing step. The preprocessed data has practical physical meaning, namely: low-frequency data corresponds to the signal outline, mid-frequency data corresponds to the signal details, and high-frequency data corresponds to the signal detail compensation information. Low-frequency components capture long-term trends (such as seasonal effects), mid-frequency components extract short-term fluctuations (such as intraday cycles), and high-frequency components filter random noise. This decomposition effectively reduces sequence complexity and provides hierarchical input for subsequent models. Compared with the traditional Empirical Mode Decomposition (EMD) method, this embodiment introduces a clustering optimization strategy to avoid mode aliasing and ensure the independence of each sub-item.

[0020] Step 1 involves adaptive time-series decomposition of the acquired original load sequence, followed by clustering of the decomposed components to obtain low-frequency, mid-frequency, and high-frequency information. This method includes the following steps: Step 11: For each univariate measurement point time series data in the original load sequence, the fully adaptive noise ensemble empirical mode decomposition (CEEMDAN) algorithm is used to perform adaptive decomposition to obtain multiple IMF components; CEEMDAN adaptively decomposes complex raw data into a series of intrinsic mode function (IMF) components with different center frequencies. After this step, the univariate sequence at each measurement point is decomposed into several IMF components, such as 12 IMF components. These components, from high frequency to low frequency, reveal information at different scales in the original signal, such as detailed fluctuations, periodic patterns, and long-term trends. Assuming the input is raw data x, set two key parameters: ε: The amplitude (standard deviation) of the added adaptive white noise; I: The number of times the ensemble average is performed for each stage (i.e., the number of times white noise is added and decomposed).

[0021] Step 11.1: Extraction of the first-order mode (IMF1), looping from i=1 to I, each loop includes the following process: 1) Generate a sequence of different white noise with an amplitude of ε. .

[0022] 2) Add white noise to the original signal to create a noisy signal: (1); 3) For noisy signals Perform standard EMD (Empirical Mode Decomposition) to obtain its first eigenmode function, denoted as . ; After the loop ends, all the results obtained from the I-th decomposition are... Calculate the average value, which will be the final first-order modal component: (2); Calculate the first-order residual: (3); Step 11.2: Extraction of the k-th mode (IMFk, k≥2); Iterate over k=2,3,...,K until the residuals can no longer be decomposed: Loop through i=1 to i, and perform the following procedure in each loop: 1) Generate a new white noise sequence .

[0023] 2) Add white noise to the residual of the previous order: .

[0024] in, This represents the k-th mode obtained after EMD decomposition of the signal.

[0025] 3) Perform EMD decomposition on this signal to obtain a modal component, denoted as . .

[0026] 4) Calculate the mean of the k-th mode iteratively: (4); 5) Update the k-th order residual: (5); Step 11.3, when the residual When the signal becomes a monotonic function and the IMF can no longer be extracted, the decomposition process stops. At this point, the original signal... It is completely decomposed into a series of IMFs and a final residual sum: (6); After the above three steps, the univariate sequence at each measurement point It is decomposed into 12 IMF components, namely ; Step 12: Calculate the sample entropy for each IMF component; Sample entropy is used to measure the complexity and randomness of time series. The larger the value, the more complex and irregular the series; the smaller the value, the more regular and smooth the series.

[0027] Specifically, for each IMF component sequence obtained in step 11, its sample entropy value is calculated one by one according to the formula for calculating sample entropy. After this step is completed, the 12 IMF components of each measurement point will correspond to 12 sample entropy feature values; The 12 IMF components at each measurement point will correspond to 12 sample entropy feature values. The formula for calculating sample entropy is as follows: (7); Where m=2, r=0.1, and N is the length of the sequence. Represents the time-series values ​​of IMF components. This represents the average proportion of any two distinct subsequences in an IMF component of length m whose maximum distance is less than a threshold r. This represents the average proportion of any two distinct subsequences in an IMF component of length m+1 whose maximum distance is less than a threshold r.

[0028] The IMF component set of a single measurement point yields 12 sample entropy feature sets, as shown below:

[0029] Where i is the measurement point number, and each sample entropy feature SE variable represents the sample entropy feature value calculated by different IMF components, that is, the value calculated by formula 7.

[0030] Step 13: Cluster the sample entropy values ​​of all IMF components of all obtained measurement point sequence data; Specifically, the K-means clustering algorithm is used; ; In this embodiment, the number of clusters is set to K=3, and the IMF components are divided into three categories; IMF components with lower sample entropy values The corresponding low-frequency components in the sequence change slowly and have strong regularity. This is an approximate term, reflecting the long-term trend of load changes; The component with the middle value of sample entropy This corresponds to the mid-frequency component, which is a detail item and reflects the periodic fluctuations of the load; Components with higher sample entropy values Then the corresponding high-frequency components This is a compensation term that reflects the random noise and instantaneous disturbances in the load sequence.

[0031] After clustering each measurement point, three component sequence data sets will be obtained:

[0032] Where i is the measurement point number; Step 14: Stack all IMF components belonging to the same cluster separately. Each cluster forms a one-dimensional time series, which are low-frequency data, medium-frequency data and high-frequency data respectively. By clustering, all IMF components belonging to the same component (low-frequency, mid-frequency, or high-frequency) are superimposed to reconstruct three new one-dimensional time series with clearer physical meanings. The details are as follows: All IMF components marked as low frequency are summed and reconstructed under the condition of equal length point-to-point. The reconstructed data mainly carries long-term trend information.

[0033] The data is reconstructed by adding all IMF components marked as intermediate frequency. The reconstructed data is performed by adding and reconstructing the data in the case of equal length point-to-point. The reconstructed data mainly carries the periodic changes and main fluctuation patterns.

[0034] It is reconstructed by adding all the IMF components marked as high frequency. The addition reconstruction is carried out under the condition of equal length point-to-point. The reconstructed data is mainly random fluctuations and disturbances.

[0035] At this point, the original multivariate input data has been transformed into three parallel sets of multivariate data matrices. These represent its low-frequency, mid-frequency, and high-frequency component matrices, respectively:

[0036] Through the above steps, the multivariate input slice data of each original load data is transformed into three sets of parallel multivariate data, representing its low-frequency approximation, mid-frequency details and high-frequency compensation components, respectively, laying the foundation for the subsequent hierarchical encoder to process the features of different data components.

[0037] In the above implementation, the preprocessing step in step 1 achieves high-quality, physically interpretable frequency band decoupling through CEEMDAN, sample entropy, and K-means. This decoupling method not only preserves the dynamic characteristics of the original load sequence at different time scales but also effectively separates noise interference, enabling subsequent models to design dedicated coding strategies for each frequency band feature. Low-frequency components capture long-term trends, mid-frequency components characterize periodic patterns, and high-frequency components focus on abnormal fluctuations; the three work synergistically to improve overall modeling accuracy and robustness. Based on this, the model can extract and reconstruct features from low-frequency, mid-frequency, and high-frequency components separately, fully leveraging the modeling advantages of various encoders at different scales.

[0038] By applying CEEMDAN for refined decomposition, the complex original load sequence can be adaptively decomposed into a series of intrinsic mode functions (IMFs) with different frequency characteristics, effectively overcoming the mode aliasing problem of traditional decomposition methods and providing high-quality basic components for subsequent analysis. Intelligent reconstruction is performed using sample entropy, an indicator that can quantify sequence complexity and randomness. This process is essentially based on the characteristics of the data itself, adaptively separating noise, detail fluctuations and long-term trends, which greatly reduces the non-stationarity of the original data and lays a solid foundation for the model to capture features at different time scales. Step 2 is the hierarchical encoding-decoding stage, which performs deep feature extraction and collaborative prediction on the low-frequency, mid-frequency, and high-frequency multivariate sequences generated in the preprocessing stage. This stage achieves effective fusion of multi-scale temporal features through a hierarchical architecture of parallel encoding and serial decoding.

[0039] Step 2: For each of the obtained low-frequency, mid-frequency, and high-frequency data, construct an encoder and decoder network to form three encoder-decoder structures. Perform parallel encoding to obtain the extracted low-frequency to high-frequency features. Then, perform serial decoding and reconstruction on the extracted features from low to high frequency, outputting the predicted data from low to high frequency step by step. The process of generating the predicted data includes the following steps: Step 21: Construct a data embedding layer to embed the input low-frequency, mid-frequency, and high-frequency data to obtain low-frequency embedding matrices, mid-frequency embedding matrices, and high-frequency embedding matrices, respectively. Specifically, data embedding processing is performed on low-frequency data, mid-frequency data, and high-frequency data respectively, including: Step 211, Numerical Embedding: Input the scalar value of each variable in the low-frequency, mid-frequency, and high-frequency data into a trainable linear projection layer, and map it into the corresponding high-dimensional vector representation, which serves as the data embedding vector. Step 212, Location Embedding: For each time step of low-frequency data, medium-frequency data, and high-frequency data, a location encoding scheme composed of sine and cosine functions of different frequencies is used to generate a corresponding time location information vector, which serves as the location embedding vector. A position coding scheme composed of sine and cosine functions of different frequencies, specifically: For data positions within a sequence, if the dimension has an even index, a sine function is used for transformation, as shown in the following formula: (8); For data positions within a sequence, if the dimension has an odd index, a cosine function is used for transformation. The transformation formula is as follows: (9); Where pos is the position of the element in the sequence, and i is the index in the positional encoding vector. dm is the total dimension of the positional encoding vector, set to 128. This is ultimately transformed into a positional embedding matrix of size [13, 12, 128]. .

[0040] To compensate for the model's inherent lack of perception of temporal order, this embodiment employs a fixed-position encoding scheme composed of sine and cosine functions. This scheme generates a unique position vector for each time step in the sequence, the value of which is composed of a combination of sine and cosine functions of different frequencies, enabling the model to clearly perceive the sequential order of data points.

[0041] Step 213: Add the numerical embedding vector and the positional embedding vector at the same time step element-wise to obtain the initial high-dimensional representations of the low-frequency, mid-frequency, and high-frequency data, i.e., the embedding matrices. This step ensures that the model can understand both the numerical meaning of the data and its position in the time stream. Specifically, the low-frequency numerical embedding vector and the positional embedding vector are added element-wise to obtain the low-frequency embedding matrix, which serves as the input to the low-frequency encoder. The intermediate-frequency numerical embedding is added to the corresponding positional embedding to obtain the intermediate-frequency embedding matrix, which serves as the input to the intermediate-frequency encoder. The high-frequency numerical embedding is added to the positional embedding to obtain the high-frequency embedding matrix, which serves as the input to the high-frequency encoder.

[0042] Specifically, the process of constructing numerical embedding vectors includes data tokenization and numerical embedding: Data tokenization, separating low-frequency, medium-frequency, and high-frequency data. The data is split into tokens. Specifically, the original data matrix size is [192, 13]. The window size W is set to 16, equivalent to a duration of 4 hours. The time series of each variable is divided into 12 blocks (tokens) according to a window of size W, with each token having a length of 16. After all measurement points are tokenized, the original data matrix... Transform into a matrix of size [13, 12, 16]. .

[0043] Numerical embedding maps the token vector of each variable to a high-dimensional space through a trainable linear projection layer. This mapping transforms the token vector numerically into a high-dimensional vector, enabling it to carry richer feature information. Here, the linear projection layer parameters are set to in:16, out:128, which means transforming a token vector of length 16 into a high-dimensional embedding vector of length 128. Finally, it is transformed into a matrix of size [13, 12, 128]. .

[0044] Finally, the values ​​at the same time step are embedded into the vector. With position embedding vector By adding elements one by one, we obtain the initial high-dimensional representations of the low-frequency, mid-frequency, and high-frequency data, namely the embedding matrix.

[0045] Step 22: Construct a hierarchical encoder stack for feature mining: For the low-frequency embedding matrix, mid-frequency embedding matrix and high-frequency embedding matrix, perform parallel feature extraction on the corresponding encoders to obtain the low-frequency feature matrix, mid-frequency feature matrix and high-frequency feature matrix respectively; The hierarchical encoder stack in this embodiment includes three parallel encoders, which are used to process the low-frequency, mid-frequency, and high-frequency embedding matrices, respectively. Each encoder consists of multiple (e.g., two) stacked coding layers with identical structures. Each encoder layer is sequentially connected to a long- and short-range attention layer, a feedforward network, a residual connection, and a layer normalization layer. The long and short distance attention layers are used to perform long-distance and short-distance attention operations on the input embedding matrix to obtain attention-enhanced features that include contextual information. In this embodiment, the long-distance attention mechanism ensures that the data at each time step in the sequence simultaneously focuses on the information of all time steps at the same measurement point.

[0046] The long-range attention mechanism processing uses a low-frequency embedding matrix. Let's take an example to illustrate. The tensor dimension is [13, 12, 128]. Here, the self-attention mechanism is implemented directly without any dimension adjustment. For the query vector, with Using the key-value vector, attention calculation is performed according to Formula 10, which allows data from each time step in the sequence to simultaneously focus on information from all time steps at the same measurement point. The subsequent mid-frequency and high-frequency embedding matrices... , Perform long-distance attention calculations using the same procedure.

[0047] The short-range attention mechanism enables data at each time step in the sequence to simultaneously focus on information from all measurement points within the same time period, thereby efficiently capturing complex global dependencies within the sequence. Specifically: Low-frequency embedding matrix Let's take an example to illustrate. The tensor dimension is [13, 12, 128]. The self-attention mechanism here first swaps the first and second dimensions. The tensor dimensions become [12, 13, 128], and then adjusted... For the query vector, with Using the key-value vector, attention calculation is performed according to Formula 10, which allows the data at each time step in the sequence to simultaneously focus on the information of all measurement points within the same time period. The subsequent mid-frequency and high-frequency embedding matrices... , Perform short-distance attention calculations using the same procedures described above.

[0048] This embodiment utilizes the aforementioned dual-dimensional attention mechanism of short-range and long-range attention to simultaneously optimize the calculation of temporal and parameter space dependencies. Specifically, short-range attention (SDA) captures subtle fluctuations within measurement points, while long-range attention (LDA) correlates changes between global measurement points. Low-frequency encoders processing low-frequency features can thereby capture long-term trend periodicity; high-frequency encoders processing high-frequency features can discover short-term correlation patterns between instantaneous fluctuations. This mechanism significantly improves training efficiency, making it particularly suitable for predicting high-frequency workload data.

[0049] Specifically, the core calculation formula for attention in both long-distance and short-distance attention layers is as follows: (10) QKV refers to Each embedding vector in the array.

[0050] Each data matrix has a shape and size of [13, 12, 128]. The long-distance attention result can be calculated by applying the above formula (10). The results imply the relationship between the 12 tokens within the same measurement point sequence.

[0051] Each data matrix has a shape and size of [13, 12, 128]. First, the first two dimensions are adjusted to [12, 13, 128]. Then, the short-range attention result can be calculated by applying the formula (10) above. The results imply the relationship between 13 measurement points of the multidimensional sequence within the same time period.

[0052] The dimensions of the short-range attention result are adjusted back to [13, 12, 128], and then cross-attention calculation is performed with the long-range attention result. The calculation result is... (Size [13,12,128]) can simultaneously contain the dependencies of two dimensions of time measurement points.

[0053] Feedforward networks are used to transform and non-linearly enhance the features output by attention layers of varying distances that already contain contextual information. Each feedforward network corresponds to an independent fully connected network at each time point and contains a non-linear activation function. Specifically, the feedforward network's response to the attention layer's output... By using features as input for deep transformation and nonlinear enhancement, higher-level feature representations are extracted. (Size [13, 12, 128]), the formula is expressed as: (11); Residual connections and layer normalization are used to apply residual connections to the output of each sublayer and then normalize them. This effectively alleviates the vanishing gradient problem in deep network training, accelerates model convergence, and stabilizes the training process.

[0054] The sub-layers include the preceding long and short distance attention layers and the feedforward network; Through the stacking of multiple coding layers Figure 1 In this process, each encoder stacks two coding layers, and each parallel encoder ultimately outputs a feature matrix rich in the depth features of its frequency band, namely a low-frequency feature matrix, a mid-frequency feature matrix, and a high-frequency feature matrix.

[0055] Step 23: Construct a hierarchical decoder stack to generate prediction components: For each encoder, set up a decoder, and connect the corresponding low-frequency decoders in sequence to realize serial decoding from low-frequency features to high-frequency features. To fuse the three frequency domain features and generate predictions for future sequences, this embodiment designs a hierarchical decoder stack consisting of three decoders connected in series.

[0056] Each decoder includes multiple decoding layers, and each decoding layer contains a decoder attention layer and a feedforward network. The decoder attention layer includes a mask self-attention mechanism layer and a cross attention mechanism layer. Masked self-attention mechanism: Let's take an example to illustrate. Query vector Q, with Given the key vector K and value vector V, and a mask matrix M constructed using the positions, the attention operation is performed according to Equation 12. The mask self-attention was calculated according to the above operating method.

[0057] in These correspond to the input embedding matrices of the three decoders, with dimensions [13, 12, 128].

[0058] To ensure that when decoding and generating the prediction for the t-th time point, the known information from the previous t-1 time points is utilized, future information leakage is prevented, and the autoregressive properties of the prediction are guaranteed.

[0059] The formula for calculating mask self-attention is as follows: (12); Here, M is the mask matrix, which has the same dimension as the attention score matrix. The rule for constructing the mask matrix is: set the mask value to 0 for visible positions. For the known and retained first t-1 time points, take them as visible positions and assign corresponding mask values. ; Mask out the invisible location at time point t. (Or set it to a negative number with a very large absolute value); Cross-attention mechanism: The output of the layer above the decoder is used as the query vector Q, and the output of the encoder corresponding to the decoder is used as the key vector K and the value vector V to perform attention operation; The cross-attention mechanism is a crucial bridge connecting the encoder and decoder. In this mechanism, the query vector Q comes from the output of the previous layer of the decoder, while the key vector K and value vector V come from the feature matrix output of the corresponding parallel encoder in step 2. For example, the first decoder uses the low-frequency feature matrix as the source of K and V. This allows the decoder to selectively retrieve relevant information from the rich contextual features extracted by the encoder to guide the current prediction.

[0060] Feedforward network: Functionally the same as the feedforward network in the encoder, it transforms and non-linearly enhances the features after cross-attention fusion.

[0061] In step 2, based on the constructed hierarchical decoder stack, the features extracted by the encoder are serially decoded and reconstructed, including the following steps: Step 231: Starting from the low-frequency feature processing, the first decoder uses the low-frequency feature matrix output by the first encoder connected to it as the context and the initialized position code as the query to perform decoding and reconstruction, generating preliminary low-frequency component prediction results that focus on long-term trends. Step 232: The second decoder uses the low-frequency component prediction result output by the first decoder as the query and the mid-frequency feature matrix output by the corresponding second encoder as the context to perform decoding and reconstruction, generating a mid-frequency component prediction result with periodic fluctuation information superimposed on the trend. Step 233: The third decoder uses the intermediate frequency component prediction result output by the second decoder as the query and the intermediate frequency feature matrix output by the corresponding third encoder as the context to perform decoding and reconstruction, generating a high frequency component prediction result containing residual information such as random fluctuations. The input of the decoder stack in the above steps is defined as the input of the first decoder being the initial positional encoding, which is determined by the prediction step size. In this embodiment, the prediction step size is set to 192, the embedding dimension is set to 128, and the token length is set to 16. Then, by applying the positional encoding formulas (8) and (9), the size of the embedded data matrix is ​​[13, 12, 128].

[0062] The input to each subsequent decoder is the output feature sequence of the previous decoder; the serial processing logic involves three decoders connected in series. The first decoder uses low-frequency features as context (K, V) to generate preliminary component predictions focusing on long-term trends. The second decoder uses the output of the first decoder as a query (Q) and introduces mid-frequency detail features (K, V) to overlay detailed information such as periodic fluctuations on top of the trend. The third decoder works similarly, using high-frequency features to further refine the prediction and capture residual information such as random fluctuations. This serial design embodies a prediction logic that progressively refines from macroscopic trends to microscopic fluctuations.

[0063] Furthermore, each decoder is connected to a linear projection head (fully connected layer) at its backend, which maps the high-dimensional features output by the decoder back to the target prediction dimension, directly outputting the component prediction result corresponding to that frequency band. The outputs of the three decoders are the component prediction results for their respective frequency bands. ; To address the shortcomings of existing multi-component collaborative modeling, current methods often directly model the original sequence, lacking an effective hierarchical decoupling mechanism. Furthermore, traditional encoder-decoder structures lack hierarchical weight allocation mechanisms when fusing multi-component results, making it difficult to adapt to dynamic changes in workload across scales. This embodiment leverages the advantages of a hierarchical encoder-decoder design with parallel encoding and serial decoding, specifically optimized for the frequency characteristics of different components. Parallel encoding: Three encoders process high, medium, and low-frequency data in parallel. The attention mechanism within each encoder focuses on mining the dependencies between different time points within the same frequency component, avoiding mutual interference between different frequency features and achieving purer and deeper feature extraction. Serial decoding: The three decoders work serially, helping the model establish a progressive prediction logic from macroscopic trends (low frequency) to microscopic fluctuations (high frequency). Subsequent decoders can refer to the previously decoded low-frequency (trend) and medium-frequency (detail) information to further refine the high-frequency (random fluctuation) prediction, enabling the final prediction result to organically integrate information from different scales, resulting in a more holistic and coordinated outcome. In step 3, the predicted values ​​of the low-frequency, mid-frequency, and high-frequency components are combined into a final predicted value. The key technology lies in the determination and optimization of the weights. ; Step 3, the process of training the constructed encoder and decoder networks, includes the following steps: Step S1: Obtain historical power load sequence data, perform adaptive time series decomposition, calculate the sample entropy of each decomposed component and perform clustering to obtain low-frequency information, medium-frequency information and high-frequency information to construct a dataset. Step S2: Using the low-frequency, mid-frequency, and high-frequency data in the first segment of the time-segmented data as input data, and the low-frequency, mid-frequency, and high-frequency data in the second segment of the segmented data as target output data, the input data are fed into the encoder and decoder networks. After parallel encoding to extract features, the features are serially decoded and reconstructed from low frequency to high frequency, and the predicted data from low frequency to high frequency is output step by step. Step S3: Based on the current weights, aggregate the prediction data from low frequency to high frequency to obtain the first aggregation result. Based on the current weights, aggregate the target output data of the model to obtain the second aggregation result. Calculate the loss value based on the first aggregation result and the second aggregation result, and use the gradient descent method to iteratively update the data to obtain the final trained encoder and decoder network and the trained weights.

[0064] In step S1, the historical power load sequence data is obtained, segmented by time, and then adaptive time-series decomposition is performed. The sample entropy of each decomposed component is calculated and clustered to obtain low-frequency, medium-frequency, and high-frequency information. The process of constructing a dataset includes the following steps: Step S11, Dataset Collection and Segmentation: Obtain historical power load sequence data and divide it into training set, validation set and test set according to the proportion; First, based on the specific power load forecasting scenario, determine the relevant time range and measurement points (such as meteorological data and weather forecast data), and collect all historical multivariate time series data that constitute the original dataset. For supervised model training and evaluation, the complete dataset needs to be divided into training, validation, and test sets according to a preset ratio (e.g., 0.7:0.2:0.1). This division process must follow the temporal order of the data to prevent future information leakage and ensure the reliability of the model evaluation results.

[0065] Step S12, Sequence Slicing and Sample Generation: Perform time series segmentation on the sequence data of the dataset to obtain slice data, and obtain training sample pairs by dividing the data according to the time sequence. Specifically, the slice data before the set time point is used as input data, and the data after the set time point is used as target prediction data in chronological order. To adapt to deep learning-based prediction models, training data needs to be transformed into sample units that the model can process. A series of continuous multivariate time series slices are generated by using a sliding window of a set length along the time axis to extract segments from the multivariate training data.

[0066] Each slice of data will be further split into two parts: 1) Model input data: a multivariate sequence of the previous consecutive time steps within each window, used for the model to learn its features; 2) Target prediction data: A multivariate sequence of future time steps following the input data within each window, serving as the target value that the model needs to fit. This operation transforms the original training data into a large number of input-output sample pairs, preparing it for model training.

[0067] Specifically, in order to process the samples into a form acceptable to the model, this embodiment uses a sliding window of fixed length L to sequentially extract multivariate training data along the time axis with a step size S, generating a series of continuous multivariate time series slices (TRS). Here, the parameter L is set to 384 (duration is 4 days), and S is set to 1.

[0068] Each slice of data in the TRS set will be further divided into two equal parts: 1) Model input data TRin: the previous 192 consecutive steps of multivariate sequence is represented as ,in 2) Target prediction data TRout: A multivariate sequence of the next 192 steps immediately following the input data. TRout is represented as... ,in .

[0069] Step S13, CEEMDAN Adaptive Decomposition: For each univariate measurement point time series data in the slice input data obtained in step S12, the fully adaptive noise ensemble empirical mode decomposition (CEEMDAN) algorithm is used to perform time series decomposition to obtain IMF components; Step S14: Calculate the sample entropy of each IMF component; Step S15: Cluster the sample entropy of all IMF components based on K-means; The K-means clustering algorithm is used to perform unsupervised clustering on the sample entropy feature values ​​of all measurement points and all IMF components obtained in step S14.

[0070] Step S16, IMF component sequence data reconstruction: Based on the clustering results of step S15, all IMF components belonging to the same category (low frequency, medium frequency, high frequency) are superimposed to reconstruct three new one-dimensional time series with clearer physical meaning.

[0071] Specifically, low-frequency data (approximate data) is reconstructed by adding all IMF components marked as "low-frequency" and mainly carries long-term trend information from the original load sequence.

[0072] Intermediate frequency (IF) data (detailed data): It is reconstructed by adding all IMF components marked "IF", and mainly carries the periodic changes and main fluctuation patterns in the original load sequence.

[0073] High-frequency data (compensated data): reconstructed by adding all IMF components marked as "high frequency", mainly containing random fluctuations and noise in the sequence.

[0074] To address the contradiction between data complexity and model generalization ability, power load sequences are influenced by multiple intertwined factors, exhibiting strong nonlinearity, time-varying nature, and randomness, as well as the problem of coordinated changes in variables. Existing models rely on manual feature engineering, which makes it difficult to adaptively extract usability features. This embodiment proposes a multivariate time series adaptive decomposition and reconstruction method based on CEEMDAN-sample entropy-Kmeans. By jointly optimizing the decomposition scale and frequency domain feature clustering, it achieves the physical interpretability separation of trend, periodic, and noise components in the load sequence. This method can dynamically identify the frequency domain affiliation of each order of IMF without prior knowledge, effectively improving the learning efficiency and prediction accuracy of subsequent deep learning models for complex load change patterns, and enhancing the robustness of the model under abrupt change scenarios.

[0075] In step S3, based on the current weights, the prediction data from low frequency to high frequency is aggregated to obtain a first aggregation result. Based on the current weights, the target output data of the model is aggregated to obtain a second aggregation result. The loss value is calculated based on the first and second aggregation results, and gradient descent is used for iterative updates to obtain the final trained encoder and decoder networks and the trained weights. The process includes the following steps: Step S31: Obtain the sample entropy corresponding to the prediction data from low frequency to high frequency, and initialize the weights based on the sample entropy; Specifically, the initial aggregation weights are determined based on the entropy values, and the quotient of the sum of the entropies of each class of samples and the sum of the entropies of all samples is used as the initial weighting weights. This allows us to initialize and obtain an initial fusion weight vector with physical meaning. The formula for calculating the weighted weight Wi for different frequency bands at each measurement point is as follows: (13); Where SE is the sample entropy of each component, K1 is the number of low-frequency components, K2 is the number of mid-frequency components, K3 is the number of high-frequency components, and i is the measurement point number, which is 1 to 13 in this embodiment. The final weight W is a weight matrix of size [13,3].

[0076] This step is determined by the sample entropy characteristics of each component (low frequency, medium frequency, and high frequency) calculated in step S14 of the data preprocessing stage. Sample entropy quantifies the complexity and irregularity of the sequence, providing a more reasonable starting point for the model and helping to accelerate the convergence of the subsequent optimization process.

[0077] Step S32: Based on the current weights, perform weighted fusion on the current predicted components to generate the first aggregated predicted value; Specifically, the prediction results of the three components output by the current model , respectively with the corresponding current training weights Multiply the results and perform a weighted summation to obtain the first aggregated prediction value: ; Step S33: Perform weighted fusion on the target output data to generate the second aggregated true value; The low-frequency, mid-frequency, and high-frequency true target values ​​in the training data The aggregated values ​​are then weighted and merged using the same weighting strategy to obtain the second aggregation result (the true target value): ; Step S34: Based on the first aggregated predicted value and the second aggregated true value, calculate the loss function to perform error calculation and obtain the predicted loss value under the current training round; Specifically, the loss function can be the mean squared error (MSE), and the formula is as follows: ; in, and These are the i-th true value and the i-th predicted value, respectively, where n is the number of samples, which is 192 in this embodiment.

[0078] Step S35: Calculate the gradient based on the predicted loss value through backpropagation: Using the backpropagation algorithm, calculate the gradient of the loss function with respect to all learnable parameters of the model (including encoder, decoder weights, and fusion weights). The gradients of are calculated to form a gradient set.

[0079] Step S36: Based on the calculated gradient, use gradient descent (such as Adam optimizer) to iteratively update all parameters to obtain the updated model parameters and weights in the current training round. Execute step S32 until the iteration cutoff condition is met to obtain the trained encoder and decoder networks and the trained weights.

[0080] Specifically, using the backpropagation algorithm and the chain rule from calculus, the gradient of the loss function with respect to each weight parameter is calculated layer by layer, starting from the output of the loss function. After calculating the gradients of all parameters, the Adam optimizer uses these gradients to actually update the model's parameters. Through this end-to-end learning mechanism, the model in this patent can automatically adjust and learn the optimal component fusion weight W that best reduces the total loss based on the overall prediction objective, making the loss convergence strategy highly adapted to the prediction task.

[0081] During the training of the above model, the weight parameters Like all other parameters of the model (such as attention weights and biases in the encoder-decoder pair), it is considered a trainable hyperparameter. Its optimization process is as follows: In each training iteration, the final predicted value after weighted aggregation is compared with the true target value using a selected loss function (such as mean squared error, MSE). Then, the gradient of the loss function with respect to all weight parameters is calculated using backpropagation, and finally, gradient descent is used for updating. Through this end-to-end learning mechanism, the model can automatically adjust and learn the optimal component fusion weights that best reduce the total loss based on the overall prediction objective, making the aggregation strategy highly adaptable to the prediction task.

[0082] To obtain a model with strong generalization ability and prevent overfitting, a separate validation set is needed for continuous monitoring and tuning.

[0083] A further technical solution involves monitoring and optimizing model performance based on the validation set, implementing iterative validation and early stopping mechanisms. Specifically, after each (or several) training iterations, the current model is forward-propagated onto the validation dataset to calculate its loss function value and other performance metrics (such as mean squared error, MSE) on the validation set. This process is used to monitor the model's performance on unseen data.

[0084] An early stopping mechanism is introduced: if the validation set loss no longer decreases or even begins to increase within several consecutive iterations, the model is determined to be overfitted, training is terminated early, and the model parameters are rolled back to the snapshot with the best performance on the validation set.

[0085] After the model training and tuning are completed, the final "best model" is evaluated fairly and objectively using a test set that has not participated in any training or tuning process.

[0086] Furthermore, based on the final model evaluation of the test set, the best trained model is used to predict loads on the test set, and the predicted results are compared with the true values. Evaluation should not rely solely on a single accuracy metric, but should employ a comprehensive evaluation metric system, including: Root Mean Square Error (RMSE); Mean Absolute Error (MAE); Mean Absolute Percentage Error (MAPE); and Accuracy Calculation (AC). Root Mean Square Error (RMSE): (14); Mean Absolute Error (MAE): (15); Mean Absolute Percentage Error (MAPE): (16); Accuracy (AC): (17); in, and These are the i-th true value and the i-th predicted value, respectively, where n is the number of samples.

[0087] In this embodiment, when aggregating the prediction results of the three components, the weights are not pre-set or easily learnable; their initial values ​​are determined by the sample entropy of each component. Sample entropy quantifies the complexity of the component sequence, giving the weight initialization a clear physical meaning, potentially accelerating model convergence and improving interpretability. Ultimately, these weights, along with other parameters of the entire model, are optimized end-to-end using gradient descent based on the loss signal generated by comparing predicted and true values. This means the model can adaptively learn the optimal way to aggregate the prediction results of the three components according to the specific prediction task, thereby significantly improving the accuracy of the final prediction.

[0088] To illustrate the effectiveness of this embodiment, a prediction test was conducted; The data used is a dimensionally processed and anonymized power load dataset from a region in northern China. The time range is approximately 14 months, from May 1, 2024 to June 23, 2025, with 96 data points collected per day at 15-minute intervals.

[0089] The power load forecasting model selected 14 operational measurement points (including the date column). In addition to the target measurement point load, there are 9 time measurement points and 4 meteorological data measurement points. The specific details of the measurement points are shown in Table 1 below: Table 1. Measurement Points for the Power Load Forecasting Business Model;

[0090] The power load dataset was tested using four commonly used prediction models—Informer, Crossformer, Autoformer, and TimeXer—along with the prediction model of this patent. Four evaluation metrics were obtained. The dataset was divided using two different methods, with the training, validation, and testing ratios being 0.7:0.2:0.1 and 0.8:0.15:0.05, respectively. The test results are shown in Table 2. Table 2 shows the test results of all prediction models;

[0091] As shown in Table 2, the method in this embodiment outperforms the four models mentioned above in all four metrics, demonstrating excellent prediction performance with high accuracy and stability. This embodiment achieves an AC accuracy of 99.34% on the power load dataset II, and the prediction effect is as follows... Figure 3 As shown.

[0092] Figure 3 The horizontal axis represents the time step number (representing the time sequence of the prediction samples, from time point 1 to time point 192); the vertical axis represents the power load value, in megawatts (MW), with a value range of approximately 17,000 MW to 19,000 MW; solid lines represent actual load data (trues), and dashed lines represent model predictions (preds). Figure 3As can be seen, the overall trend of the model prediction values ​​in this embodiment is highly consistent with the actual load, accurately capturing the overall load change trend, periodic fluctuations, and local inflection points. In different load level ranges (such as the early high load segment, the middle low segment, and the later rising segment), the prediction curves closely fit the actual values ​​with almost no significant deviation. At locations with sudden load changes or drastic fluctuations, such as time steps 60-70 and 130-150, the prediction results still maintain good responsiveness and tracking accuracy, demonstrating the model's ability to model high-frequency disturbances. The overall curves exhibit low error and high fitting characteristics, further verifying the effectiveness of the multi-frequency component decoupling and fusion mechanism in this model structure. Figure 3 The experimental results shown fully demonstrate the significant advantages of the prediction model proposed in this embodiment in multi-scale feature modeling, component collaborative fusion and end-to-end learning mechanism, and have good application prospects and industrial value.

[0093] Example 2 Based on Example 1, this example provides a power load forecasting system based on time-series decomposition and attention mechanisms, including: The multi-scale time series decomposition module is configured to perform adaptive time series decomposition on the acquired original load sequence, calculate the sample entropy of each decomposed component and perform clustering to obtain low-frequency data, mid-frequency data and high-frequency data. The model prediction module is configured to construct a set of encoder and decoder networks for each of the obtained low-frequency, mid-frequency and high-frequency data, perform parallel encoding to extract features, and then serially decode and reconstruct the features from low frequency to high frequency, outputting the prediction data from low frequency to high frequency step by step. The adaptive weighted fusion module is configured to initialize weights based on sample entropy and optimize the trained weights during the encoder and decoder network training process. The trained weights are then used to weight and fuse the prediction data from low frequency to high frequency to obtain the predicted value of power load.

[0094] It should be noted that each module in this embodiment corresponds one-to-one with each step in embodiment 1, and their specific implementation process is the same, so it will not be repeated here.

[0095] Example 3 Based on Embodiment 1, this embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, they complete the steps in the power load forecasting method based on time-series decomposition and attention mechanism described in Embodiment 1.

[0096] Example 4 Based on Embodiment 1, this embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, they complete the steps in the power load forecasting method based on time-series decomposition and attention mechanism described in the embodiment.

[0097] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0098] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A power load forecasting method based on time-series decomposition and attention mechanism, characterized in that, Includes the following steps: The original load sequence is adaptively decomposed into time series, and the sample entropy of each component after decomposition is calculated and clustered to obtain low-frequency data, medium-frequency data and high-frequency data. For each of the obtained low-frequency, mid-frequency, and high-frequency data, an encoder and decoder network is constructed. After parallel encoding to extract features, the features are serially decoded and reconstructed from low frequency to high frequency, and the predicted data from low frequency to high frequency is output step by step. The weights are initialized based on sample entropy and optimized during the training of the encoder and decoder networks to obtain the trained weights. The trained weights are then used to weight and fuse the prediction data from low frequency to high frequency to obtain the predicted value of power load.

2. The power load forecasting method based on time-series decomposition and attention mechanism as described in claim 1, characterized in that, The method for adaptive time-series decomposition of the acquired raw load sequence, and clustering the decomposed components to obtain low-frequency, mid-frequency, and high-frequency information, includes the following steps: For each univariate measurement point time series data in the original load sequence, the fully adaptive noise ensemble empirical mode decomposition algorithm is used to perform adaptive decomposition and obtain multiple IMF components; Calculate the sample entropy for each IMF component; Cluster the sample entropy values ​​of all IMF components of all obtained measurement point sequence data; After clustering, all IMF components belonging to the same category are superimposed, and each category forms a one-dimensional time series, namely low-frequency data, medium-frequency data, and high-frequency data.

3. The power load forecasting method based on time-series decomposition and attention mechanism as described in claim 1, characterized in that: The process of generating prediction data includes the following steps: A data embedding layer is constructed to embed the input low-frequency, mid-frequency, and high-frequency data, resulting in low-frequency embedding matrices, mid-frequency embedding matrices, and high-frequency embedding matrices, respectively. A hierarchical encoder stack is constructed for feature mining. For the low-frequency embedding matrix, the mid-frequency embedding matrix, and the high-frequency embedding matrix, parallel feature extraction is performed on the corresponding encoders to obtain the low-frequency feature matrix, the mid-frequency feature matrix, and the high-frequency feature matrix, respectively. A hierarchical decoder stack is constructed to generate prediction components. For each encoder, a decoder is set up, and the corresponding low-frequency decoders are connected in sequence to realize serial decoding from low-frequency features to high-frequency features.

4. The power load forecasting method based on time-series decomposition and attention mechanism as described in claim 3, characterized in that: Data embedding processing is performed on low-frequency, mid-frequency, and high-frequency data respectively, including: The scalar values ​​of each variable in the low-frequency, mid-frequency, and high-frequency data are input into a trainable linear projection layer and mapped to the corresponding high-dimensional vector representation, which serves as the data embedding vector. For each time step of low-frequency, medium-frequency, and high-frequency data, a position encoding scheme composed of sine and cosine functions of different frequencies is used to generate a corresponding time position information vector, which serves as the position embedding vector. The numerical embedding vector and the position embedding vector at the same time step are added element by element to obtain the initial high-dimensional representations of the low-frequency, mid-frequency, and high-frequency data, namely the embedding matrices.

5. The power load forecasting method based on time-series decomposition and attention mechanism as described in claim 3, characterized in that: Based on the constructed hierarchical decoder stack, the features extracted by the encoder are serially decoded and reconstructed, including the following steps: Starting with the decoding of low-frequency features, the first decoder uses the low-frequency feature matrix output by the first encoder connected to it as context and the initialized position code as query to perform decoding and reconstruction, generating preliminary low-frequency component prediction results that focus on long-term trends. The second decoder uses the low-frequency component prediction result output by the first decoder as the query and the mid-frequency feature matrix output by the corresponding second encoder as the context to perform decoding and reconstruction, generating a mid-frequency component prediction result with periodic fluctuation information superimposed on the trend. The third decoder uses the intermediate frequency component prediction result output by the second decoder as the query and the intermediate frequency feature matrix output by the corresponding third encoder as the context to perform decoding and reconstruction, generating a high frequency component prediction result containing residual information such as random fluctuations.

6. The power load forecasting method based on time-series decomposition and attention mechanism as described in claim 1, characterized in that: The process of training the encoder and decoder networks includes the following steps: Historical power load sequence data is acquired, adaptive time series decomposition is performed, sample entropy is calculated for each decomposed component and clustered to obtain low-frequency information, medium-frequency information and high-frequency information to construct a dataset. The low-frequency, mid-frequency, and high-frequency data from the first segment of the time-segmented data are used as input data, and the low-frequency, mid-frequency, and high-frequency data from the second segment of the data are used as target output data. These data are input to the encoder and decoder networks, where features are extracted in parallel. The features are then serially decoded and reconstructed from low frequency to high frequency, and the predicted data from low frequency to high frequency is output step by step. Based on the current weights, the prediction data from low frequency to high frequency is aggregated to obtain the first aggregation result. Based on the current weights, the target output data of the model is aggregated to obtain the second aggregation result. The loss value is calculated based on the first aggregation result and the second aggregation result. Gradient descent is used for iterative updates to obtain the final trained encoder and decoder networks and the trained weights.

7. The power load forecasting method based on time-series decomposition and attention mechanism as described in claim 6, characterized in that: Based on the current weights, the predicted data from low frequency to high frequency is aggregated to obtain the first aggregation result. Based on the current weights, the target output data of the model is aggregated to obtain the second aggregation result. The loss value is calculated based on the first and second aggregation results, and gradient descent is used for iterative updates to obtain the final trained encoder and decoder networks and trained weights. The process includes the following steps: Step S31: Obtain the sample entropy corresponding to the prediction data from low frequency to high frequency, and initialize the weights based on the sample entropy; Step S32: Based on the current weights, perform weighted fusion on the current predicted components to generate the first aggregated predicted value; Step S33: Perform weighted fusion on the target output data to generate the second aggregated true value; Step S34: Based on the first aggregated predicted value and the second aggregated true value, calculate the loss function to perform error calculation and obtain the predicted loss value under the current training round; Step S35: Calculate the gradient based on the predicted loss value through backpropagation: Calculate the gradient of the loss function with respect to all learnable parameters of the model using the backpropagation algorithm. Step S36: Based on the calculated gradient, use gradient descent to iteratively update all parameters to obtain the updated model parameters and weights for the current training round. Execute step S32 until the iteration cutoff condition is met to obtain the trained encoder and decoder networks and the trained weights.

8. A power load forecasting system based on time-series decomposition and attention mechanism, characterized in that, include: The multi-scale time series decomposition module is configured to perform adaptive time series decomposition on the acquired original load sequence, calculate the sample entropy of each decomposed component and perform clustering to obtain low-frequency data, mid-frequency data and high-frequency data. The model prediction module is configured to construct a set of encoder and decoder networks for each of the obtained low-frequency, mid-frequency and high-frequency data, perform parallel encoding to extract features, and then serially decode and reconstruct the features from low frequency to high frequency, outputting the prediction data from low frequency to high frequency step by step. The adaptive weighted fusion module is configured to initialize weights based on sample entropy and optimize the trained weights during the encoder and decoder network training process. The trained weights are then used to weight and fuse the prediction data from low frequency to high frequency to obtain the predicted value of power load.

9. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the steps in the power load forecasting method based on time-series decomposition and attention mechanism as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, complete the steps in the power load forecasting method based on time-series decomposition and attention mechanism as described in any one of claims 1-7.

Images