Power load prediction method and system based on multi-modal and multi-scale feature fusion

Abstract

The invention provides a multi-modal multi-scale feature fusion-based power load prediction method and system, and relates to the technical field of power load prediction. According to the method, a cross-modal attention mechanism is adopted to fuse multi-scale load time sequence features and satellite image features; decoding the fusion features through a multi-step prediction decoder to obtain a multi-step power load prediction result; the acquisition mode of the satellite image features comprises the following steps: performing feature extraction on a satellite image based on a GRVT model to obtain the satellite image features; the GRVT model includes a number of stacked gated residual encoder network elements. According to the method, satellite image feature extraction is carried out based on the GRVT model, the global information processing advantage of Transform is utilized, the single-layer information retaining capability of a residual connection mechanism is also utilized, the two are combined to efficiently extract satellite image features, the satellite image features are fused during power load prediction, and the situation that the data type is single, and the prediction efficiency is high is avoided. And the precision of power load prediction is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power load forecasting, and in particular to a method and system for power load forecasting by fusion of multi-modal and multi-scale features. Background Art

[0002] Power load forecasting is an important part of power system planning and the basis for the economic operation of power systems. It is extremely important for both power system planning and operation.

[0003] The existing power load forecasting is mainly carried out through historical load data and numerical meteorological data, such as CN118332415A, a power system load forecasting method and system based on DBO-SVR algorithm, in 2024.07.12, the parameters with the greatest correlation with the power load are selected from the parameters related to the power system load, and a training set is established; and a support vector regression model is constructed based on the training set; the hyperparameters of the support vector regression model are determined by the dung beetle optimization algorithm; the global optimal position of the dung beetle population is updated through iterative optimization of the support vector regression model, and the optimized support vector regression model is obtained for load forecasting. CN118333225A, a power load forecasting method, device and electronic device based on deep learning, in 2024.07.12, a load forecasting method based on deep learning is proposed, the original power load data is decomposed based on the VMD algorithm to obtain multiple reconstructed sub-modal data, the reconstructed data is divided into data to obtain a training set, a validation set and a test set, and the training set and the validation set are input into the Crossformer network model for training to obtain a power load forecasting model.

[0004] From the above description, it can be seen that the existing power load forecasting mainly relies on historical load data and numerical meteorological data, the data type is relatively simple and the forecasting accuracy is low. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the deficiencies in the prior art, the present invention provides a method and system for power load forecasting by fusion of multi-modal and multi-scale features, which solves the technical problem of low prediction accuracy of the existing power load forecasting methods.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] In a first aspect, the present invention provides a method for power load forecasting based on multi-modal and multi-scale feature fusion, comprising:

[0010] S1. Obtain a historical load sequence and decompose the historical load sequence to obtain a component sequence set including a trend component, a periodic component, and a high-frequency component;

[0011] S2, using a multi-scale Transformer model to extract multi-scale load time series features for each component in the component sequence set;

[0012] S3, using the cross-modal attention mechanism to fuse the multi-scale load time series features and satellite image features to obtain fused features;

[0013] S4, decoding the fused features through a multi-step prediction decoder to obtain a multi-step power load prediction result;

[0014] The satellite image features are obtained by extracting features from the satellite image based on a GRVT model to obtain the satellite image features; the GRVT model includes a plurality of stacked gated residual encoder network units.

[0015] Preferably, the decomposing the historical load sequence includes:

[0016] The historical load series are decomposed using the CEEMDAN algorithm.

[0017] Preferably, the extracting multi-scale load time series features of each component in the component sequence set by using a multi-scale Transformer model includes:

[0018] The multi-scale Transformer model includes a patch segmentation unit, n multi-head self-attention modules and a fusion module; wherein,

[0019] The patch segmentation unit segments each component in the component sequence set into a component set of n patches of different sizes without overlapping;

[0020] n multi-head self-attention modules process the component sets of n patches of different sizes and output n single-scale features;

[0021] The fusion module fuses n single-scale features to obtain multi-scale load time series features.

[0022] Preferably, the multi-scale Transformer model includes several fully connected layers, and SwiGLU is used as the activation function in the fully connected layers. The expression of the SwiGLU activation function is as follows:

[0023] FFN SwiGLU =SwiGLU(X input ,W1,V)W2

[0024] SwiGLU(X input ,W1,V)W2=Swish β (X input W1X input V)W2

[0025] Swish β (X input )=xσ(βX input )

[0026] Among them, W1 and V are the two weight matrices of the SwiGLU module, and W2 is the second-layer fully connected weight matrix of the original FFN. represents the bit-by-bit multiplication of the Hadamard product, Swish is the activation function, β is a learnable parameter of the Swish activation function, and when β=1, it is equivalent to the SiLU activation function. input is the input data of the fully connected layer.

[0027] Preferably, the GRVT model further includes a patch unit, a linear projection + position encoding unit and a fully connected layer;

[0028] Among them, the patch unit divides the satellite image into multiple patches;

[0029] The linear projection + position encoding unit linearly projects multiple patches and then performs position encoding to obtain the input features of the gated residual encoder;

[0030] After several stacked gated residual encoders process the input features, the encoded features that fuse shallow information and deep information are obtained.

[0031] The encoded features are passed through a fully connected layer to output satellite image features.

[0032] Preferably, a cross-modal attention mechanism is used to fuse multi-scale load time series features and satellite image features to obtain fused features, including:

[0033] Multi-scale load time series characteristics F tem After a learnable weight matrix, the output is regarded as the query vector Q tem ,include:

[0034] Q tem =W tem F tem

[0035] Satellite image features F spa Linearly transformed by a learnable weight matrix to generate two sets of outputs: the key vector K spa Sum value vector V spa ,include:

[0036] K spa =W spa_k F spa

[0037] V spa =W spa_v F spa

[0038] Use the dot product operation to calculate the time series query vector Q tem With each image key vector K spa The similarity between the two is obtained by scaling the similarity value and applying the softmax function for normalization to obtain the attention score, including:

[0039]

[0040] Using the attention score, the value vector V of the image spa Perform weighted summation to obtain the fusion feature Fusion, including:

[0041] Fusion=Score·V spa .

[0042] Preferably, the multi-step prediction decoder includes z alternately stacked decoder units and a z+1th fully connected layer, each decoder integrates the same layer normalization, multi-head self-attention mechanism and a fully connected layer, and the features output by the fully connected layer of the last decoder unit are then passed through the z+1th fully connected layer to output the multi-step power load prediction result, where z is a positive integer, z≥2.

[0043] In a second aspect, the present invention provides a multi-modal and multi-scale feature fusion power load forecasting system, comprising:

[0044] A sequence decomposition module is used to obtain the historical load sequence and decompose the historical load sequence to obtain a component sequence set including trend component, periodic component and high-frequency component;

[0045] A feature extraction module is used to extract multi-scale load time series features from each component in the component sequence set using a multi-scale Transformer model;

[0046] The feature fusion module is used to fuse the multi-scale load time series features and satellite image features using a cross-modal attention mechanism to obtain fused features;

[0047] A decoding prediction module is used to decode the fusion features through a multi-step prediction decoder to obtain a multi-step power load prediction result;

[0048] The satellite image features are obtained by extracting features from the satellite image based on a GRVT model to obtain the satellite image features; the GRVT model includes a plurality of stacked gated residual encoder network units.

[0049] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program for power load forecasting based on multi-modal and multi-scale feature fusion, wherein the computer program enables a computer to execute the power load forecasting method based on multi-modal and multi-scale feature fusion as described above.

[0050] In a fourth aspect, the present invention provides an electronic device, comprising:

[0051] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for performing a power load forecasting method using multimodal and multi-scale feature fusion as described above.

[0052] (III) Beneficial effects

[0053] The present invention provides a method and system for power load forecasting based on multi-modal and multi-scale feature fusion. Compared with the prior art, it has the following beneficial effects:

[0054] The present invention first obtains the historical load sequence and decomposes the historical load sequence to obtain a component sequence set including trend components, periodic components, and high-frequency components; then uses a multi-scale Transformer model to extract multi-scale load time series features for each component in the component sequence set; uses a cross-modal attention mechanism to fuse the multi-scale load time series features and satellite image features to obtain fused features; finally, decodes the fused features through a multi-step prediction decoder to obtain a multi-step power load forecast result; wherein, the satellite image feature acquisition method includes: extracting features from satellite images based on a GRVT model to obtain satellite image features; the GRVT model includes a number of stacked gated residual encoder network units. The present invention extracts satellite image features based on the GRVT model, which not only utilizes the advantages of the Transformer in processing global information, but also utilizes the ability of the residual connection mechanism to retain single-layer information, and combines the two to efficiently extract satellite image features, so as to integrate satellite image features into power load forecasting, avoid the single data type, and improve the accuracy of power load forecasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 The block diagram of the power load forecasting method based on multi-modal and multi-scale feature fusion;

[0057] Figure 2 A flow chart of the power load forecasting method based on multi-modal and multi-scale feature fusion;

[0058] Figure 3 This is a schematic diagram of the structure of the multi-head attention module in the multi-scale Transformer model;

[0059] Figure 4 It is the structural diagram of GRVT model;

[0060] Figure 5 Schematic diagram of the structure of the multi-step prediction decoder. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] The embodiments of the present application provide a method and system for power load forecasting that integrates multi-modal and multi-scale features, thereby solving the technical problem of low prediction accuracy of existing power load forecasting methods, and integrating satellite image features into power load forecasting to improve prediction accuracy.

[0063] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0064] Existing power load forecasting methods mainly rely on historical load data and numerical meteorological data, while ignoring the potential value of visual information such as satellite cloud images. At the same time, existing literature generally uses traditional deep learning models such as LSTM and TCN when processing time series data, and has not fully utilized the advantages of Transformer and its improved models in sequence modeling. In addition, end-to-end multimodal technology has not been widely used in the field of power load forecasting, which limits the ability of forecasting models to integrate multiple data types to improve forecasting accuracy.

[0065] To solve the above problems, an embodiment of the present invention proposes an innovative power load forecasting method, which integrates a gated residual vision transformer (GRVT), a multi-scale transformer model (MST) and multimodal feature fusion technology. In view of the outstanding performance of Transformer in the fields of time series analysis and image signal processing, the embodiment of the present invention is committed to exploring how to effectively fuse the inherent characteristics of these two data modalities to optimize the accuracy of time series prediction. Specifically, the embodiment of the present invention first performs feature extraction operations on satellite images and designs a novel network architecture-GRVT, which can efficiently integrate shallow and deep feature information in neural networks. In addition, in order to further enhance the network's nonlinear representation ability of complex data patterns, this study uses SwiGLU instead of the traditional ReLU activation function, aiming to enhance the nonlinear mapping ability of negative weights, thereby achieving more refined feature aggregation. Furthermore, an embodiment of the present invention implements an MST model to capture the multi-scale characteristics of historical power load data, and then organically combines the satellite image data processed by GRVT with the historical power load data processed by MST through a multimodal fusion strategy, and finally outputs an accurate prediction value of the power load.

[0066] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0067] The present invention provides a method for power load forecasting based on multi-modal and multi-scale feature fusion, such as Figure 1 As shown, including:

[0068] S1. Obtain a historical load sequence and decompose the historical load sequence to obtain a component sequence set including a trend component, a periodic component, and a high-frequency component;

[0069] S2, using a multi-scale Transformer model to extract multi-scale load time series features for each component in the component sequence set;

[0070] S3, using the cross-modal attention mechanism to fuse the multi-scale load time series features and satellite image features to obtain fused features;

[0071] S4, decoding the fused features through a multi-step prediction decoder to obtain a multi-step power load prediction result;

[0072] The satellite image features are obtained by extracting features from the satellite image based on a GRVT model to obtain the satellite image features; the GRVT model includes a plurality of stacked gated residual encoder network units.

[0073] The embodiment of the present invention extracts satellite image features based on the GRVT model, which not only utilizes the advantage of Transformer in processing global information, but also utilizes the ability of the residual connection mechanism to retain single-layer information. The two are combined to efficiently extract satellite image features, so that satellite image features can be integrated into power load forecasting, avoiding the single data type and improving the accuracy of power load forecasting.

[0074] The following is combined with Figure 2 The flowchart shown is described in detail:

[0075] In one embodiment, S1, a historical load sequence is obtained, and the historical load sequence is decomposed to obtain a component sequence set including a trend component, a periodic component, and a high-frequency component. The specific implementation process is as follows:

[0076] The power load exhibits a multi-frequency coexistence behavior pattern. Specifically, in the typical scenario of residential electricity consumption in summer, the operation cycles of various household appliances such as washing machines, air conditioners, refrigerators and routers show significant differences. In view of this complexity, it is particularly important to adopt the technical means of sequence decomposition when forecasting power load. This method aims to subdivide the original power load time series into multiple different components, and then effectively extract and analyze the sequence characteristics at each time scale to improve the accuracy and reliability of the forecast.

[0077] The embodiment of the present invention uses the CEEMDAN (Complete Ensemble Empirical Mode Decomposition with Adaptive Noise) algorithm to decompose the historical load series. The CEEMDAN algorithm adds adaptive noise to the original signal and updates the residual signal and the intrinsic mode function (IMFs) in each iteration, thereby achieving effective signal decomposition. The main steps of the algorithm are as follows:

[0078] 1) Add white noise to the original load sequence to generate a new signal sequence.

[0079] 2) Perform EMD decomposition on each new signal to obtain a series of IMFs.

[0080] 3) Calculate the average of the IMFs of the same order of all new signals to obtain the final IMFs.

[0081] 4) Subtract the obtained IMFs from the original signal to obtain a new residual signal.

[0082] 5) Repeat steps 1-4 until the residual signal meets the stopping criterion.

[0083] The decomposed signal mainly includes the following components, which together constitute the component sequence set X:

[0084] Trend component: The long-term trend component in the power load time series helps to analyze the long-term changes in load data.

[0085] Cyclic component: such as seasonal changes in the daily load curve (high in summer and low in winter), differences between weekdays and weekends in the weekly load curve, etc. The high load period in summer is often closely related to the widespread use of refrigeration equipment such as air conditioners.

[0086] High-frequency component: separates the random fluctuation part in the power load time series, which is usually difficult to predict but can be described and simulated to a certain extent through statistical models.

[0087] It should be noted that, in the specific implementation process, the historical load series can also be decomposed by other decomposition algorithms, such as wavelet decomposition algorithm.

[0088] In one embodiment, S2, a multi-scale Transformer model is used to extract multi-scale load time series features from each component in the component sequence set. The specific implementation process is as follows:

[0089] The time series prediction problem aims to predict the most likely future sequence of length O based on a given historical sequence of length I. This process is usually expressed as an "input-I-predict-O" framework. In the long-term prediction scenario, when O increases significantly, the complexity of the problem increases significantly, mainly in two aspects: one is how to effectively process and model complex and changeable time dynamic patterns; the other is how to maximize the use of historical information to break through the bottleneck of information utilization while ensuring computational efficiency.

[0090] To address the above challenges, the embodiment of the present invention incorporates the "patch" concept into the design of the time series prediction model and constructs a network architecture with a multi-scale Transformer as the core.

[0091] The multi-scale Transformer model includes a patch segmentation unit, multiple multi-head self-attention modules, a fusion module, etc. In the embodiment of the present invention, four multi-head self-attention modules are included. The structure of the multi-head attention module is as follows: Figure 3 As shown in Figure 2, the fusion module includes vector concatenation operation, layer normalization, multi-head self-attention mechanism and fully connected layer.

[0092] This architecture not only captures the different frequency information contained in the data from a multi-scale perspective, but also introduces an adaptive segmentation mechanism to achieve flexible and efficient division of data at multiple time granularities, thereby promoting the deep extraction of multi-scale information. In addition, in order to further optimize the nonlinear mapping capability of the model, the embodiment of the present invention selects SwiGLU (Swish-Gated Linear Unit) as the activation function, aiming to improve the expressiveness and generalization ability of the model when processing complex time series data through its unique activation characteristics.

[0093] First, the patch segmentation unit segments each component in the component sequence set X into patches of different sizes without overlapping, representing different time granularities. The patch size set is N = {1, 3, 5, 7} (i.e., the trend component, periodic component, and high-frequency component are divided into 1, 3, 5, and 7, respectively), that is: X p ={X p1 ,X p3 ,X p5 ,X p7}, X p1 Including trend component, period component, high frequency component with size 1, X p3 Including trend component of size 3, periodic component, high frequency component, and so on. The patch is mapped to token through linear projection layer and position encoding. That is, the larger the patch size, the larger the time granularity represented by a single token. The four tokens are input into four multi-head self-attention modules in parallel to extract sequence features at the corresponding scale. Given the input X of the nth MST unit p The specific processing process is as follows:

[0094] S n1 =FFN(MHSA(LN(X p1 )))

[0095] S n3 =FFN(MHSA(LN(X p3 )))

[0096] S n5 =FFN(MHSA(LN(X p5 )))

[0097] S n7 =FFN(MHSA(LN(X p7 )))

[0098] Among them, FFN represents a fully connected layer using SwiGLU as the activation function, and MHSA represents a multi-head self-attention mechanism. The output of the last MST unit is used as the single-scale feature at that scale, expressed as {S1, S3, S5, S7}. The four single-scale features are fused through the cross-scale multi-head self-attention mechanism in the fusion module to fuse features of different scales. The specific process is as follows:

[0099] X can =Concat(S1,S3,S5,S7)

[0100] F tem =FFN(MHSA(LN(X can )))

[0101] Among them, Concat represents the vector concatenation operation, F tem Represents the multi-scale load time series characteristics of the final output of the MST module.

[0102] SwiGLU is introduced as an activation function in each fully connected layer to replace the traditional ReLU function. SwiGLU also follows the basic architecture of combining a fully connected layer with an activation function, but its uniqueness lies in the use of two independent weight matrices to perform different transformations on the input, followed by a Hadamard product operation through the Swish activation function. Given that the FFN (Feed-Forward Network) module has a second fully connected layer embedded in it, the FFN layer that integrates the SwiGLU activation function contains a total of three weight matrices. This design further enriches the parameter space and expression capabilities of the model. The specific process is as follows:

[0103] FFN SwiGLU =SwiGLU(X input ,W1,V)W2

[0104] SwiGLU(X input ,W1,V)W2=Swish β (X input W1X input V)W2

[0105] Swish β (X input )=xσ(βX input )

[0106] Among them, W1 and V are the two weight matrices of the SwiGLU module, and W2 is the second-layer fully connected weight matrix of the original FFN. represents the bit-by-bit multiplication of the Hadamard product, Swish is the activation function, β is a learnable parameter of the Swish activation function, and when β=1, it is equivalent to the SiLU activation function.input It is the input data of FFN layer.

[0107] In one embodiment, S3, a cross-modal attention mechanism is used to fuse multi-scale load time series features and satellite image features to obtain fused features. The specific implementation process is as follows:

[0108] In an embodiment of the present invention, the satellite image features include cloud image meteorological features extracted from satellite cloud images. Compared with numerical weather forecasts, satellite cloud images have the following advantages in extracting meteorological features: satellite cloud images can provide real-time cloud distribution and intuitively display weather phenomena, such as clouds and cyclones, while numerical weather forecasts rely on a large number of calculations and model predictions; satellite cloud images can cover a wide area, and are particularly suitable for monitoring weather conditions in areas with few stations such as oceans and plateaus, while numerical weather forecasts may have limitations in accuracy and data acquisition in these areas; modern meteorological satellites usually have multiple observation channels, which can capture radiation information in different bands, and help identify different types of clouds and atmospheric phenomena, while numerical forecast models rely on physical models and parameterization schemes.

[0109] Therefore, the embodiment of the present invention designs a GRVT model to extract cloud image meteorological features from satellite cloud images, thereby providing important support for power load prediction.

[0110] The GRVT model of the embodiment of the present invention is as follows Figure 4 As shown in Figure 1, it includes a patch unit, a linear projection + position encoding unit, several stacked gated residual encoders, and a fully connected layer. The patch unit divides the satellite image into multiple patches; the linear projection + position encoding unit linearly projects the patch and then performs position encoding to obtain the input features of the gated residual encoder; after several stacked gated residual encoders process the input features, the encoding feature S that integrates shallow information and deep information is obtained. l , the encoding feature S l The satellite image features are output through the fully connected layer. In the embodiment of the present invention, the satellite image is a satellite cloud image, and the satellite image features are cloud image meteorological features. The gated residual encoder network unit is to introduce residual connections in adjacent Transformer encoders.

[0111] In the specific implementation process, a series of stacked gated residual encoder (GRE) network units can simultaneously extract the shallow and deep information of the satellite cloud image. Given the input satellite cloud image I sate ∈R H×W×C , where H×W represents the spatial resolution of the satellite cloud image, and C represents the number of input channels of the image. First, the standard image is divided into multiple patches through a non-overlapping segmentation operation, represented as x p. The size of each patch is P×P×C, and the total number of patches is:

[0112] N patch =(H×W) / (P×P)

[0113] The patch is flattened into a one-dimensional vector of dimension P×P×C and further mapped through a linear projection layer. Since the Transformer encoder itself has no position information, position encoding is needed to add position information to the model. The process of position encoding can be expressed as:

[0114] PE (pos,2i) =sin(pos / 10000 2i / d_model )

[0115] PE (pos,2i+1) =cos(pos / 10000 2i / d_model )

[0116] Among them, pos represents the position of the element in the vector, i represents the dimension index, d_model represents the dimension of the GRVT model, PE (pos,2i) Represents the position encoding result of the 2ith dimension of the word at position pos. Similarly, PE (pos,2i+1)) Represents the position encoding result of the 2i+1th dimension of the word at position pos.

[0117] Feature collapse is a common phenomenon in the ViT architecture. As the model increases in network layers, features extracted from different image blocks lose their discriminability, increase similarity or lose difficult-to-distinguish features. This phenomenon previously appeared as a natural phenomenon of ViT using the attention mechanism, but gradually merged information from multiple image blocks until the entire network layer. In order to avoid feature collapse, the embodiment of the present invention introduces residuals between attention layers. For a single GRE unit, given an input feature X l , the processing process can be described as:

[0118]

[0119] Among them, MHSA represents the multi-head self-attention mechanism, and LN represents layer normalization. The output of each GRE unit not only carries the information of the current processing stage, but also directly serves as the input of the subsequent GRE unit, forming a progressive feature fusion mechanism. Specifically, the first GRE unit follows the standard Transformer encoder processing flow to generate an initial feature representation S0, which not only represents the shallow features of the data, but also serves as the basic residual term of GRVT, aiming to retain and transmit key underlying information. This residual term is regulated by a carefully designed Sigmoid function gating mechanism to dynamically determine the appropriate proportion of deep information to be incorporated in subsequent processing. Subsequently, after each subsequent GRE unit executes the multi-head self-attention mechanism, its processing result will be gated residually connected with the above-mentioned shallow information (ie, the residual term) to output the encoded feature S l . , the final encoding feature S l The meteorological characteristics of the cloud image output by the fully connected layer are represented as F spa This process allows the model to maintain sensitivity and coherence to shallow information while enhancing deep feature learning. That is, the output of the current GRE unit not only enriches the feature representation of the current layer, but also provides a more comprehensive and enhanced input for subsequent GRE units by updating the residual information, which can effectively promote the depth and breadth of the entire model in feature extraction and representation learning.

[0120] It should be noted that, in the specific implementation process, the satellite image features may also include night light features extracted from night light images, etc.

[0121] The introduction of cross-modal attention aims to effectively solve the multi-scale load time series characteristics F tem With cloud map meteorological characteristics F spa The fusion problem in the power load forecasting task, although the two modes are essentially different, they both show indispensable importance in building load forecasting models. Traditional methods often fail to fully explore the intrinsic correlation and complementarity between the two modes at the encoder stage, thus limiting the further improvement of the model prediction accuracy.

[0122] In view of this, the embodiment of the present invention proposes a cross-modal attention mechanism, which aims to strengthen the information interaction and integration between time series and image features through a novel fusion strategy. The mechanism can dynamically identify and focus on the most critical information fragments for load forecasting in the two modalities, and through the allocation of attention weights, it can achieve enhanced representation of important information and effective filtering of redundant information.

[0123] The multi-scale load time series feature F tem The input is a learnable weight matrix that is optimized through the training process, and the output is considered as the query vector Qtem , which will be used in the subsequent interaction process with image features:

[0124] Q tem =W tem F tem

[0125] Cloud Atlas Meteorological Features spa Linearly transformed by a learnable weight matrix to generate two sets of outputs: the key vector K spa Sum value vector V spa These two sets of vectors are used for subsequent attention score calculation and final output generation respectively:

[0126] K spa =W spa_k F spa

[0127] V spa =W spa_v F spa

[0128] Use the dot product operation to calculate the time series query vector Q tem With each image key vector K spa This step essentially measures the correlation between a certain time point in the time series and a specific region or feature in the image. The obtained similarity value (i.e., the dot product result) is normalized by the softmax function after scaling to obtain the attention scores, which reflect the multi-scale load time series features F tem For cloud image meteorological features F spa The degree of attention paid to different parts of the

[0129]

[0130] Using the attention score calculated in the previous step, the value vector V of the image spa This process is essentially to extract and integrate relevant information from image features according to the query requirements of the time series.

[0131] Fusion=Score·V spa

[0132] The result of the weighted summation, the fusion feature, is the final output of the cross-modal attention mechanism. This output not only contains the dynamic information of the time series, but also integrates the key visual information in the image, providing a richer and more comprehensive feature representation for subsequent prediction or decision-making tasks.

[0133] In one embodiment, S4, the fusion feature is decoded by a multi-step prediction decoder to obtain a multi-step power load prediction result. The specific implementation process is as follows:

[0134] When processing multi-step prediction tasks, an embodiment of the present invention designs a standardized decoder architecture. In view of the fact that the effectiveness of the start token in the dynamic decoding process has been verified, the present invention innovatively introduces an extended power load embedding sequence, namely X_token, as the starting signal of the decoding process. This start symbol is intended to characterize the historical segment of the power load before the prediction sequence, thereby giving the model the ability to capture and identify potential trends in the power load. Subsequently, the carefully designed fusion feature Fusion is input into the alternately stacked decoder units, each decoder standardizedly integrates the same layer normalization, multi-head self-attention (MHSA) mechanism and a fully connected layer (FFN) layer; multiple final decoders are connected in series, and the output of the last decoder is connected to a fully connected layer to output the multi-step power load prediction result Y. The structure of the multi-step prediction decoder in an embodiment of the present invention is as follows Figure 5 It should be noted that SwiGLU is also used as the activation function in the fully connected layer of the multi-step prediction decoder.

[0135] The present invention provides a multi-modal and multi-scale feature fusion power load forecasting system, comprising:

[0136] A sequence decomposition module is used to obtain the historical load sequence and decompose the historical load sequence to obtain a component sequence set including trend component, periodic component and high-frequency component;

[0137] A feature extraction module is used to extract multi-scale load time series features from each component in the component sequence set using a multi-scale Transformer model;

[0138] The feature fusion module is used to fuse the multi-scale load time series features and satellite image features using a cross-modal attention mechanism to obtain fused features;

[0139] A decoding prediction module is used to decode the fusion features through a multi-step prediction decoder to obtain a multi-step power load prediction result;

[0140] The satellite image features are obtained by extracting features from the satellite image based on a GRVT model to obtain the satellite image features; the GRVT model includes a plurality of stacked gated residual encoder network units.

[0141] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0142] 1. The embodiment of the present invention extracts satellite image features based on the GRVT model, which not only utilizes the advantage of Transformer in processing global information, but also utilizes the ability of the residual connection mechanism to retain single-layer information. The two are combined to efficiently extract satellite image features, so that satellite image features can be integrated into power load forecasting, avoiding a single data type and improving the accuracy of power load forecasting.

[0143] 2. The embodiment of the present invention uses SwiGLU's MST to extract the time series characteristics of the power load sequence, which can not only comprehensively characterize the change information of the load data on different time scales, but also significantly enhance the nonlinear mapping capability of the network structure, thereby achieving more refined feature aggregation and laying the foundation for accurate prediction of power load.

[0144] 3. The multimodal attention in the embodiment of the present invention is used to fuse temporal features and image features, which can not only identify and enhance the relevant information between different modalities, but also maintain the uniqueness and integrity of each modal feature during the fusion process, and fully explore the intrinsic correlation and complementarity between these various modalities, thereby further improving the prediction accuracy.

[0145] 4. The embodiment of the present invention designs a Transformer structure based on multi-scale fusion, which can adaptively select the time granularity division method corresponding to the modal component, thereby improving the adaptability of the method.

[0146] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0147] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for power load forecasting based on multi-modal and multi-scale feature fusion, characterized in that: include: S1. Obtain a historical load sequence and decompose the historical load sequence to obtain a component sequence set including a trend component, a periodic component, and a high-frequency component; S2, using a multi-scale Transformer model to extract multi-scale load time series features for each component in the component sequence set; S3, using the cross-modal attention mechanism to fuse the multi-scale load time series features and satellite image features to obtain fused features; S4, decoding the fused features through a multi-step prediction decoder to obtain a multi-step power load prediction result; The satellite image features are obtained by extracting features from the satellite image based on a GRVT model to obtain the satellite image features; the GRVT model includes a plurality of stacked gated residual encoder network units.

2. The method for power load forecasting based on multi-modal and multi-scale feature fusion according to claim 1, characterized in that: The decomposing of the historical load sequence includes: The historical load series are decomposed using the CEEMDAN algorithm.

3. The method for power load forecasting based on multi-modal and multi-scale feature fusion according to claim 1, characterized in that: The method of extracting multi-scale load time series features from each component in the component sequence set by using a multi-scale Transformer model includes: The multi-scale Transformer model includes a patch segmentation unit, n multi-head self-attention modules and a fusion module; wherein, The patch segmentation unit segments each component in the component sequence set into a component set of n patches of different sizes without overlapping; n multi-head self-attention modules process the component sets of n patches of different sizes and output n single-scale features; The fusion module fuses n single-scale features to obtain multi-scale load time series features.

4. The method for power load forecasting based on multi-modal and multi-scale feature fusion according to any one of claims 1 to 3, characterized in that: The multi-scale Transformer model includes several fully connected layers, and SwiGLU is used as the activation function in the fully connected layer. The expression of the SwiGLU activation function is as follows: FFN SwiGLU =SwiGLU(X input ,W1,V)W2 Swish β (X input )=xσ(βX input ) Among them, W1 and V are the two weight matrices of the SwiGLU module, and W2 is the second-layer fully connected weight matrix of the original FFN. represents the bit-by-bit multiplication of the Hadamard product, Swish is the activation function, β is a learnable parameter of the Swish activation function, and when β=1, it is equivalent to the SiLU activation function. input is the input data of the fully connected layer.

5. The method for power load forecasting based on multi-modal and multi-scale feature fusion according to any one of claims 1 to 3, characterized in that: The GRVT model also includes a patch unit, a linear projection + position encoding unit and a fully connected layer; Among them, the patch unit divides the satellite image into multiple patches; The linear projection + position encoding unit linearly projects multiple patches and then performs position encoding to obtain the input features of the gated residual encoder; After several stacked gated residual encoders process the input features, the encoded features that fuse shallow information and deep information are obtained. The encoded features are passed through a fully connected layer to output satellite image features.

6. The method for power load forecasting based on multi-modal and multi-scale feature fusion according to any one of claims 1 to 3, characterized in that: The cross-modal attention mechanism is used to fuse the multi-scale load time series features and satellite image features to obtain fused features, including: Multi-scale load time series characteristics F tem After a learnable weight matrix, the output is regarded as the query vector Q tem ,include: Q tem =W tem F tem Satellite image features F spa Linearly transformed by a learnable weight matrix to generate two sets of outputs: the key vector K spa Sum value vector V spa ,include: K spa =W spa_k F spa V spa =W spa_v F spa Use the dot product operation to calculate the time series query vector Q tem With each image key vector K spa The similarity between them is obtained by scaling the similarity value and normalizing it with the softmax function to obtain the attention score, including: Using the attention score, the value vector V of the image spa Perform weighted summation to obtain the fusion feature Fusion, including: Fusion=Score·V spa 。 7. The method for power load forecasting based on multi-modal and multi-scale feature fusion according to any one of claims 1 to 3, characterized in that: The multi-step prediction decoder includes z alternately stacked decoder units and the z+1th fully connected layer. Each decoder integrates the same layer normalization, multi-head self-attention mechanism and a fully connected layer. The features output by the fully connected layer of the last decoder unit are then passed through the z+1th fully connected layer to output the multi-step power load prediction result, where z is a positive integer, z≥2.

8. A multi-modal and multi-scale feature fusion power load forecasting system, characterized in that: include: A sequence decomposition module is used to obtain the historical load sequence and decompose the historical load sequence to obtain a component sequence set including trend component, periodic component and high-frequency component; A feature extraction module is used to extract multi-scale load time series features from each component in the component sequence set using a multi-scale Transformer model; The feature fusion module is used to fuse the multi-scale load time series features and satellite image features using a cross-modal attention mechanism to obtain fused features; A decoding prediction module is used to decode the fusion features through a multi-step prediction decoder to obtain a multi-step power load prediction result; The satellite image features are obtained by extracting features from the satellite image based on a GRVT model to obtain the satellite image features; the GRVT model includes a plurality of stacked gated residual encoder network units.

9. A computer-readable storage medium, characterized in that: It stores a computer program for power load prediction using multi-modal and multi-scale feature fusion, wherein the computer program enables a computer to execute a power load prediction method using multi-modal and multi-scale feature fusion as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing a multi-modal and multi-scale feature fusion power load forecasting method as described in any one of claims 1 to 7.

Images