A power load prediction decoupling method based on a large language model

Abstract

The application belongs to the technical field of power load prediction, and discloses a power load prediction decoupling method based on a large language model, which decomposes the obtained historical power load data sequence into a trend component, a seasonal component and a residual component; for the trend component and the seasonal component, trend embedding and seasonal embedding are respectively output based on a convolution layer and a linear layer; the residual component is aligned with word features extracted from a prompt word in a high-order semantic structure, input of the large language model is constructed based on the aligned residual component and the word features, and residual connection is performed on output of the large language model to obtain residual embedding; the trend embedding, the seasonal embedding and the residual embedding are fused, and the fused features are input into a prediction head to predict output of power load prediction results of multiple continuous time steps in the future. According to the complexity of each component, a special sub-model is designed to realize more accurate reasoning of different components, so that the long-term prediction accuracy and the generalization are improved.

Description

Technical Field

[0001] This invention belongs to the field of power load forecasting technology, specifically relating to a decoupling method for power load forecasting based on a large language model. Background Technology

[0002] Accurate load forecasting (LF) is crucial in real-world scenarios such as generation planning and power resource dispatch. Due to significant differences in the characteristic distributions of different load datasets, improving the long-term and generalized forecasting capabilities of models has become a key challenge.

[0003] In recent years, Large Language Models (LLMs) have demonstrated exceptional generalization capabilities across various tasks due to their powerful representational abilities. Inspired by this, researchers have proposed several LLM-based prediction models aimed at enhancing the model's ability to capture information from workload data and delve deeper into its potential temporal dependencies, thereby alleviating the bottlenecks of current models in long-term prediction and generalization tasks. For example, GPT2 (Generative Pre-trained Transformer 2) pioneered the use of fine-tuned pre-trained LLMs to perform LF tasks, proving the feasibility of pre-trained LLMs in workload data modeling; TIME-LLM (Time Series Large Language Model) reprograms workload data using cue features before inputting it into the pre-trained LLM, explicitly encoding temporal dependencies and providing the LLM with a temporally structured input.

[0004] To alleviate the problem of semantic space discretization in LLM (Language Modeling), researchers have proposed novel solutions: semantic alignment strategies. For example, S2IP-LLM (Semantic Space Informed Prompt Learning with LLM for TimeSeries Forecasting) aligns payload data with word features of a pre-trained LLM through similarity matching. CALF (Cross-modal Aligning Large language models for time series Forecasting) proposes a cross-modal alignment module and feature regularization loss to perform alignment. CA (Context-Alignment) uses a graph structure to align payload data with language components. However, these LLM-based methods often treat regular and fluctuating components in the original data equally, weakening the model's ability to model the crucial fluctuating components, leading to poor performance in long-term and generalized predictions. Summary of the Invention

[0005] The purpose of this invention is to provide a decoupling method for power load forecasting based on a large language model. A dedicated sub-model is designed according to the complexity of each component to achieve more accurate inference for different components, thereby improving long-term forecast accuracy and generalization.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A decoupling method for power load forecasting based on a large language model includes the following steps:

[0008] The acquired historical power load data series is decomposed into trend components, seasonal components, and residual components;

[0009] For trend components and seasonal components, trend embeddings and seasonal embeddings are output based on convolutional layers and linear layers, respectively.

[0010] The residual components are aligned with the word features extracted from the prompt words using a higher-order semantic structure. The input of the large language model is constructed based on the aligned residual components and word features. The output of the large language model is then subjected to residual concatenation to obtain the residual embedding.

[0011] The algorithm integrates trend embedding, seasonal embedding, and residual embedding, and uses the integrated features to predict the power load forecast results for multiple consecutive time steps in the future through the prediction head.

[0012] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0013] Preferably, trend components and seasonal components are used as input components, and trend embeddings and seasonal embeddings are used as output embeddings. Then, the step of outputting trend embeddings and seasonal embeddings based on convolutional layers and linear layers respectively for the trend components and seasonal components includes:

[0014] The input components are reshaped, and multiple linear layers are used to spatially map the reshaped features. The outputs of all linear layers are combined to obtain the mapped input pattern.

[0015] Multiple one-dimensional convolutional layers are used to process the input components and concatenate their features. Then, a linear layer is used to process the concatenated features to obtain the spatial weights of the input components.

[0016] The output embedding is obtained by integrating the spatial weights and mapped input patterns with the Einstein summation convention and the linear layer.

[0017] Preferably, the trend component and the seasonal component share the same convolutional layer and linear layer.

[0018] Preferably, the step of aligning the residual components with the word features extracted from the prompt words in a higher-order semantic structure includes:

[0019] The residual components are segmented to obtain the residual segmented components;

[0020] A linear layer is used to perform a linear transformation on the residual block components, projecting the residual block components onto a feature space of the same dimension as the word features;

[0021] Normalize the projected residual block components and word features;

[0022] The cosine similarity algorithm is used to calculate the correlation matrix between the normalized residual block components and word features;

[0023] The association matrix, normalized residual block components, and word features are processed by a hypergraph neural network, and the aligned residual components are output.

[0024] Preferably, the input for constructing the large language model based on the aligned residual components and word features includes:

[0025] The word features are used as prefix data, and the aligned residual components are used as suffix data to construct the input of the large language model.

[0026] Preferably, the residual concatenation of the output of the large language model to obtain the residual embedding includes:

[0027] The residual components are passed through a linear layer to obtain transformed features;

[0028] The final transformation features and output of the large language model The residual embedding is obtained by adding the 3D features. This refers to the dimension of power load data.

[0029] Preferably, the prediction head employs a linear layer with a LeakyReLU activation function.

[0030] This invention provides a decoupling method for power load prediction based on a large language model. It designs dedicated sub-models according to the complexity of each component, enabling more accurate inference for different components. Specifically, an LLM (Linguistic Model-based) is used as the backbone network in the fluctuation sub-model. However, the distributional difference between the LLM training corpus and the load data limits the LLM's representation ability for the load data. Therefore, a high-order semantic structure alignment module is designed to align the fluctuation components with the cue word features to a shared structural space, fully releasing the LLM's representation ability for the fluctuation components. For regular components, a lightweight regular sub-model is designed based on convolutional and linear layers to learn the regular components. Through the hierarchical integration of convolutional and linear layers, this sub-model achieves progressive inference from local representation to global representation abstraction, extracting global patterns while retaining key local information. Thanks to the model design combining large and small models, this invention achieves high predictive performance and robust generalization ability in an efficient manner. Attached Figure Description

[0031] Figure 1 This is a framework diagram of the power load forecasting decoupling method based on a large language model according to the present invention;

[0032] Figure 2 The flowchart shows the power load forecasting decoupling method based on a large language model according to the present invention.

[0033] Figure 3 This is a schematic diagram of the high-order semantic structure alignment module of the present invention;

[0034] Figure 4 This is a schematic diagram showing the results of the experiment in this invention to explore the impact of the number of mappings and the learning rate on the prediction performance of DelLM;

[0035] Figure 5 This is a schematic diagram of the efficiency analysis results in the experiment of this invention;

[0036] Figure 6 This is a t-SNE diagram showing the distribution of residual features before and after alignment using a higher-order semantic structure alignment module in the experiments of this invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0039] Current LLM-based forecasting methods often treat the regular and volatile components of the original data equally, weakening the model's ability to model the volatile components, which are crucial for long-term forecasting. For example... Figure 1 As shown, to alleviate this challenge, this invention proposes a decoupling framework based on LLM (DeLLM). This framework designs personalized sub-models according to the differences in component complexity, fully exploring the representations of each component to improve prediction accuracy. First, a fluctuation sub-model composed of LLM and a high-order semantic structure alignment module is designed to understand relatively complex fluctuation components. The alignment module aligns the fluctuation components with the cue word representations to a shared structural space, enhancing the ability of LLM to handle complex fluctuation components. Furthermore, for easily learnable rule components, a lightweight rule sub-model is designed to efficiently and accurately model these rule components. The fusion and prediction module integrates fluctuation features and rule features to generate predicted values. The model design of this invention achieves high prediction performance while maintaining lightweight characteristics.

[0040] An LF task can be described as: given a length of and Data in each dimension Train a parameterized model predict Future in Dimensions Data at each time step This embodiment uses power load forecasting as an example for illustration. In other embodiments, it can also be applied to fields such as trajectory forecasting, temperature forecasting, and weather forecasting.

[0041] like Figure 2 As shown in this embodiment, a power load forecasting decoupling method based on a large language model includes the following steps:

[0042] Step 1: Decompose the acquired historical power load data sequence into trend components, seasonal components, and residual components.

[0043] This embodiment uses the STL (Seasonal and Trend decomposition using Loess) method to decompose the original data into trend, seasonal, and residual components. Based on the varying complexity of each component, this embodiment designs a customized sub-model to learn them. A higher-order semantic structure alignment module aligns the fluctuation components with cue word representations to a shared structural space, enhancing the LLM's ability to learn complex fluctuation components.

[0044] Step 2: For the trend component and the seasonal component, output the trend embedding and seasonal embedding respectively based on the convolutional layer and the linear layer.

[0045] Given the inherent regularity of trend and seasonal components, this embodiment divides each component into multiple segments and directly models the spatial relationships between different segments in each dimension. Specifically, this embodiment designs a lightweight rule sub-model based on linear and convolutional layers to infer trend and seasonal component information from local features to global patterns, and uses the same lightweight rule sub-model to learn the trend component. and seasonal ingredients .

[0046] In general, using trend and seasonal components as input components and trend embeddings and seasonal embeddings as output embeddings, the specific processing steps include: reshaping the input components and using multiple linear layers to spatially map the reshaped features, combining the outputs of all linear layers to obtain the mapped input pattern; using multiple one-dimensional convolutional layers to process the input components and concatenate the features, then using linear layers to process the concatenated features to obtain the spatial weights of the input components; and using the Einstein summation convention to integrate the spatial weights and the mapped input pattern with the linear layers to obtain the output embedding.

[0047] The processing procedure is further illustrated using a lightweight rule submodel to handle seasonal components as an example. First, the process is... Remodeled into a segmented seasonal pattern ,in It is the length of each segment. It is a number of segments. Secondly, it adopts... Each linear layer corresponds to a segmented seasonal pattern. Perform spatial mapping to generate the mapped seasonal pattern. This fully leverages the advantages of reshaping operations in focusing on local features and linear layers in modeling linear relationships. Third, it uses... Each one-dimensional convolutional layer processes the original data. Feature concatenation is then performed. Finally, a linear layer with a softmax activation function is used to learn the spatial weights of the seasonal components. This modeling strategy leverages the advantages of channel independence in mitigating data distribution differences and the ability of convolutional layers to capture local features. Finally, it uses the Einstein Summation Convention. With linear layer Integrating spatial weights and mapped seasonal patterns and output seasonal embeddings As shown below:

[0048] (1)

[0049] Similarly, the trend embedding of the trend components extracted by the lightweight rule submodel is... By combining convolutional and linear layers, DelLM effectively captures the inherent linear and regular characteristics of trend and seasonal components.

[0050] Step 3: Align the residual components with the word features extracted from the prompt words using a higher-order semantic structure. Construct the input of the large language model based on the aligned residual components and word features, and perform residual connections on the output of the large language model to obtain the residual embedding.

[0051] The wave sub-model consists of a high-order semantic structure alignment module and an LLM backbone. The input to the wave sub-model includes residual block components. Word features extracted from prompt words by LLM embeddings ,in It is the length of each block. It is the number of blocks. It's the span. It refers to the size of the vocabulary. This refers to the feature dimension. This example uses the ETTh dataset as an example to provide a prompt word as shown below:

[0052] Dataset description: The Electricity Transformer Temperature (ETT) is a crucial indicator in the electric power long-term deployment. This datasetconsists of 2 years data from two separated counties in China. To explore thegranularity on the long sequence time-series forecasting (LSTF) problem,different subsets are created, {ETTh1, ETTh2} for 1-hour-level and ETTm1 for15-minutes-level. Each data point consists of the target value oiltemperature and 6 power load features. The train / val / test is 12 / 4 / 4 months.

[0053] External description: This is the residual component following STLprocessing, exhibiting significant volatility and determining the amplitude of fluctuations in the original data.

[0054] Task description: forecast the next 96 steps given the previous 720 steps information.

[0055] like Figure 3 As shown in the diagram, the ascending embedding is the representation of the term ascending, which is considered as a hyperedge embedding. The descending embedding is similar. (Segment) Indicates the first Each normalized residual is considered a hypergraph node. This embodiment designs a high-order semantic structure alignment module, which aligns the residual block components. Features of words Semantic information between them enhances LLM's understanding of residual components. Contextual comprehension ability. Specifically, firstly, regarding residual components... Perform block processing to obtain residual block components. Using a linear layer for residual block components A linear transformation is performed to project the data onto a feature space of the same dimension as the word features. Then, the projected residual block components and word features are normalized. Finally, a cosine similarity algorithm is used. The correlation matrix between the normalized residual block components and word features is calculated. Finally, a simplified hypergraph neural network is used. (From the paper "Hypergraph Neural Networks") This module implements information propagation from word features to residual components, thereby aligning the semantic information between residual components and word features. Specifically, it processes the association matrix, normalized residual block components, and word features using a hypergraph neural network, outputting the aligned residual components. The algorithm flow of this module can be represented as follows:

[0056] (2)

[0057] (3)

[0058] in, Represents the correlation matrix; and These represent the normalized residual block components and word features, respectively; This represents the residual components after alignment. This embodiment uses word features... As prefix data, the aligned residual components Inputs for constructing an LLM as suffix data As shown below:

[0059] (4)

[0060] LLM final output .Will Features after input linear layer processing and The end The residual embedding is obtained by adding the features of dimension 1. .

[0061] Step 4: Fuse trend embedding, seasonal embedding, and residual embedding, and pass the fused features through the prediction head to predict the power load forecast results for multiple consecutive time steps in the future.

[0062] After processing through the two sub-models mentioned above, DelLM obtained... , and Based on the principle of decomposition and fusion, the fusion and prediction module first adds the three features, and then outputs the final predicted value through a linear layer with the LeakyReLU activation function. Furthermore, during model training, the parameters of the large speech model are frozen, and the parameters of the lightweight rule sub-model, the high-order semantic structure alignment module, and the fusion and prediction module are iteratively updated. This embodiment uses a hybrid temporal loss function to evaluate the prediction error of DelLM and performs iterative updates, as shown below:

[0063] (5)

[0064] (6)

[0065] in, Indicates the total loss. Indicates average absolute loss. Indicates mean squared loss. The weight vector representing the mean absolute loss. Y represents the activation function, and Y represents the actual label data.

[0066] To verify the effectiveness of the method of the present invention, the following experiments were conducted in this embodiment.

[0067] (1) Experimental setup:

[0068] Datasets: This experiment was conducted on six general time series datasets. For long-term forecasting, the ETT series datasets (ETTh1, ETTh2, ETTm1, ETTm2) were used, along with weather and electricity datasets. For generalized forecasting, four datasets were used: ETTh1, ETTh2, ETTm1, and ETTm2. All datasets are publicly available.

[0069] Baseline Models: This experiment selected representative LTSF models from recent years as the baseline models. These baseline models can be categorized as follows:

[0070] (1) LLM-based models: Cross-Modal LLM Fine-Tuning (CALF; from the paper "CALF: Aligning llms for time series forecasting via cross-modal fine-tuning"), LLM-TPF (from the paper "LLM-TPF: Multiscale Temporal Periodicity-Semantic Fusion LLMs for Time Series Forecasting"), Semantic Space Informed Prompt Learning with LLM (S2IP-LLM; from the paper "S2IP-LLM: Semantic space informed prompt learning with LLM for time series forecasting"), LLMTime (from the paper "Large Language Models Are Zero-Shot Time Series Forecasters"), and Time-LLM (from the paper "Time-LLM: Time Series Forecasting by Reprogramming Large Language Models").

[0071] (2) Transformer-based models: iTransformer (from the paper "iTransformer: Inverted Transformers Are Effective for Time Series Forecasting") and Patch TimeSeries Transformer (PatchTST; from the paper "A Time Series is Worth 64 Words: Long-term Forecasting with Transformers");

[0072] (3) Mamba-based model: TimePro (from the literature "TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state");

[0073] (4) Models based on linear layers: Series-Core Fused Time Series Forecaster (SOFTS; from the literature "SOFTS: Efficient multivariate time series forecasting with series-core fusion") and DLinear (from the literature "Are Transformers Effective for Time Series Forecasting").

[0074] Implementation Details: The experiment utilizes an NVIDIA RTX 3090 graphics processor for hardware acceleration. Following the experimental procedures outlined in the paper "GPT4TS: Generative Pre-trained Transformer for Time Series Forecasting," this experiment uses a pre-trained GPT2-small model (from the paper "Language Models are Unsupervised Multitask Learners") as the LLM in the DelLM model. This model contains 6 hidden layers. The feature dimensions of the DelLM are... D is set to 768, hyperparameter The selection range is {2, 4, 6, 8}, and the selection range for the convolution kernel of a one-dimensional convolutional layer is {4, 8, 12}. The selection range is {12, 24}. Adam is used as the optimizer during training. To ensure the reproducibility of the experiment, the random seed is set to 2021.

[0075] (2) Long-term forecast:

[0076] This section systematically compares the performance of DeLLM and baseline models in long-term prediction. As shown in Table 1, DeLLM demonstrates superior performance in most prediction scenarios, significantly outperforming the baseline model. Specifically, DeLLM achieves the best MSE in 75% (18 / 24) of prediction scenarios and the best MAE in 79% (19 / 24) of prediction scenarios, demonstrating its significant advantage in capturing long-term dependencies. On the ETTm2, Weather, and Electricity datasets, DeLLM achieves comprehensive leadership, performing best in all evaluation metrics. Compared to LLM-TPF, CALF, and S2IP-LLM, DeLLM reduces MSE by an average of 2.1%, 10.4%, and 2.6%, respectively, and MAE by 1.5%, 3.9%, and 3.4%, respectively. These results indicate that DeLLM can capture complex latent long-term dependencies from time-series data, thus achieving more accurate and robust long-term predictions. This provides strong support for its application in real-world scenarios. This experiment was conducted with L=720 and a prediction length of... In all the tables of this experiment, the best result for each scenario is marked in red, and the second-best result is marked in blue.

[0077] Table 1 Long-term forecast results

[0078]

[0079] (3) Generalization prediction:

[0080] Besides long-term prediction, generalization prediction is also a key dimension for evaluating model performance on the LSTF task. Therefore, this experiment performed generalization prediction to compare the generalization ability of DelLM with the baseline model. As shown in Table 2, the generalization predictions in Table 2... Indicates in the dataset Train the model on the dataset and... The test model was then deployed. Experimental results show that in most prediction scenarios, DelLM's generalization ability significantly outperforms the baseline model. Specifically, DelLM exhibits the best performance in both MSE and MAE in 81.3% (13 / 16) of the scenarios. Compared to LLM-TPF, DelLM's MSE and MAE are reduced by an average of 6.6% and 2.3%, respectively; compared to CALF, the average reductions are 9.6% and 3.7%; and compared to S2IP-LLM, the average reductions are 2.5% and 3.4%. These results fully demonstrate DelLM's powerful generalization prediction ability. This is mainly attributed to DelLM's "large + small" model collaborative framework design and the proposed high-order semantic structure alignment module.

[0081] Table 2 Generalization prediction results

[0082]

[0083] (4) Ablation experiment:

[0084] Frame ablation experiments: To verify the rationale for DelLM, frame ablation experiments were conducted on the ETTh1 and ETTh2 datasets, focusing on verifying the impact of the proposed "large + small" model on frame performance. As shown in Table 3, DelLM outperforms its variants in all cases. Specifically, in , and In the case of [unspecified event], the prediction accuracy of the model decreased by an average of 1.7%, 4.4%, and 0.9% compared to DelLM. This result fully demonstrates the rationality of the "large + small" model framework, which helps DelLM understand the regular and volatile components in time series data, thereby achieving better prediction performance. This means replacing B with A. This indicates that the positions of A and B have been interchanged. MSE was selected as the evaluation metric for all ablation experiments. LR and V represent the lightweight regular submodel and the wave submodel, respectively.

[0085] Table 3 Frame Ablation Experiment

[0086]

[0087] Alignment Module Ablation Experiment: To verify the necessity and effectiveness of the higher-order semantic structure alignment module, this study replaced it with different alignment methods for comparison. As shown in Table 4, when the higher-order semantic structure alignment module was replaced with the alignment module of S2IP-LLM, the alignment module of CALF, or the concatenation operation (i.e., CAT), the prediction accuracy of the model decreased by an average of 1.7%, 2.9%, and 2.3% compared with DeLLM, respectively. This indicates that the higher-order semantic structure alignment module designed in this paper helps LLM understand the complex dynamic residual components in time series data, thereby obtaining better prediction performance.

[0088] Table 4 Alignment Module Ablation Experiment

[0089]

[0090] LLM Ablation Experiments: To evaluate the impact of different LLMs on the predictive performance of the proposed framework, ablation experiments were conducted on GPT2-small. Specifically, GPT2-small in the DelLM was replaced with T5-base (from the paper "Exploring the limits of transfer learning with a unified text-to-text transformer"), T5-small (from the paper "Exploring the limits of transfer learning with a unified text-to-text transformer"), and Roberta-base (from the paper "Roberta: A robustly optimized bert pretraining approach"). As shown in Table 5, the results show that when using T5-base, T5-small, and Roberta-base, respectively, the model performance decreased by an average of 1.5%, 1.7%, and 1.2% compared to using GPT2-small. Nevertheless, these three variant models still outperformed LLM-TPF, with average performance improvements of 2.6%, 2.3%, and 2.9%. These results not only indicate that GPT2-small is more suitable for the framework designed in this study, but also demonstrate the rationality and advancement of this framework.

[0091] Table 5 LLM ablation experiment.

[0092]

[0093] Ablation experiments with different hyperparameter values: This study focuses on analyzing and The impact of hyperparameters on DelLM performance. For example... Figure 4 As shown, these two hyperparameters exhibit significant differences across different datasets. On the Weather dataset, when DelLM achieves optimal performance when the value is set to 4. On the ETTm1 dataset, when... DelLM achieves optimal performance when the value is 0.0004. These results indicate that appropriate settings... and This is crucial for realizing the potential of DelLM.

[0094] (5) Efficiency analysis:

[0095] This section comprehensively compares the efficiency of DeLLM and LLM-based baseline models on the ETTh1 dataset using metrics such as model parameters, memory consumption, gigaflops per second (GFLOPs), training time, and MAE. To ensure fairness, both DeLLM and LLM-based baseline models were configured with the same batch size, sequence length, prediction length, and early stopping patience value. Experimental results are as follows: Figure 5 As shown, except for second-best training time, DelLM performs best in terms of model parameters, memory consumption, GFLOPs, and MAE. Specifically, DelLM improves upon the strongest baseline model by 50.8%, 50.8%, 66.7%, and 1.0% in terms of parameters, memory consumption, GFLOPs, and MAE, respectively. These results demonstrate that DelLM not only has fewer parameters but also converges faster, has lower computational complexity, and is less dependent on hardware resources. These characteristics are valuable in real-world deployment environments. In conclusion, DelLM not only excels in prediction accuracy but also offers significant advantages in efficiency and resource consumption. Figure 5 Except for MAE, all other metrics are standardized using their respective maximum values.

[0096] (6) Quantitative analysis:

[0097] This section uses quantitative analysis to illustrate how the designed high-order semantic structure alignment module enhances LLM's ability to understand the context of residual components. For example... Figure 6 As shown in figure a, before alignment, the distribution of residual features is quite mixed, making it difficult for LLM to effectively identify the key features of rising and falling trajectories. In contrast, as shown in figure a... Figure 6 As shown in b, the distribution of aligned residual features exhibits a clear structural characteristic. Combined with the ablation experiment results presented earlier, this demonstrates that the proposed high-order semantic structure alignment module plays a crucial role in guiding LLM to understand residual components.

[0098] In summary, the DeLLM proposed in this invention effectively alleviates the bottleneck of insufficient modeling of wave components caused by the equal treatment of all components in current LLM-based methods, and fully utilizes the advantages of LLM in representation modeling and the lightweight characteristics of small models. To further enhance the representation ability of LLM for wave components, a high-order semantic structure alignment module was designed. Extensive experiments show that DeLLM exhibits excellent long-term prediction performance and generalization ability while maintaining its lightweight characteristics.

[0099] In another embodiment, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a power load forecasting decoupling method based on a large language model.

[0100] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0101] In another embodiment, the present invention also provides a computer device, including a processor and a memory storing a plurality of computer instructions, wherein the computer instructions, when executed by the processor, implement the steps of a power load forecasting decoupling method based on a large language model.

[0102] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the method of the present invention by running the computer program stored in the memory.

[0103] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0104] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0105] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A decoupling method for power load forecasting based on a large language model, characterized in that, Includes the following steps: The acquired historical power load data series is decomposed into trend components, seasonal components, and residual components; For trend components and seasonal components, trend embeddings and seasonal embeddings are output based on convolutional layers and linear layers, respectively. The residual components are aligned with the word features extracted from the prompt words using a higher-order semantic structure. The input of the large language model is constructed based on the aligned residual components and word features. The output of the large language model is then subjected to residual concatenation to obtain the residual embedding. The algorithm integrates trend embedding, seasonal embedding, and residual embedding, and uses the integrated features to predict the power load forecast results for multiple consecutive time steps in the future through the prediction head.

2. The power load forecasting decoupling method based on a large language model according to claim 1, characterized in that, Using trend components and seasonal components as input components, and trend embeddings and seasonal embeddings as output embeddings, the step of outputting trend embeddings and seasonal embeddings based on convolutional layers and linear layers respectively for the trend components and seasonal components includes: The input components are reshaped, and multiple linear layers are used to spatially map the reshaped features. The outputs of all linear layers are combined to obtain the mapped input pattern. Multiple one-dimensional convolutional layers are used to process the input components and concatenate their features. Then, a linear layer is used to process the concatenated features to obtain the spatial weights of the input components. The output embedding is obtained by integrating the spatial weights and mapped input patterns with the Einstein summation convention and the linear layer.

3. The power load forecasting decoupling method based on a large language model according to claim 1, characterized in that, The trend component and the seasonal component share convolutional layers and linear layers.

4. The power load forecasting decoupling method based on a large language model according to claim 1, characterized in that, The step of aligning the residual components with the word features extracted from the prompt words in a higher-order semantic structure includes: The residual components are segmented to obtain the residual segmented components; A linear layer is used to perform a linear transformation on the residual block components, projecting the residual block components onto a feature space of the same dimension as the word features; Normalize the projected residual block components and word features; The cosine similarity algorithm is used to calculate the correlation matrix between the normalized residual block components and word features; The association matrix, normalized residual block components, and word features are processed by a hypergraph neural network, and the aligned residual components are output.

5. The power load forecasting decoupling method based on a large language model according to claim 1, characterized in that, The input to the large language model constructed based on the aligned residual components and word features includes: The word features are used as prefix data, and the aligned residual components are used as suffix data to construct the input of the large language model.

6. The power load forecasting decoupling method based on a large language model according to claim 1, characterized in that, The residual concatenation of the output of the large language model to obtain residual embeddings includes: The residual components are passed through a linear layer to obtain transformed features; The final transformation features and the output of the large language model The residual embedding is obtained by adding the 3D features. This refers to the dimension of the power load data.

7. The power load forecasting decoupling method based on a large language model according to claim 1, characterized in that, The prediction head uses a linear layer with the LeakyReLU activation function.

Images