Power load prediction method, device, equipment and medium
By employing a cross-modal alignment and large language model-based power load forecasting method, the problem of insufficient forecast stability in heterogeneous scenarios with multi-source data is solved, achieving high-precision, robust, and interpretable power load forecasting, which is applicable to short-term and medium-to-long-term load forecasting of power systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XI AN JIAOTONG UNIV
- Filing Date
- 2026-01-14
- Publication Date
- 2026-04-17
AI Technical Summary
Existing technologies for power load forecasting suffer from insufficient forecast stability and weak interpretability in heterogeneous scenarios with multi-source data. Furthermore, large language models are prone to losing accurate numerical features and dynamic evolution patterns of the system during the time series to text conversion process, resulting in insufficient forecast accuracy and robustness.
A power load prediction method based on cross-modal alignment and a large language model is adopted. By constructing a dual-modal coding architecture and a cross-modal similarity alignment mechanism, historical power load data is input into the time series coding branch and the large language model. Time series embeddings and semantic embeddings are extracted respectively, and feature alignment between modes is achieved through a channel similarity retrieval mechanism. Finally, the power load prediction result is output.
It improves the accuracy, robustness, and interpretability of power load forecasting, and is applicable to short-term and medium-to-long-term load forecasting scenarios in power systems. It also reduces computational costs and enhances the model's generalization performance.
Smart Images
Figure CN121886366A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of power load forecasting technology, and in particular to a power load forecasting method, apparatus, equipment and medium. Background Technology
[0002] Accurate load forecasting is an important foundation for the safe and economical operation of power systems and is of key significance for grid dispatching, generation planning, and electricity market transactions.
[0003] In recent years, with the breakthroughs of Large Language Models (LLMs) in the field of natural language processing, their potential in multimodal learning and time series modeling has attracted attention. Existing research has attempted to input time series data into language models in text form, using prompt engineering to enable the models to understand time series patterns.
[0004] However, converting time series data to text can easily result in the loss of precise numerical features and dynamic evolution patterns of the system. It is difficult to achieve deep collaborative calibration of semantic information and time series features. In cases of sparse data, noise interference, or new electricity consumption scenarios, semantic and numerical data may become disconnected, leading to a need to improve the accuracy, robustness, and interpretability of power load forecasting. Summary of the Invention
[0005] Therefore, it is necessary to provide a power load forecasting method, apparatus, equipment, and medium to address the aforementioned technical problems, thereby overcoming the deficiencies in the existing technology and achieving more accurate power load forecasting.
[0006] The following technical solution is adopted in this specification: This specification provides a method for predicting electricity load, including: Historical time series data are obtained from different data sources associated with the power system to be predicted; Historical time series data from different data sources are input into the prompt encoding branch of the pre-trained power load prediction model. The prompt encoding branch converts the historical time series data from different data sources into single-source natural language texts that describe the attributes, time series characteristics and power load correlation information of the data sources, and encodes each single-source natural language text to obtain prompt embeddings that include the semantic information of each data source. Historical time series data from different data sources are input into the time series encoding branch of the pre-trained power load prediction model. The time series encoding branch maps the historical time series data from different data sources into single-source time series features that characterize the overall time series change characteristics of the corresponding data sources, and models the dependency relationship of each single-source time series feature to obtain the time series embedding that characterizes the overall dynamic evolution of the power system to be predicted. The cross-modal fusion module of the pre-trained power load prediction model is fused with cue embedding and time series embedding to obtain the enhanced cross-modal embedding of power load. The cross-modal enhancement of power load is embedded into the decoding module of the pre-trained power load prediction model to obtain the power load prediction result of the power system to be predicted.
[0007] Furthermore, the cue encoding branch includes a large language model and an embedding compression unit; obtaining the cue embedding, which includes semantic information from each data source, specifically includes: The large language model is used to transform historical time series data from different data sources into single-source natural language text that describes data source attributes, time series trends, and power load correlation information. Each single-source natural language text is encoded by an embedded compression unit. During the encoding process, the single-source natural language text is split into multiple tokens, and the preceding tokens of each single-source natural language text are aggregated into the ending token. The ending tokens corresponding to each single-source natural language text are then integrated to obtain a hint embedding that includes semantic information from each data source. in, Indicates the first A terminal token corresponding to a single-source natural language text; Indicates by the first A matrix consisting of query vectors of single-source natural language text preceding tokens; Indicates by the first A matrix consisting of key vectors of preorder tokens of a single-source natural language text; the superscript T indicates a device; Indicates the dimension of the key vector; Indicates by the first The single-source natural language text is generated by the first A matrix consisting of value vectors of preorder tokens from single-source natural language texts; Indicates the total number of data sources; ~ The first single-source natural language text ~ A terminal token corresponding to a single-source natural language text; This is a hint embedded.
[0008] Furthermore, the time series coding branch includes an inverse embedding layer and a Transformer encoder; obtaining the time series embedding characterizing the overall dynamic evolution of the power system to be predicted specifically includes: By using reverse embedding, historical time series data from different data sources are mapped into single-source time series features that characterize the overall temporal variation features of the corresponding data sources: in, Total number of data sources; and These are the first and second mapping weights learned by the linear mapping layer through pre-training. For from the Historical time series data from multiple data sources; For the first Single-source time-series characteristics of a data source; By modeling the dependencies of single-source time-series features corresponding to each data source using a Transformer encoder, a time-series embedding representing the overall dynamic evolution of the power system to be predicted is obtained. in, ~ For the first data source ~ the Single-source time-series characteristics of each data source, where the superscript T indicates transpose. express The dimension is , The length of a single-source temporal feature; For Transformer encoders; Embedded for time series.
[0009] Furthermore, the cross-modal fusion module includes a projection unit, a channel similarity calculation unit, and a weighted feature fusion unit. The process of obtaining the cross-modal enhanced embedding of the power load specifically includes: The prompt is embedded through the projection unit. and time series embedding Project to shared space: in, , and These represent the first, second, and third shared projection matrices learned during the pre-training process, respectively. Representing time series embedding The projection results in the shared space are used as query vectors for channel similarity calculation; Indicates embedded prompt The projection results in the shared space, used as key vectors for channel similarity calculation; Indicates embedded prompt The projection results in the shared space are used as a value vector for channel similarity calculation; The hint embedding is obtained through the channel similarity calculation unit. and time series embedding Channel similarity matrix : The superscript T indicates transpose; Indicates the dimension of the key vector; The cue embedding is channel-weighted by a weighted feature fusion unit and linearly superimposed with the time series embedding to obtain the cross-modal enhanced embedding of power load: in, It is a cross-modal enhancement embedding of power load.
[0010] Furthermore, the decoding module includes a masked self-attention layer, a cross-modal attention layer, and a feedforward network layer. Obtaining the power load prediction result for the power system to be predicted specifically includes: Cross-modal enhancement embedding of power load through masked self-attention layer Modeling time-sequential dependencies: in, It is an upper triangular mask matrix used to prevent the decoding module from accessing future information and to ensure the causality of the prediction; , and To enhance embedding based on cross-modal The generated query vector, key vector, and value vector; This represents time-dependent power characteristics; the superscript T indicates transpose. Modeling temporal dependencies through a cross-modal attention layer Embedded with prompts Modeling cross-modal interaction relationships between them: in, For features modeled by time-series dependencies The generated query vector; and Each is embedded by a prompt The generated key vectors and value vectors are used to provide supplementary semantic information; It is a cross-modal electric characteristic; The power load prediction results are output through the feedforward network layer: in, This is the result of electricity load forecasting; and These are the first and second prediction matrices learned during the pre-training process.
[0011] Furthermore, the power load forecasting model incorporates a loss function that combines forecast accuracy and modal alignment constraints. Achieve pre-training: Where M is the time step; For the first The predicted load value at each time step, For the first The actual load value at each time step; and These are the first and second weighting coefficients; This is the parameter set for the power load forecasting model. Indicates the L2 regularization term; For modal alignment regularization: For prompt embedding and time series embedding The channel similarity matrix; It is an identity matrix.
[0012] This specification provides a power load forecasting device, including: The data acquisition module is used to acquire historical time series data from different data sources associated with the power system to be predicted; The prompt embedding module is used to input historical time series data from different data sources into the prompt encoding branch of the pre-trained power load prediction model. The prompt encoding branch converts the historical time series data from different data sources into single-source natural language texts that describe the attributes, time series characteristics and power load correlation information of the data sources, and encodes each single-source natural language text to obtain prompt embeddings that include the semantic information of each data source. The time series embedding module is used to input historical time series data from different data sources into the time series encoding branch of the pre-trained power load prediction model. The time series encoding branch maps the historical time series data from different data sources into single-source time series features that represent the overall time series change characteristics of the corresponding data sources, and models the dependency relationship of each single-source time series feature to obtain the time series embedding that represents the overall dynamic evolution of the power system to be predicted. The fusion module is used to fuse the cue embedding and time series embedding into the pre-trained cross-modal fusion module of the power load prediction model to obtain the enhanced cross-modal embedding of power load. The prediction module is used to embed the cross-modal power load enhancement into the decoding module of the pre-trained power load prediction model and decode it to obtain the power load prediction result of the power system to be predicted.
[0013] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described power load forecasting method.
[0014] This specification provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described power load forecasting method.
[0015] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: This invention addresses the pain points of heterogeneous multi-source data, insufficient prediction stability in complex scenarios, and weak interpretability in power load forecasting. It accurately models the dynamic evolution of the power system through time series branching, focusing on the temporal characteristics of power load changes with multiple factors. Through cue coding branching, it transforms multi-source time series data into semantically rich natural language descriptions, capturing implicit correlations and complex patterns that are difficult to quantify between data sources. After cross-modal fusion, the two form a semantic guidance-temporal calibration closed loop: semantic information compensates for the blind spots in power load forecasting under data sparsity, noise interference, or new scenarios, while temporal features correct the insufficient perception of precise numerical changes in power load by pure language models. Ultimately, it simultaneously improves the accuracy, robustness, and interpretability of power load forecasting. Attached Figure Description
[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0017] Figure 1 This is one of the flowcharts illustrating a power load forecasting method provided in this specification; Figure 2 This is the second flowchart illustrating a power load forecasting method provided in this specification. Figure 3 This is a schematic diagram of the structure of a power load forecasting device provided in this specification; Figure 4 This is a schematic diagram of a computer device provided for this specification. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.
[0019] The technical solution provided by this invention can be applied to power load forecasting scenarios. With the rapid development of new power systems and new energy sources, power load exhibits a high degree of randomness and multi-factor coupling characteristics. Accurate load forecasting is an important foundation for the safe and economical operation of power systems and is of key significance for grid dispatching, power generation planning, and electricity market transactions.
[0020] Traditional load forecasting methods are typically based on statistical modeling and time series analysis, such as ARIMA, exponential smoothing, and regression analysis. These models rely on manual feature extraction and have limited ability to characterize nonlinear relationships and the time-series dependencies of multidimensional variables, making them ill-suited for handling complex fluctuations and abrupt changes in load data.
[0021] In recent years, deep learning methods have demonstrated powerful feature extraction capabilities in time series forecasting. Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformer models have been widely applied in power load forecasting, automatically capturing the correlation between time and variables. However, these models still suffer from two main problems: first, they have limited model parameters and are highly dependent on training data, resulting in a significant drop in predictive performance when there are insufficient samples or high noise levels; second, they struggle to utilize external semantic information or cross-domain knowledge, thus limiting the model's generalization ability.
[0022] In recent years, with breakthroughs in Large Language Models (LLMs) in natural language processing, their potential in multimodal learning and time series modeling has attracted attention. Existing research has attempted to input time series data into language models as text, using prompt engineering to enable the model to understand time series patterns. However, existing methods often face the "modal coupling problem," where directly concatenating time series embeddings and language embeddings makes it difficult to distinguish between effective information and noise, leading to decreased prediction performance. Furthermore, the long length of the prompt text significantly increases computational overhead, limiting its real-time application in power systems.
[0023] Therefore, designing a prediction method that can fully integrate time series structural information, utilize the semantic knowledge of large language models, and maintain computational efficiency is a pressing technical problem in the field of power load forecasting. Based on this, this invention provides a power load forecasting method based on cross-modal alignment and a large language model, aiming to solve the problem of insufficient robustness of traditional time series models in small-sample, multivariate load forecasting, as well as the difficulties in modeling multi-source time series data, severe modal information coupling, and high computational costs in existing technologies. This method achieves the fusion of time series and linguistic knowledge by constructing a dual-modal coding architecture and a cross-modal similarity alignment mechanism. The specific steps are as follows: First, historical power load data is input into the time series coding branch, and decoupled but weak time series embeddings are extracted through a reverse embedding layer and a lightweight pre-normalized Transformer encoder; second, the same load data is converted into natural language prompts, input into a frozen large language model to obtain entangled but robust prompt embeddings, and a design is made to retain only the last token representation to compress computation. Furthermore, based on a cross-modal alignment module using channel similarity retrieval, the temporal features in the robust cue embedding are aligned and injected into the time series embedding, thereby enhancing robustness. Finally, the fused embedding is input into a multivariate Transformer decoder to output power load forecast results for several future time points. This method leverages the knowledge transfer capabilities of large language models and cross-modal retrieval mechanisms to improve load forecasting accuracy, generalization performance, and computational efficiency, making it suitable for short-term and medium-to-long-term load forecasting scenarios in power systems.
[0024] The following is combined Figures 1-2 The power load forecasting method of the present invention is described in detail.
[0025] Figure 1 This is one of the flowcharts illustrating a power load forecasting method provided in this specification, such as... Figure 1 As shown, the method includes the following: S101. Obtain historical time series data from different data sources associated with the power system to be predicted.
[0026] For example, the different data sources associated with the power system to be predicted include internal data sources of the power system and external related data sources. The internal data sources may cover time-series data that directly reflect the operating status of the power system, such as historical load data of each node of the regional power grid, transmission line operating parameters, transformer load data, and power grid dispatch instruction records. The external related data sources may include time-series data that indirectly affect changes in power load, such as meteorological time-series data (e.g., temperature, humidity, wind speed, precipitation), regional socio-economic activity data (e.g., industrial production schedules, business hours, holiday arrangements), energy policy adjustment records, and equipment maintenance plans. Various data sources can be flexibly selected and supplemented according to the actual prediction scenario, and this application embodiment does not impose any restrictions on this.
[0027] S102. Input historical time series data from different data sources into the prompt encoding branch of the pre-trained power load prediction model. The prompt encoding branch converts the historical time series data from different data sources into single-source natural language texts that describe the attributes, time series characteristics and power load correlation information of the data sources, and encodes each single-source natural language text to obtain prompt embeddings that include semantic information of each data source.
[0028] The prompting encoding branch includes a large language model and an embedding compression unit.
[0029] For example, a large language model is used to transform historical time series data from different data sources into single-source natural language text that describes data source attributes, time series trends, and power load correlation information.
[0030] For example, each single-source natural language text is encoded by an embedding compression unit. During the encoding process, the single-source natural language text is split into multiple tokens, and the preceding tokens of each single-source natural language text are aggregated into the ending token. The ending tokens corresponding to each single-source natural language text are then integrated to obtain a hint embedding that includes semantic information from each data source. in, Indicates the first A terminal token corresponding to a single-source natural language text; Indicates by the first A matrix consisting of query vectors of single-source natural language text preceding tokens; Indicates by the first A matrix consisting of key vectors of preorder tokens of a single-source natural language text; the superscript T indicates a device; Indicates the dimension of the key vector; Indicates by the first The single-source natural language text is generated by the first A matrix consisting of value vectors of preorder tokens from single-source natural language texts; Indicates the total number of data sources; ~ The first single-source natural language text ~ A terminal token corresponding to a single-source natural language text; This is a hint embedded.
[0031] S103. Input historical time series data from different data sources into the time series encoding branch of the pre-trained power load prediction model. The time series encoding branch maps the historical time series data from different data sources into single-source time series features that represent the overall time series change characteristics of the corresponding data sources, and models the dependency relationship of each single-source time series feature to obtain the time series embedding that represents the overall dynamic evolution of the power system to be predicted.
[0032] The time-series coding branch includes the inverse embedding layer and the Transformer encoder.
[0033] For example, historical time series data from different data sources are mapped to single-source time series features representing the overall temporal variation characteristics of the corresponding data sources through back embedding: in, Total number of data sources; and These are the first and second mapping weights learned by the linear mapping layer through pre-training. For from the Historical time series data from multiple data sources; For the first Single-source time-series characteristics of each data source.
[0034] For example, by modeling the dependencies of single-source time-series features corresponding to each data source using a Transformer encoder, a time-series embedding representing the overall dynamic evolution of the power system to be predicted is obtained: in, ~ For the first data source ~ the Single-source time-series characteristics of each data source, where the superscript T indicates transpose. express The dimension is , The length of a single-source temporal feature; For Transformer encoders; Embedding for time series S104. The cue embedding and time series embedding are fused into the cross-modal fusion module of the pre-trained power load prediction model to obtain the cross-modal enhanced embedding of power load.
[0035] The cross-modal fusion module includes a projection unit, a channel similarity calculation unit, and a weighted feature fusion unit.
[0036] For example, the prompt is embedded through the projection unit. and time series embedding Project to shared space: in, , and These represent the first, second, and third shared projection matrices learned during the pre-training process, respectively. Representing time series embedding The projection results in the shared space are used as query vectors for channel similarity calculation; Indicates embedded prompt The projection results in the shared space, used as key vectors for channel similarity calculation; Indicates embedded prompt The projection results in the shared space are used as a value vector for channel similarity calculation.
[0037] For example, the cue embedding is obtained through the channel similarity calculation unit. and time series embedding Channel similarity matrix : The superscript T indicates transpose; This represents the dimension of the key vector.
[0038] For example, the cue embedding is channel-weighted by a weighted feature fusion unit and linearly superimposed with the time series embedding to obtain the cross-modal enhanced embedding of power load: in, It is a cross-modal enhancement embedding of power load.
[0039] S105. Decode the power load prediction model by embedding the cross-modal enhancement of the power load into the decoding module of the pre-trained power load prediction model to obtain the power load prediction result of the power system to be predicted.
[0040] The decoding module includes a mask self-attention layer, a cross-modal attention layer, and a feedforward network layer.
[0041] For example, cross-modal enhancement embedding of power load is achieved through a masked self-attention layer. Modeling time-sequential dependencies: in, It is an upper triangular mask matrix used to prevent the decoding module from accessing future information and to ensure the causality of the prediction; , and To enhance embedding based on cross-modal The generated query vector, key vector, and value vector; This represents time-dependent power characteristics; the superscript T indicates transpose.
[0042] For example, modeling temporal dependencies through cross-modal attention layers. Embedded with prompts Modeling cross-modal interaction relationships between them: in, For features modeled by time-series dependencies The generated query vector; and Each is embedded by a prompt The generated key vectors and value vectors are used to provide supplementary semantic information; It represents the cross-modal electrical characteristics.
[0043] For example, the power load forecast results are output through the feedforward network layer: in, This is the result of electricity load forecasting; and These are the first and second prediction matrices learned during the pre-training process.
[0044] For example, the power load forecasting model uses a loss function that combines forecast accuracy with mode alignment constraints. Achieve pre-training: Where M is the time step; For the first The predicted load value at each time step, For the first The actual load value at each time step; and These are the first and second weighting coefficients; This is the parameter set for the power load forecasting model. Indicates the L2 regularization term; For modal alignment regularization: For prompt embedding and time series embedding The channel similarity matrix; It is an identity matrix.
[0045] Figure 2 This is the second flowchart illustrating a power load forecasting method provided in this specification, as shown below. Figure 2 As shown, the method first takes "multi-source time series data such as load, voltage, and temperature" as input. On one hand, it is fed into the prompt encoding branch, where it undergoes processing including "sequence data to natural language conversion," "tokenizer splitting text tokens," "freezing LLM to extract semantic embeddings," and finally "token embedding compression" to obtain the prompt encoding. On the other hand, the multi-source time series data is fed into the time series encoding branch, where it undergoes "reverse embedding" and "time series encoding" to obtain the time series embedding. Subsequently, the prompt encoding and time series embedding enter the "cross-modal alignment" module to complete feature matching. The aligned features are then processed by "multivariate Transformer decoding" and "linear mapping," finally outputting the predicted power load. In a specific embodiment, the power load prediction method of this application may include the following steps:
[0046] Step S1: Collect multivariate time series data including historical load, voltage, current, temperature, humidity, holidays, and weather level.
[0047] First, multi-source data from the power system is collected and preprocessed. The collected data includes power load. ,Voltage Current Temperature ,humidity Wind speed Holiday markings Economic Activity Index Multiple time-related variables are combined to form a multidimensional time series dataset. Let the sampling time interval be... At any time Each sample can be represented as follows:
[0048] ; in, This represents the number of input features, each This corresponds to a single feature component. (Past) The input sequence for each time slice is defined as:
[0049] .
[0050] Predicting the target is the future. Power load sequence for each time slice: .
[0051] In the data preprocessing stage, the feature variables are first normalized to ensure that different physical quantities (such as power, voltage, and temperature) are on a uniform scale. Second, missing data is imputed by interpolation, and outliers are denoised using moving median filtering. Then, input-output sample pairs are constructed using the sliding window method. For categorical variables such as holidays and weekends, one-hot encoding is used to map them to numerical features. Finally, a training sample set is formed. , as input for subsequent models.
[0052] Step S2: Establish a dual-modal coding structure, including a time-series coding branch and a cue coding branch. The time-series coding branch uses a reverse embedding mechanism to input the entire time series of each power variable as a single token into a lightweight pre-normalized Transformer encoder, thereby extracting the temporal dynamic features of each variable. The cue coding branch encapsulates the time-series data into cue text describing it in natural language, such as "From time t1 to time t2, the load values are v1 to v..." T The overall trend value is ΔT", which is input into a pre-trained large language model to extract semantically enhanced cue embeddings, fully utilizing the external knowledge and transfer capabilities of the language model. To simultaneously model the temporal variation patterns and semantic relationships of power system load data, this invention proposes a dual-modal coding structure, including a time-series coding branch and a large language model cue coding branch.
[0053] For example, the time series encoding branch includes a reverse embedding layer, a layer normalization layer, a multi-head self-attention layer, and a feedforward neural network layer. The reverse embedding layer converts a time series of length T into a learnable vector representation, improving model stability and convergence speed through residual connections and pre-normalization structures. The multi-head self-attention mechanism generates three sets of representations—query, key, and value—through a linear projection matrix, calculates attention weights between variables, and captures the complex interaction characteristics of power load in both the time and variable dimensions, thereby obtaining a decoupled and representative time series embedding. In the time series encoding branch, the multivariate time series is input as a whole into the embedding model. Assume the power system includes... If there are 1 main observed feature, then the input matrix can be represented as:
[0054] ; Among them, the first The line represents the first These features are in recent A historical sequence of time slices. Feature embeddings are extracted through a linear mapping layer:
[0055] ; in , For the embedding dimension, each feature is mapped to a length of [length missing]. The vectors form the input matrix:
[0056] .
[0057] This embedding structure is called "reverse embedding," which treats each variable as a single-source temporal feature, enabling the model to model global dependencies at the variable level.
[0058] Then The core calculation process in the lightweight pre-normalized Transformer encoder is as follows: ; in, .
[0059] This mechanism can capture the temporal correlations between different power characteristics (such as temperature, humidity, and load) to obtain a time-series embedded output: .
[0060] For example, the generation of the prompt text includes: combining the observation values, sampling frequency, and trend descriptions of each variable within the time period [t1, t2] in natural language form; when processing the prompt, the large language model uses a masked self-attention mechanism to converge the information of all preceding tokens to the last token. The embedding vector of this token represents the overall semantics and dynamic features of the time series and serves as a key input for cross-modal alignment. In the prompt encoding branch, to incorporate the semantic prior information of the large language model, this invention converts the time series data into natural language prompt text. Let the time interval be... The prompt text can then be expressed as: "From time..." arrive The power load values are as follows: The average temperature is The overall trend is ."in, This indicates the overall load change trend. The cue is input into a frozen large language model (such as a GPT-like model), and the cue embedding is calculated through its multi-layer masked self-attention structure.
[0061] .
[0062] To reduce computational costs, this invention employs a last token embedding compression mechanism, enabling the language model to automatically aggregate all contextual information into the last token during internal attention computation. ; in This is the global semantic representation of the prompt text. The final result is the language modality embedding matrix:
[0063] .
[0064] Step S3: A channel similarity-based retrieval mechanism is used to achieve feature fusion between time-series and language modalities in the cross-modal alignment module. This module generates query, key, and value vectors through trilinear mapping, calculates the channel similarity matrix between the time-series embedding and the cue embedding, and retrieves the most relevant robust features from the cue embedding to achieve adaptive cross-modal information alignment. The embedding compression mechanism of the last token identifies the last token in the cue sequence of the large language model and uses it as the global representation of the cue. This token embedding learns the comprehensive information of all preceding time tokens through masked self-attention during the training phase and is stored offline during the inference phase to avoid redundant computation. The offline embedding, after unified dimensional mapping, can directly participate in the cross-modal alignment and prediction stages, effectively reducing computational resource consumption and improving the real-time performance of the model in long-cycle prediction scenarios of power systems.
[0065] After completing the bimodal feature extraction, it is necessary to match and fuse the time series embedding (HT) and the language embedding (HP). To this end, this invention designs a cross-modal alignment module, which achieves information alignment between the two modalities through a channel similarity retrieval mechanism. For example, the cross-modal alignment module includes a channel similarity calculation unit and a weighted feature fusion unit. The channel similarity calculation unit uses a trilinear mapping to generate queries respectively. ,key Sum Calculate the channel similarity matrix M between the time series embedding and the cue embedding. T The weighted feature fusion unit performs channel weighting on the cue embedding based on the similarity matrix and linearly superimposes it with the original time series embedding to obtain the fused cross-modal embedding H. C This enables robust time-series features to be retrieved from the cue embeddings. Through this module, the time-series branch provides a decoupled dynamic structure, and the large language model branch provides a robust semantic context. The alignment and fusion of the two significantly enhances the generalization performance of load prediction.
[0066] First, and Project them separately to a shared representation space: .
[0067] Calculate the channel similarity matrix between the time series modality and the language modality: ; in, Indicates the first The time series feature channel and the first The matching strength of each prompt semantic channel.
[0068] Based on the similarity matrix, the prompt features are weighted and fused: ; Received This is a cross-modal fusion feature that combines structured temporal information with semantic knowledge.
[0069] To maintain consistency in modal matching, a regularization term is introduced: ; in The identity matrix is used, and the similarity matrix is constrained to have a near-diagonal structure to prevent modal mismatch. Through this mechanism, the model can automatically retrieve the most relevant knowledge to power load changes from language prompts, improving its adaptability to complex scenarios such as sudden fluctuations and extreme weather.
[0070] Step S4: Employ the embedding compression mechanism of the last token, and leverage the mask self-attention characteristic of the large language model to guide the model to centrally express all temporal information in the last token of the prompt text, and retain only the embedding of the token as a global representation to participate in cross-modal alignment and prediction, thereby effectively reducing the computational overhead and inference time of the large language model.
[0071] For example, the multivariate Transformer decoder includes a masked self-attention layer, a cross-modal attention layer, and a feedforward network layer. The masked self-attention layer is used to model the order dependency in the temporal dimension, the cross-modal attention layer is used to learn the interaction and semantic complementarity relationships between variables, and the feedforward layer is used to nonlinearly map the output prediction results. The decoder maintains the integrity of the temporal dependency structure through residual connections and utilizes a linear projection function. Generate power load forecasts X̂ for the next M time points. M The model optimization objective function includes the prediction error term L. pre and regularization term L reg The mean squared error is used as the main loss metric, and overfitting is prevented by L2 norm constraints. The features after cross-modal fusion are... The input is fed into a multivariate Transformer decoder to achieve multi-step load prediction. The decoder consists of a masked self-attention layer, a cross-modal attention layer, and a feedforward layer. The formula for calculating the masked self-attention layer is:
[0072] ; in, It is an upper triangular mask matrix used to prevent the decoder from accessing future information, ensuring the causality of the prediction.
[0073] The cross-modal attention layer further integrates time series and linguistic semantic features: .
[0074] The output composite features are processed through a linear mapping layer: .
[0075] Get the future Power load forecast results for each time slot: .
[0076] The loss function comprehensively considers both prediction accuracy and mode alignment consistency. ; in, These are the weighting coefficients. This is the set of model parameters.
[0077] S5: The fused cross-modal data is embedded into the multivariate Transformer decoder. Long-term dependencies between variables are established through the masked self-attention layer and the cross-modal attention layer. After linear projection, the predicted power load values for several future times are output.
[0078] Once the model is trained, it can perform real-time predictions under various power scenarios. Input recent Historical data for each time slice By combining automatically generated semantic prompts, the future can be predicted. System load per time slice.
[0079] In regional power grid load forecasting scenarios, by inputting semantic information such as "weekday load increase trend" and "significant fluctuations in electricity consumption during holidays" through prompt templates, the model can accurately capture periodic characteristics and social behavior patterns. In renewable energy grid connection scenarios, external conditions such as "increased wind speed" and "enhanced solar radiation" can be introduced into the prompts to strengthen the model's ability to perceive fluctuations in renewable energy output. In abnormal fluctuation detection, when the prompts include descriptions such as "equipment maintenance" and "sudden power outage," the model can adaptively adjust the forecast trend and output the corrected load value.
[0080] The model achieves dynamic collaborative learning of time series modalities and language modalities through a cross-modal alignment mechanism. The knowledge transfer provided by the large language model can effectively alleviate the problems of data scarcity and noise interference. The model has good generalization ability under different regional, seasonal and climatic conditions. Additional semantic information, such as equipment maintenance plans and descriptions of meteorological anomalies, can be flexibly introduced through prompt templates, thereby further improving the model's prediction stability and reliability for extreme load changes.
[0081] The present invention has the following beneficial effects: By using a channel similarity retrieval mechanism, dynamic alignment between time series modalities and language modalities is achieved, and robust time series information is retrieved from cue embeddings, avoiding information redundancy and modal noise interference caused by traditional splicing methods.
[0082] The time series of each electricity variable is treated as a whole as a token input model, thereby capturing long-range dependencies at the variable level and significantly improving the ability to model multidimensional features.
[0083] By guiding the large language model to focus semantics on the final token embedding, and storing and using only the embedding for subsequent predictions, the large-scale repetitive reasoning process is avoided, thereby improving reasoning speed and reducing resource consumption.
[0084] The time series branch provides decoupled structural features, while the large language model branch provides external knowledge semantics. After cross-modal fusion, the two can maintain stable prediction performance under conditions of scarce samples, noise interference, or new scenarios.
[0085] The prompt template can be flexibly adjusted according to changes in the scenario, such as adding climate descriptions, equipment maintenance plans or social event information, so that the model has the integrated capability of "semantic understanding - numerical prediction".
[0086] By combining offline token storage with a lightweight Transformer architecture, this method can significantly reduce computational resource consumption compared to traditional deep learning models while ensuring high-accuracy prediction, making it widely applicable to real-time load scheduling and energy management scenarios.
[0087] The power load forecasting device provided by the present invention is described below. The power load forecasting device described below can be referred to in correspondence with the power load forecasting method described above.
[0088] Figure 3 This is a schematic diagram of a power load forecasting device provided in this specification. For example, please refer to [link to relevant documentation]. Figure 3 As shown, the power load forecasting device may include: The data acquisition module is used to acquire historical time series data from different data sources associated with the power system to be predicted.
[0089] The prompt embedding module is used to input historical time series data from different data sources into the prompt encoding branch of the pre-trained power load prediction model. The prompt encoding branch converts the historical time series data from different data sources into single-source natural language text that describes the data source attributes, time series characteristics and power load correlation information, and encodes each single-source natural language text to obtain prompt embeddings that include semantic information from each data source.
[0090] The time series embedding module is used to input historical time series data from different data sources into the time series encoding branch of the pre-trained power load prediction model. The time series encoding branch maps the historical time series data from different data sources into single-source time series features that represent the overall time series change characteristics of the corresponding data sources, and models the dependency relationship of each single-source time series feature to obtain the time series embedding that represents the overall dynamic evolution of the power system to be predicted.
[0091] The fusion module is used to fuse the cue embedding and time series embedding into the pre-trained cross-modal fusion module of the power load prediction model to obtain the cross-modal enhanced embedding of power load.
[0092] The prediction module is used to embed the cross-modal power load enhancement into the decoding module of the pre-trained power load prediction model and decode it to obtain the power load prediction result of the power system to be predicted.
[0093] Specific limitations regarding the power load forecasting device can be found in the limitations on power load forecasting described above, and will not be repeated here. Each module in the aforementioned power load forecasting device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in a computer device, or stored in software within the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.
[0094] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The provided method for predicting power load.
[0095] Figure 4 This specification provides a schematic diagram of a computer device, such as... Figure 4 At the hardware level, the computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The provided method for predicting power load.
[0096] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.
[0097] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
Claims
1. A method for predicting electricity load, characterized in that, include: Historical time series data are obtained from different data sources associated with the power system to be predicted; Historical time series data from different data sources are input into the prompt encoding branch of the pre-trained power load prediction model. The prompt encoding branch converts the historical time series data from different data sources into single-source natural language texts that describe the attributes, time series characteristics and power load correlation information of the data sources, and encodes each single-source natural language text to obtain prompt embeddings that include the semantic information of each data source. Historical time series data from different data sources are input into the time series encoding branch of the pre-trained power load prediction model. The time series encoding branch maps the historical time series data from different data sources into single-source time series features that characterize the overall time series change characteristics of the corresponding data sources, and models the dependency relationship of each single-source time series feature to obtain the time series embedding that characterizes the overall dynamic evolution of the power system to be predicted. The cross-modal fusion module of the pre-trained power load prediction model is fused with cue embedding and time series embedding to obtain the enhanced cross-modal embedding of power load. The cross-modal enhancement of power load is embedded into the decoding module of the pre-trained power load prediction model to obtain the power load prediction result of the power system to be predicted.
2. The power load forecasting method as described in claim 1, characterized in that, The prompting encoding branch includes a large language model and an embedding compression unit; The process of obtaining the embedded prompts that include semantic information from each data source specifically includes: The large language model is used to transform historical time series data from different data sources into single-source natural language text that describes data source attributes, time series trends, and power load correlation information. Each single-source natural language text is encoded by an embedded compression unit. During the encoding process, the single-source natural language text is split into multiple tokens, and the preceding tokens of each single-source natural language text are aggregated into the ending token. The ending tokens corresponding to each single-source natural language text are then integrated to obtain a hint embedding that includes semantic information from each data source. in, Indicates the first A terminal token corresponding to a single-source natural language text; Indicates by the first A matrix consisting of query vectors of single-source natural language text preceding tokens; Indicates by the first A matrix consisting of key vectors of preorder tokens of a single-source natural language text; the superscript T indicates a device; Indicates the dimension of the key vector; Indicates by the first The single-source natural language text is generated by the first A matrix consisting of value vectors of preorder tokens from single-source natural language texts; Indicates the total number of data sources; ~ The first single-source natural language text ~ A terminal token corresponding to a single-source natural language text; This is a hint embedded.
3. The power load forecasting method as described in claim 1, characterized in that, The time-series coding branch includes an inverse embedding layer and a Transformer encoder; The acquisition of the time series embedding characterizing the overall dynamic evolution of the power system to be predicted specifically includes: By using reverse embedding, historical time series data from different data sources are mapped into single-source time series features that characterize the overall temporal variation features of the corresponding data sources: in, Total number of data sources; and These are the first and second mapping weights learned by the linear mapping layer through pre-training. For from the Historical time series data from multiple data sources; For the first Single-source time-series characteristics of each data source; By modeling the dependencies of single-source time-series features corresponding to each data source using a Transformer encoder, a time-series embedding representing the overall dynamic evolution of the power system to be predicted is obtained. in, ~ For the first data source ~ Single-source time-series characteristics of each data source, where the superscript T indicates transpose. express The dimension is , The length of a single-source temporal feature; For Transformer encoders; Embedded for time series.
4. The power load forecasting method as described in claim 1, characterized in that, The cross-modal fusion module includes a projection unit, a channel similarity calculation unit, and a weighted feature fusion unit. The process of obtaining the cross-modal enhanced embedding of the power load specifically includes: The prompt is embedded through the projection unit. and time series embedding Project to shared space: in, , and These represent the first, second, and third shared projection matrices learned during the pre-training process, respectively. Representing time series embedding The projection results in the shared space are used as query vectors for channel similarity calculation; Indicates embedded prompt The projection results in the shared space are used as key vectors for channel similarity calculation; Indicates embedded prompt The projection results in the shared space are used as a value vector for channel similarity calculation; The hint embedding is obtained through the channel similarity calculation unit. and time series embedding Channel similarity matrix : The superscript T indicates transpose; Indicates the dimension of the key vector; The cue embedding is channel-weighted by a weighted feature fusion unit and linearly superimposed with the time series embedding to obtain the cross-modal enhanced embedding of power load: in, It is a cross-modal enhancement embedding of power load.
5. The power load forecasting method as described in claim 1, characterized in that, The decoding module includes a masked self-attention layer, a cross-modal attention layer, and a feedforward network layer. Obtaining the power load prediction result for the power system to be predicted specifically includes: Cross-modal enhancement embedding of power load through masked self-attention layer Modeling time-sequential dependencies: in, It is an upper triangular mask matrix used to prevent the decoding module from accessing future information and to ensure the causality of the prediction; , and To enhance embedding based on cross-modal The generated query vector, key vector, and value vector; This represents time-dependent power characteristics; the superscript T indicates transpose. Modeling temporal dependencies through a cross-modal attention layer Embedded with prompts Modeling cross-modal interaction relationships between them: in, For features modeled by time-series dependencies The generated query vector; and Embedded by prompts respectively The generated key vectors and value vectors are used to provide supplementary semantic information; It is a cross-modal power characteristic; The power load prediction results are output through the feedforward network layer: in, This is the result of electricity load forecasting; and These are the first and second prediction matrices learned during the pre-training process.
6. The power load forecasting method as described in claim 1, characterized in that, The power load forecasting model uses a loss function that integrates forecast accuracy and modal alignment constraints. Achieve pre-training: Where M is the time step; For the first The predicted load value at each time step, For the first The actual load value at each time step; and These are the first and second weighting coefficients; This is the parameter set for the power load forecasting model. Indicates the L2 regularization term; For modal alignment regularization: For prompt embedding and time series embedding The channel similarity matrix; It is an identity matrix.
7. A power load forecasting device, characterized in that, include: The data acquisition module is used to acquire historical time series data from different data sources associated with the power system to be predicted; The prompt embedding module is used to input historical time series data from different data sources into the prompt encoding branch of the pre-trained power load prediction model. The prompt encoding branch converts the historical time series data from different data sources into single-source natural language texts that describe the attributes, time series characteristics and power load correlation information of the data sources, and encodes each single-source natural language text to obtain prompt embeddings that include the semantic information of each data source. The time series embedding module is used to input historical time series data from different data sources into the time series encoding branch of the pre-trained power load prediction model. The time series encoding branch maps the historical time series data from different data sources into single-source time series features that represent the overall time series change characteristics of the corresponding data sources, and models the dependency relationship of each single-source time series feature to obtain the time series embedding that represents the overall dynamic evolution of the power system to be predicted. The fusion module is used to fuse the cue embedding and time series embedding into the pre-trained cross-modal fusion module of the power load prediction model to obtain the enhanced cross-modal embedding of power load. The prediction module is used to embed the cross-modal power load enhancement into the decoding module of the pre-trained power load prediction model and decode it to obtain the power load prediction result of the power system to be predicted.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the power load forecasting method as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the power load forecasting method as described in any one of claims 1 to 6.
Citation Information
Cited By
Automobile demand prediction method and system based on large model time sequence cross-modal alignment
CN122089380A