Multi-scale wind and light storage power prediction method and device based on large language model

Abstract

The present application relates to wind and light heat storage multi-energy power prediction, smart grid and artificial intelligence technical field, especially in kind of based on large language model's multi-scale wind and light heat storage power prediction method and device, wherein, method includes: obtain historical wind and light heat storage multi-energy power data and the time corresponding meteorological and operating condition data, and the historical wind power data is pretreated, to obtain the wind and light heat storage multi-energy semantic embedding vector suitable for large language model;Meteorological and operating condition data are used to construct hybrid multi-source knowledge base, to retrieve the multi-source knowledge fragment related to wind and light heat storage multi-energy semantic embedding vector, and the above-mentioned data is input into pre-training large language model, to predict the future wind and light heat storage multi-energy power sequence of different length according to prediction scale parameter. Therefore, the present wind power prediction method solves the problems of insufficient semantic information utilization, insufficient meteorological knowledge fusion, weak cross-prediction window generalization ability and high large model fine-tuning cost.

Description

Technical Field

[0001] This invention relates to the fields of multi-energy power prediction for wind, solar, and thermal energy storage, smart grids, and artificial intelligence, and particularly to a method and apparatus for multi-scale wind, solar, and thermal energy storage power prediction based on a large language model with LoRA (Low-Rank Adaptation) fine-tuning and RAG (Retrieval-Augmented Generation) enhancement. Background Technology

[0002] Wind, solar, energy storage, and thermal energy are increasingly important energy sources in new power systems. Due to rapid changes in meteorological factors such as wind speed, air pressure, and temperature, wind power output exhibits significant randomness, volatility, and non-stationarity. Similarly, the output and load of multi-energy sources (wind, solar, energy storage, and thermal) also exhibit significant randomness, volatility, coupling, and non-stationarity due to rapid changes in factors such as wind speed, air pressure, temperature, solar irradiance, load demand, energy storage state of charge, and heating conditions. With the large-scale grid connection of wind power, photovoltaics, energy storage, and heating resources, prediction errors can affect dispatch plans, reserve capacity allocation, and renewable energy consumption. In severe cases, they can lead to wind curtailment and economic losses, including wind and solar power curtailment, energy storage charge and discharge deviations, heating imbalances, and economic losses. Therefore, improving the accuracy, interpretability, and cross-prediction time domain generalization ability of multi-energy power prediction for wind, solar, energy storage, and thermal energy is a key issue in smart grid operation. Summary of the Invention

[0003] This invention provides a multi-scale wind, solar, and thermal power prediction method and device based on a large language model to solve the problems of insufficient utilization of semantic information, insufficient integration of meteorological knowledge, weak generalization ability across prediction windows, and high cost of fine-tuning large models in existing wind power prediction methods.

[0004] A first aspect of this invention provides a multi-scale wind, solar, and thermal energy storage power prediction method based on a large language model, comprising the following steps: Acquire historical multi-energy power data of the target wind, solar and thermal storage station or integrated energy system, as well as meteorological and operational data corresponding to the time of the historical multi-energy power data. The historical power data of wind, solar, thermal storage and energy storage are preprocessed to obtain semantic embedding vectors of wind, solar, thermal storage and energy storage that are adapted to a large language model. A hybrid multi-source knowledge base is constructed using the meteorological and operational data to retrieve multi-source knowledge fragments related to the semantic embedding vector of the wind, solar, thermal storage and energy storage multi-energy system. The historical power data of wind, solar, thermal storage and other energy sources, as well as multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage and other energy sources, are input into a pre-built pre-trained large language model to predict future power sequences of different lengths of wind, solar, thermal storage and other energy sources based on the prediction scale parameter.

[0005] Optionally, the preprocessing of the historical wind, solar, and thermal energy power data to obtain a semantic embedding vector for wind, solar, and thermal energy systems adapted to a large language model includes: A cubic spline interpolation function is constructed for the historical wind, solar, thermal storage, and multi-energy power data to calculate missing values, and the missing values ​​are used to complete the historical wind, solar, thermal storage, and multi-energy power data to obtain a new power sequence. The normal distribution characteristics of the new power sequence are statistically analyzed to remove outliers in the new power sequence, so as to obtain the final preprocessed power sequence; The final preprocessed power sequence is normalized to obtain a normalized numerical sequence. The normalized numerical sequence is divided into multiple data segments, and semantic transformation is performed on each data segment to obtain a semantic embedding vector for wind, solar, thermal storage and other energy sources that is adapted to a large language model.

[0006] Optionally, the step of constructing a hybrid multi-source knowledge base using the meteorological and operational data, and retrieving multi-source knowledge fragments related to the semantic embedding vector of the wind, solar, and thermal energy storage multi-energy system from the hybrid multi-source knowledge base, includes: The meteorological and operational data are fragmented to obtain multiple multi-source knowledge fragments; Vector representations are embedded in each multi-source knowledge fragment to construct the hybrid multi-source knowledge base; Calculate the similarity between the semantic embedding vector of the wind, solar, thermal storage and each multi-source knowledge fragment in the hybrid multi-source knowledge base; Based on the similarity, retrieve the multi-source knowledge fragments most relevant to the semantic embedding vector of the wind, solar, thermal storage and energy multi-energy source from the hybrid multi-source knowledge base.

[0007] Optionally, the step of inputting the historical wind, solar, and thermal energy multi-energy power data and multi-source knowledge fragments related to the semantic embedding vectors of the wind, solar, and thermal energy multi-energy systems into a pre-constructed large language model to predict future wind, solar, and thermal energy multi-energy power sequences of different lengths based on prediction scale parameters includes: The self-attention query matrix, key matrix, and value matrix in the pre-trained large language model are efficiently fine-tuned using a low-rank adaptation fine-tuning method, while keeping the remaining parameters of the pre-trained large language model frozen, so as to obtain a large language model predictor suitable for multi-energy power prediction tasks of wind, solar, thermal storage, and heat storage. The historical power data of wind, solar, thermal storage and other energy sources, as well as the multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage and other energy sources, are input into the large language model predictor suitable for the power prediction task of wind, solar, thermal storage and other energy sources, so as to output the future prediction sequences of different lengths according to the prediction scale parameter.

[0008] A second aspect of the present invention provides a multi-scale wind, solar, and thermal energy storage power prediction device based on a large language model, comprising: The acquisition module is used to acquire historical multi-energy power data of wind, solar and thermal storage at the target wind, solar and thermal storage station or integrated energy system, as well as meteorological and operating condition data corresponding to the time of the historical multi-energy power data of wind, solar and thermal storage. The preprocessing module is used to preprocess the historical wind, solar, thermal storage and power data of multiple energy sources to obtain a semantic embedding vector of multiple energy sources that is adapted to a large language model. The retrieval module is used to construct a hybrid multi-source knowledge base using the meteorological and operational data, so as to retrieve multi-source knowledge fragments related to the semantic embedding vector of the wind, solar and thermal energy storage in the hybrid multi-source knowledge base; The prediction module is used to input the historical power data of wind, solar, thermal storage and other energy sources, as well as multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage and other energy sources, into a pre-built pre-trained large language model, so as to predict the power sequences of wind, solar, thermal storage and other energy sources of different lengths in the future according to the prediction scale parameter.

[0009] Optionally, the preprocessing module includes: The completion unit is used to construct a cubic spline interpolation function for the historical wind, solar, thermal storage, and multi-energy power data to calculate missing values, and to complete the historical wind, solar, thermal storage, and multi-energy power data based on the missing values ​​to obtain a new power sequence. The removal unit is used to statistically analyze the normal distribution characteristics of the new power sequence to remove outliers in the new power sequence and obtain the final preprocessed power sequence. A normalization unit is used to normalize the final preprocessed power sequence to obtain a normalized numerical sequence. The semantic transformation unit is used to divide the normalized numerical sequence into multiple data segments and perform semantic transformation on each data segment to obtain a multi-energy semantic embedding vector of wind, solar, thermal storage and adapting to a large language model.

[0010] Optionally, the retrieval module includes: The fragmentation unit is used to fragment the meteorological and operational data to obtain multiple multi-source knowledge fragments. A construction unit is used to embed vector representations into each multi-source knowledge fragment to construct the hybrid multi-source knowledge base; The computing unit is used to calculate the similarity between the semantic embedding vector of the wind, solar and thermal energy storage multi-energy system and each multi-source knowledge fragment in the hybrid multi-source knowledge base; The retrieval unit is used to retrieve, based on the similarity, multi-source knowledge fragments from the hybrid multi-source knowledge base that are most relevant to the semantic embedding vector of the wind, solar, thermal, and energy storage multi-energy system.

[0011] Optionally, the prediction module includes: The fine-tuning unit is used to efficiently fine-tune the parameters of the self-attention query matrix, key matrix and value matrix in the pre-trained large language model using a low-rank adaptation fine-tuning method, while keeping the remaining parameters of the pre-trained large language model frozen, so as to obtain a large language model predictor suitable for multi-energy power prediction tasks of wind, solar and thermal storage. The prediction unit is used to input the historical power data of wind, solar and thermal energy storage and multi-energy ...

[0012] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-scale wind, solar, and thermal power prediction method based on a large language model as described in the above embodiments.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described multi-scale wind, solar, and thermal power prediction method based on a large language model.

[0014] The multi-scale wind, solar, and thermal energy storage power prediction method and device based on a large language model proposed in this invention improves the accuracy, interpretability, and cross-input / output window and multi-scale prediction transfer capabilities of multi-energy power prediction, while reducing the computational overhead of adapting the large language model to multi-energy prediction tasks.

[0015] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0016] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart illustrating a multi-scale wind, solar, and thermal energy storage power prediction method based on a large language model, according to an embodiment of the present invention; Figure 2 This is a comparison chart of training errors under different input / output window lengths and prediction scales according to an embodiment of the present invention; Figure 3 This is a comparison chart of MAE (Mean Absolute Error) and MSE (Mean Squared Error) errors of different model structures in an ablation experiment according to an embodiment of the present invention. Figure 4 This is a curve comparing the actual and predicted power values ​​of a multi-energy system (wind, solar, and thermal energy storage) according to an embodiment of the present invention. Figure 5 A block diagram of a unified wind, solar, and thermal energy power prediction device based on a large language model according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention.

[0017] Explanation of reference numerals in the attached figures: 50-Multi-scale wind, solar and thermal power prediction device based on large language model, 501-Acquisition module, 502-Preprocessing module, 503-Retrieval module, 504-Prediction module, 601-Memory, 602-Processor, 603-Communication interface. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0019] The following describes, with reference to the accompanying drawings, a multi-scale wind, solar, and thermal energy storage power prediction method and apparatus based on a large language model according to embodiments of the present invention. Addressing the problems mentioned in the background section regarding insufficient semantic information utilization, inadequate multi-source knowledge fusion, weak generalization ability across input / output windows and prediction scales, and high cost of full-scale fine-tuning of large models in existing multi-energy prediction methods for wind, solar, and thermal energy storage, the present invention provides a multi-scale wind, solar, and thermal energy storage power prediction method based on LoRA fine-tuning and RAG enhancement using a large language model. In this method, historical wind power, photovoltaic power, energy storage charging / discharging power, and heat load data of the target wind, solar, and thermal energy storage station or integrated energy system are acquired, along with multi-source data such as temperature, air pressure, wind speed, solar irradiance, humidity, load demand, energy storage state of charge, and heating conditions. The multi-energy numerical sequences of wind, solar, and thermal energy storage are preprocessed and then vectorized. Embedding is performed to obtain an expression adapted to the large language model; a hybrid multi-source knowledge base is constructed, and multi-source knowledge retrieval is performed based on the retrieval enhancement generation RAG framework, with the retrieval results used as enhanced context input to the large language model; low-rank adaptation LoRA is used to fine-tune only the query, key, and value matrices of the large language model's self-attention layer, while freezing the remaining parameters, to obtain a large language model predictor suitable for the wind, solar, thermal storage, and multi-energy power prediction task; historical wind, solar, thermal storage, and multi-energy power data, as well as multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage, and multi-energy, are input into the large language model predictor suitable for the wind, solar, thermal storage, and multi-energy power prediction task to output future multi-scale, multi-timestep wind, solar, thermal storage, and multi-energy power prediction results.

[0020] Specifically, Figure 1 This is a flowchart illustrating a multi-scale wind, solar, and thermal energy storage power prediction method based on a large language model, provided in an embodiment of the present invention.

[0021] like Figure 1 As shown, the multi-scale wind-solar-thermal storage power prediction method based on a large language model includes the following steps: In step S101, historical multi-energy power data of wind, solar and thermal storage at the target wind, solar and thermal storage station or integrated energy system, as well as meteorological and operational condition data corresponding to the time of the historical multi-energy power data, are obtained.

[0022] In actual implementation, multi-energy data on wind, solar, and thermal energy storage are collected from the area to be tested. , To determine the total number of data samples, a segment is extracted from the multi-energy data of wind, solar, thermal storage, and energy storage in the area to be tested as historical multi-energy power data. , The input window length is, For the predicted time The previous The system acquires the historical multi-energy power data of wind, solar, thermal storage, and energy storage at each sampling point, and obtains meteorological and operational data corresponding to the historical multi-energy power data time. The meteorological and operational data include temperature, air pressure, wind speed, solar irradiance, humidity, load demand, energy storage charge status, and heating conditions.

[0023] In step S102, the historical power data of wind, solar, thermal storage and energy storage are preprocessed to obtain semantic embedding vectors of wind, solar, thermal storage and energy storage that are adapted to the large language model.

[0024] In some embodiments, historical wind, solar, and thermal energy power data are preprocessed to obtain semantic embedding vectors for wind, solar, and thermal energy systems adapted to a large language model, including: A cubic spline interpolation function is constructed for historical wind, solar, thermal storage, and multi-energy power data to calculate missing values. The missing values ​​are then used to complete the historical wind, solar, thermal storage, and multi-energy power data to obtain a new power sequence. The normal distribution characteristics of the new power sequence are statistically analyzed to remove outliers and obtain the final preprocessed power sequence. The final preprocessed power sequence is normalized to obtain a normalized numerical sequence. The normalized numerical sequence is divided into multiple data segments, and semantic transformation is performed on each data segment to obtain a semantic embedding vector for wind, solar, thermal storage and other energy sources that is adapted to a large language model.

[0025] In practice, the missing data is first filled using cubic spline interpolation. This technique connects known data points by constructing cubic spline functions, giving the filled curve a second-order continuous derivative, thus maintaining the smoothness and coherence of the data. This method not only accurately fills in missing values ​​but also effectively captures the fluctuation patterns of the original data. Furthermore, cubic spline interpolation exhibits high computational efficiency when processing large-scale historical power datasets, making it suitable for practical applications. Z-score normalization is then used to screen for outliers in the newly filled power sequences. The Z-score method identifies outliers based on the standard deviation of the data point from the overall mean. Specifically, for each data point, the deviation from the mean is calculated and divided by the standard deviation to obtain the Z-score. Data points with Z-scores exceeding the range are marked as outliers, thus efficiently identifying extreme biases in the data. This completion and outlier detection process significantly improves the overall data quality, laying a solid data foundation for fine-tuning the large model.

[0026] Specifically, the time-series structure of historical power data is first clarified: assume that the historical power data to be processed is a discrete-time series. , This represents the total number of data samples, with a time interval of 15 minutes. for The actual power value at each time point (unit: MW). The preprocessing objective is to eliminate missing data (denoted as the set of missing timestamps). ) and anomalies (denoted as the set of anomaly moments) , A continuous and smooth preprocessed power sequence was obtained. This provides high-quality data support for subsequent model training.

[0027] For time series data Let the known valid data points be... , ( (Number of valid samples after removing missing points) Construct a cubic spline interpolation function Satisfy each sub-interval , ( )Inside, It is a cubic polynomial:

[0028] In the formula, The coefficients of the cubic polynomial in the k-th subinterval (to be solved). This represents the relative time within the sub-interval.

[0029] To ensure The continuity and smoothness must satisfy the following constraints: The interpolation function must pass through all known valid data points, that is:

[0030] Adjacent sub-intervals at the connection point The first derivatives at each point are equal (to avoid abrupt changes in power trends), that is:

[0031] Adjacent sub-nodes The second derivatives at each point are equal, that is:

[0032] Boundary constraints:

[0033] After solving for the coefficients and substituting the missing time, the missing values ​​can be calculated using a cubic polynomial, resulting in a new power sequence with the missing values ​​filled in. .

[0034] Furthermore, outliers in historical power data from wind, solar, and thermal energy storage often originate from sensor anomalies. The Z-score method, by statistically analyzing the normal distribution characteristics of power data, quantifies the degree of deviation to identify anomalies. The specific operation is as follows: Power sequence after missing values ​​are filled Calculate its mean With unbiased standard deviation :

[0035] For each moment Calculate its Z-score (quantifying the degree of deviation of the power from the mean at that moment):

[0036] Set the anomaly detection threshold as follows >5, for power values ​​deemed abnormal The final preprocessed power sequence was obtained by using the "power mean replacement between adjacent normal times" method.

[0037] In the formula, and The value must be non-abnormal (if it is still abnormal at an adjacent time, then extend to the most recent normal time to ensure that the replacement value conforms to the power trend).

[0038] Furthermore, the final preprocessed power sequence is normalized, as follows:

[0039] In the formula, and The input sequences are respectively The minimum and maximum values ​​in the normalized numerical sequence will be used to find the normalized values. After segmentation, several data fragments are obtained. Through this processing, the power of multiple energy sources such as wind, solar, and thermal energy storage under different installed capacities or different operating ranges is mapped to a unified numerical range, which is beneficial for large language models to focus on sequence trends and fluctuation patterns.

[0040] Then, the normalized numerical sequence The sequence can be divided into a series of segments; the process of block segmentation can be represented as follows:

[0041] In the formula, Indicates the first A multi-energy power segment combining wind, solar, and thermal storage. Indicates the number of segments. This represents a segmentation operation used to divide a long sequence segment into shorter sequence segments. Each segment... It can further extract statistical features such as mean, maximum, minimum, rate of change, and fluctuation range.

[0042] Furthermore, semantic transformation is performed on these small fragments to form semantic representations. This gives numerical fragments semantic attributes that can be understood by large language models.

[0043]

[0044] In the formula, Represents the normalized numerical sequence The derived semantic representation, This represents semantic transformation operations.

[0045] For the semantic representation Tokenization yields:

[0046] In the formula, This represents the word segmentation function. Indicates a combination of multiple word segments The first of the composition A sequence of word segments.

[0047] Subsequently, each lexical unit is mapped to a high-dimensional embedding vector and combined with positional encoding to preserve sequence order. This step enables numerical multi-energy power sequences of wind, solar, and thermal storage to be converted into embedding representations that can be processed by large language models, i.e., semantic embedding vectors of wind, solar, and thermal storage adapted to large language models. This achieves alignment between the numerical sequence and the language model input space.

[0048]

[0049]

[0050]

[0051] In the formula, Indicates the function of embedding The generated high-dimensional vector, This represents positional encoding, used to preserve the original order of the sequence. Indicates the embedding location index. This represents the embedding vector dimension. Positional encoding enables the model to distinguish the predictive meaning of the same numerical form appearing at different time points. Since the power output of wind, solar, and thermal energy storage systems exhibits a clear temporal sequence, if only lexical embeddings are used without positional encoding, the model struggles to distinguish between different evolutionary processes such as "increase followed by decrease" and "decrease followed by increase." By introducing positional encoding, the model can preserve the sequential relationships between segments, thus more accurately modeling the continuous variation of power output from wind, solar, and thermal energy storage systems.

[0052] In step S103, a hybrid multi-source knowledge base is constructed using meteorological and operational data to retrieve multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage, and other energy sources.

[0053] In some embodiments, a hybrid multi-source knowledge base is constructed using meteorological and operational condition data to retrieve multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal energy storage, and other energy sources, including: Meteorological and operational data are fragmented to obtain multiple multi-source knowledge fragments; Vector representations are embedded in each multi-source knowledge fragment to construct a hybrid multi-source knowledge base; Calculate the similarity between the semantic embedding vectors of wind, solar, thermal storage and each multi-source knowledge fragment in the hybrid multi-source knowledge base; Based on similarity, retrieve the multi-source knowledge fragments most relevant to the semantic embedding vectors of wind, solar, thermal storage, and other energy sources from a hybrid multi-source knowledge base.

[0054] In actual implementation, meteorological and operational data are fragmented to obtain multiple multi-source knowledge fragments. Each multi-source knowledge fragment is embedded as a vector representation to achieve efficient retrieval, as detailed below: in, Indicates the first Each knowledge fragment is represented by a vector, which includes both external meteorological conditions and power station response, and is used to describe the correspondence between meteorological changes and the output of multiple energy sources such as wind, solar, and thermal energy storage. All knowledge vectors together constitute a knowledge vector base. ,in This represents the total number of knowledge fragments.

[0055] Furthermore, a hybrid multi-source knowledge base is constructed using vector-represented multi-source knowledge fragments, specifically expressed as:

[0056] In the formula, This represents the construction of a multi-source knowledge base. Representing the Each includes temperature Sea level pressure Site pressure Wind speed and power A fragment.

[0057] Furthermore, similarity is then calculated to retrieve the knowledge fragments most relevant to the multi-energy semantic vector. Cosine similarity is preferably used for similarity calculation, and the formula is as follows:

[0058] In the formula, This indicates the current semantic embedding of multiple energy sources including wind, solar, and thermal energy storage, and the first... The dot product of knowledge vectors, and Let L1 and L2 represent the L2 norms of the two vectors, respectively. Cosine similarity measures the consistency of the directions of two vectors and is suitable for comparing the semantic correlation between the current operating status of wind, solar, thermal, and energy storage systems and historical multi-source knowledge fragments.

[0059] Most relevant The retrieval process for a knowledge fragment can be represented as follows:

[0060] In the formula, This represents the most relevant meteorological fragments extracted using RAG. Represents the calculation of the dot product of vectors. This represents Euclidean norm operations.

[0061] To avoid interference from low-relevance knowledge fragments in prediction, a similarity threshold can be set. .when At times, historical knowledge fragments are not forcibly incorporated; instead, predictions are made using only current meteorological and operational condition summaries and semantic embeddings of wind, solar, and thermal energy storage. In this process, the retrieved fragments are used as augmentation information for prediction. By setting a similarity threshold, noise introduced by low-relevance knowledge fragments can be avoided, thereby improving the robustness of the retrieval augmentation prediction process.

[0062] By constructing a multi-source knowledge base, RAG technology can extract meteorological and operational information related to input wind, solar, thermal storage, and other energy sources, thereby improving the interpretability and accuracy of the prediction results generated by the large language model.

[0063] In step S104, historical power data of wind, solar, thermal storage and other energy sources, as well as multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage and other energy sources, are input into a pre-built pre-trained large language model to predict future power sequences of different lengths of wind, solar, thermal storage and other energy sources based on the prediction scale parameter.

[0064] In some embodiments, historical wind, solar, thermal storage, and multi-energy power data, along with multi-source knowledge fragments related to the semantic embedding vectors of these energy sources, are input into a pre-built large language model to predict future wind, solar, thermal storage, and multi-energy power sequences of different lengths based on prediction scale parameters. This includes: A low-rank adaptation fine-tuning method is used to efficiently fine-tune the parameters of the self-attention query matrix, key matrix, and value matrix in the pre-trained large language model, while keeping the remaining parameters of the pre-trained large language model frozen, so as to obtain a large language model predictor suitable for multi-energy power prediction tasks of wind, solar, thermal storage, and heat storage. Historical power data of wind, solar, thermal storage and other energy sources, as well as multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage and other energy sources, are input into a large language model predictor suitable for the power prediction task of wind, solar, thermal storage and other energy sources, so as to output future prediction sequences of different lengths according to the prediction scale parameter, wherein the different lengths include short-term, medium-term, long-term or other specified output lengths.

[0065] In actual implementation, in order to enable the large model originally used for processing natural language to predict numerical wind power, photovoltaic power, energy storage charging and discharging power and heat load, this embodiment of the invention uses LoRA to perform efficient and lightweight fine-tuning of the large model. This method freezes the original weights of the large model to retain its generalization ability.

[0066] Specifically, when performing LoRA fine-tuning, the frozen weight matrix is... Introduce two trainable low-rank matrices and , The fine-tuned forward propagation process can be represented as:

[0067] In the formula, As a result of reasoning, For input, These are low-rank trainable weight parameters.

[0068] Specifically, this embodiment of the invention only updates the parameters of the query matrix, key matrix, and value matrix of the self-attention mechanism. These three matrices are the core parameters of the large model Transformer architecture; therefore, by fine-tuning only the query matrix, key matrix, and value matrix, the large language model can be adapted to multi-energy prediction tasks involving wind, solar, and thermal energy storage.

[0069] These three matrices are represented as , and The updated formula is equivalent to:

[0070]

[0071]

[0072] in, , , , , , These are the low-rank trainable matrices corresponding to the query, key, and value matrices, respectively.

[0073] low-rank matrix and Pre-trained weights participate in gradient updates. , and Keep the parameters frozen to reduce the number of trainable parameters and the risk of overfitting.

[0074] The value obtained by the dot product operation is the attention score, and the specific expression is as follows:

[0075] In the formula, Indicates by and The calculated attention score, express Dimensions This is used to scale the dot product results to avoid excessively high attention scores when the dimensionality is too high. The function gradient is too small. and Represent The original components and updated components. A single attention head in , and The application method is as follows:

[0076] In the formula, Indicates the first A single point of attention, Attention scores are normalized to a probability distribution ranging from 0 to 1. Multi-head attention is an extension of the attention mechanism that initializes multiple single attention heads and concatenates them. Multi-head structures can simultaneously capture short-term fluctuations, long-term trends, and correlations with weather and operating conditions from different subspaces.

[0077]

[0078] In the formula, It refers to splicing Multi-head attention obtained from a single attention head. During the training phase, let the loss function be... The learning rate is Due to the original weight matrix Fixed, the gradient is only for low-rank matrices. and Perform the calculation.

[0079] With matrix For example, its gradient calculation is as follows: Regarding matrices The gradient is given by the following equation:

[0080] Regarding matrices The gradient is given by the following equation:

[0081] Subsequently, updates were performed via the optimizer. and The parameters and their update rules are defined as follows:

[0082]

[0083] During LoRA fine-tuning, only and The large language model can be used for training, while all other parameters remain fixed, ultimately resulting in a large language model predictor suitable for multi-energy power prediction tasks involving wind, solar, and thermal energy storage. The specific expression is as follows: In the formula, This is the large model after fine-tuning. and These are the pre-trained weights and the fine-tuned weights, respectively. For the coded multi-energy power of wind, solar and thermal storage, This is for data splicing operations, used to combine multi-energy data from wind, solar, thermal storage, and related meteorological and operational information. The text prompts include prediction step size, input / output window settings, historical sequence statistics, and instructions for the multi-energy prediction task involving wind, solar, thermal, and energy storage. This efficient update strategy targeting the self-attention mechanism maintains the generalization ability of a large language model while reducing the number of trainable parameters.

[0084] Furthermore, historical power data of wind, solar, and thermal energy storage, along with multi-source knowledge fragments related to the semantic embedding vectors of these energy sources, are input into a large language model predictor suitable for power prediction tasks of wind, solar, and thermal energy storage, to output future multi-timescale power prediction results of wind, solar, and thermal energy storage. in, This is the length of the output window. It should be noted that... and It can be configured according to different forecasting needs.

[0085] like Figure 2-4 As shown, the training error of the multi-scale wind, solar and thermal power prediction method based on a large language model proposed in this embodiment of the invention is compared under different input and output window lengths, the MAE and MSE errors of different model structures in the ablation experiment are compared, and the actual and predicted values ​​of multi-energy power of wind, solar and thermal storage are compared. It can be seen that the embodiments of the invention improve the prediction accuracy, interpretability and cross-prediction time domain generalization ability of multi-energy power of wind, solar and thermal storage.

[0086] In summary, the multi-scale wind, solar, and thermal energy storage power prediction method based on a large language model proposed in this embodiment of the invention improves the prediction accuracy, interpretability, and cross-input / output window and multi-scale prediction transfer capabilities of the multi-energy power prediction method, while reducing the computational cost of adapting the large language model to the multi-energy prediction task of wind, solar, and thermal energy storage.

[0087] Next, referring to the accompanying drawings, we describe the multi-scale wind, solar, and thermal power prediction device based on a large language model proposed according to an embodiment of the present invention.

[0088] Figure 5 This is a block diagram of a multi-scale wind, solar and thermal power prediction device based on a large language model, provided in an embodiment of the present invention.

[0089] like Figure 5 As shown, the multi-scale wind, solar and thermal power prediction device 50 based on a large language model includes: an acquisition module 501, a preprocessing module 502, a retrieval module 503 and a prediction module 504.

[0090] The acquisition module 501 acquires historical multi-energy power data of the target wind-solar-thermal storage power station or integrated energy system, as well as meteorological and operational condition data corresponding to the time of the historical multi-energy power data. The preprocessing module 502 preprocesses the historical multi-energy power data to obtain semantic embedding vectors for the wind-solar-thermal storage system that are adapted to the large language model. The retrieval module 503 constructs a hybrid multi-source knowledge base using meteorological and operational condition data to retrieve multi-source knowledge fragments related to the semantic embedding vectors of the wind-solar-thermal storage system from the hybrid multi-source knowledge base. The prediction module 504 inputs the historical multi-energy power data and the multi-source knowledge fragments related to the semantic embedding vectors of the wind-solar-thermal storage system into a pre-built, pre-trained large language model to predict future multi-energy power sequences of different lengths based on prediction scale parameters.

[0091] In some embodiments, the preprocessing module 502 includes: The completion unit is used to construct a cubic spline interpolation function for historical wind, solar, thermal storage and power data to calculate missing values, and to complete the historical wind, solar, thermal storage and power data based on the missing values ​​to obtain a new power sequence. The removal unit is used to statistically analyze the normal distribution characteristics of the new power sequence in order to remove outliers in the new power sequence and obtain the final preprocessed power sequence. The normalization unit is used to normalize the final preprocessed power sequence to obtain a normalized numerical sequence. The semantic transformation unit is used to divide the normalized numerical sequence into multiple data segments and perform semantic transformation on each data segment to obtain a multi-energy semantic embedding vector of wind, solar, thermal storage and adapting to a large language model.

[0092] In some embodiments, the retrieval module 503 includes: Fragmentation units are used to fragment meteorological and operational data to obtain multiple multi-source knowledge fragments. Building units are used to embed vector representations into each multi-source knowledge fragment to construct a hybrid multi-source knowledge base; The computing unit is used to calculate the similarity between the semantic embedding vector of wind, solar, thermal storage and each multi-source knowledge fragment in the hybrid multi-source knowledge base; The retrieval unit is used to retrieve, based on similarity, multi-source knowledge fragments from a hybrid multi-source knowledge base that are most relevant to the semantic embedding vectors of wind, solar, thermal, and energy storage.

[0093] In some embodiments, the prediction module 504 includes: The fine-tuning unit is used to efficiently fine-tune the parameters of the self-attention query matrix, key matrix and value matrix in the pre-trained large language model using low-rank adaptation fine-tuning, while keeping the remaining parameters of the pre-trained large language model frozen, so as to obtain a large language model predictor suitable for multi-energy power prediction tasks of wind, solar and thermal storage. The prediction unit is used to input historical power data of wind, solar, thermal storage and multi-energy sources, as well as multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage and multi-energy sources, into a large language model predictor suitable for the power prediction task of wind, solar, thermal storage and multi-energy sources, so as to output future prediction sequences of different lengths according to the prediction scale parameter.

[0094] It should be noted that the foregoing explanation of the embodiment of the multi-scale wind, solar and thermal power prediction method based on a large language model also applies to the multi-scale wind, solar and thermal power prediction device based on a large language model in this embodiment, and will not be repeated here.

[0095] The multi-scale wind, solar, and thermal energy storage power prediction device based on a large language model proposed in this embodiment of the invention improves the accuracy, interpretability, and cross-input / output window and multi-scale prediction transfer capabilities of multi-energy power prediction based on wind, solar, and thermal energy storage, while reducing the computational overhead of adapting the large language model to multi-energy prediction tasks based on wind, solar, and thermal energy storage.

[0096] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0097] The electronic device may include a memory 601, a processor 602, and a computer program stored in the memory 601 and executable on the processor 602. When the processor 602 executes the program, it implements the multi-scale wind, solar, and thermal energy storage power prediction method based on a large language model provided in the above embodiments.

[0098] Furthermore, electronic devices also include: Communication interface 603 is used for communication between memory 601 and processor 602.

[0099] The memory 601 is used to store computer programs that can run on the processor 602.

[0100] The memory 601 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0101] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0102] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.

[0103] Processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0104] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described multi-scale wind, solar, and thermal power prediction method based on a large language model.

[0105] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0106] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0107] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0108] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0109] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0110] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0111] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0112] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A multi-scale wind-solar-thermal storage power prediction method based on a large language model, characterized in that, Includes the following steps: Acquire historical multi-energy power data of the target wind, solar and thermal storage power station or integrated energy system, as well as meteorological and operational condition data corresponding to the time of the historical multi-energy power data. The historical power data of wind, solar, thermal storage and energy storage are preprocessed to obtain semantic embedding vectors of wind, solar, thermal storage and energy storage that are adapted to a large language model. A hybrid multi-source knowledge base is constructed using the meteorological and operational data to retrieve multi-source knowledge fragments related to the semantic embedding vector of the wind, solar, thermal storage and energy storage multi-energy system. The historical power data of wind, solar, thermal storage and other energy sources, as well as multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage and other energy sources, are input into a pre-built pre-trained large language model to predict future power sequences of different lengths of wind, solar, thermal storage and other energy sources based on the prediction scale parameter.

2. The multi-scale wind, solar, and thermal energy storage power prediction method based on a large language model according to claim 1, characterized in that, The preprocessing of the historical wind, solar, and thermal energy power data to obtain semantic embedding vectors for wind, solar, and thermal energy systems adapted to a large language model includes: A cubic spline interpolation function is constructed for the historical wind, solar, thermal storage, and multi-energy power data to calculate missing values, and the missing values ​​are used to complete the historical wind, solar, thermal storage, and multi-energy power data to obtain a new power sequence. The normal distribution characteristics of the new power sequence are statistically analyzed to remove outliers in the new power sequence, so as to obtain the final preprocessed power sequence; The final preprocessed power sequence is normalized to obtain a normalized numerical sequence. The normalized numerical sequence is divided into multiple data segments, and semantic transformation is performed on each data segment to obtain a semantic embedding vector for wind, solar, thermal storage and other energy sources that is adapted to a large language model.

3. The multi-scale wind, solar, and thermal energy storage power prediction method based on a large language model according to claim 1, characterized in that, The construction of a hybrid multi-source knowledge base using the meteorological and operational data, to retrieve multi-source knowledge fragments related to the semantic embedding vectors of the wind, solar, and thermal energy storage systems, includes: The meteorological and operational data are fragmented to obtain multiple multi-source knowledge fragments; Vector representations are embedded in each multi-source knowledge fragment to construct the hybrid multi-source knowledge base; Calculate the similarity between the semantic embedding vector of the wind, solar, thermal storage and each multi-source knowledge fragment in the hybrid multi-source knowledge base; Based on the similarity, retrieve the multi-source knowledge fragments most relevant to the semantic embedding vector of the wind, solar, thermal storage and energy multi-energy source from the hybrid multi-source knowledge base.

4. The multi-scale wind, solar, and thermal energy storage power prediction method based on a large language model according to claim 1, characterized in that, The step of inputting the historical wind, solar, and thermal energy multi-energy power data and multi-source knowledge fragments related to the semantic embedding vectors of the wind, solar, and thermal energy multi-energy sources into a pre-constructed large language model to predict future wind, solar, and thermal energy multi-energy power sequences of different lengths based on prediction scale parameters includes: The self-attention query matrix, key matrix, and value matrix in the pre-trained large language model are efficiently fine-tuned using a low-rank adaptation fine-tuning method, while keeping the remaining parameters of the pre-trained large language model frozen, so as to obtain a large language model predictor suitable for multi-energy power prediction tasks of wind, solar, thermal storage, and heat storage. The historical power data of wind, solar, thermal storage and other energy sources, as well as the multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage and other energy sources, are input into the large language model predictor suitable for the power prediction task of wind, solar, thermal storage and other energy sources, so as to output the future prediction sequences of different lengths according to the prediction scale parameter.

5. A multi-scale wind-solar-thermal storage power prediction device based on a large language model, characterized in that, include: The acquisition module is used to acquire historical multi-energy power data of wind, solar and thermal storage at the target wind, solar and thermal storage station or integrated energy system, as well as meteorological and operating condition data corresponding to the time of the historical multi-energy power data of wind, solar and thermal storage. The preprocessing module is used to preprocess the historical wind, solar, thermal storage and power data of multiple energy sources to obtain a semantic embedding vector of multiple energy sources of wind, solar, thermal storage and power sources that is adapted to a large language model. The retrieval module is used to construct a hybrid multi-source knowledge base using the meteorological and operational data, so as to retrieve multi-source knowledge fragments related to the semantic embedding vector of the wind, solar and thermal energy storage in the hybrid multi-source knowledge base; The prediction module is used to input the historical power data of wind, solar, thermal storage and other energy sources, as well as multi-source knowledge fragments related to the semantic embedding vectors of wind, solar, thermal storage and other energy sources, into a pre-built pre-trained large language model, so as to predict the power sequences of wind, solar, thermal storage and other energy sources of different lengths in the future according to the prediction scale parameter.

6. The multi-scale wind, solar, and thermal power prediction device based on a large language model according to claim 5, characterized in that, The preprocessing module includes: The completion unit is used to construct a cubic spline interpolation function for the historical wind, solar, thermal storage, and multi-energy power data to calculate missing values, and to complete the historical wind, solar, thermal storage, and multi-energy power data based on the missing values ​​to obtain a new power sequence. The removal unit is used to statistically analyze the normal distribution characteristics of the new power sequence to remove outliers in the new power sequence and obtain the final preprocessed power sequence. A normalization unit is used to normalize the final preprocessed power sequence to obtain a normalized numerical sequence. The semantic transformation unit is used to divide the normalized numerical sequence into multiple data segments and perform semantic transformation on each data segment to obtain a multi-energy semantic embedding vector of wind, solar, thermal storage and adapting to a large language model.

7. The multi-scale wind, solar, and thermal energy storage power prediction device based on a large language model according to claim 5, characterized in that, The retrieval module includes: The fragmentation unit is used to fragment the meteorological and operational data to obtain multiple multi-source knowledge fragments. A construction unit is used to embed vector representations into each multi-source knowledge fragment to construct the hybrid multi-source knowledge base; The computing unit is used to calculate the similarity between the semantic embedding vector of the wind, solar and thermal energy storage multi-energy system and each multi-source knowledge fragment in the hybrid multi-source knowledge base; The retrieval unit is used to retrieve, based on the similarity, multi-source knowledge fragments from the hybrid multi-source knowledge base that are most relevant to the semantic embedding vector of the wind, solar, thermal, and energy storage multi-energy system.

8. The multi-scale wind, solar, and thermal power prediction device based on a large language model according to claim 5, characterized in that, The prediction module includes: The fine-tuning unit is used to efficiently fine-tune the parameters of the self-attention query matrix, key matrix and value matrix in the pre-trained large language model using a low-rank adaptation fine-tuning method, while keeping the remaining parameters of the pre-trained large language model frozen, so as to obtain a large language model predictor suitable for multi-energy power prediction tasks of wind, solar and thermal storage. The prediction unit is used to input the historical power data of wind, solar and thermal energy storage and multi-energy ...

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-scale wind, solar, and thermal energy storage power prediction method based on a large language model as described in any one of claims 1-4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the multi-scale wind, solar and thermal power prediction method based on a large language model as described in any one of claims 1-4.

Images