Power grid load prediction method, device and equipment based on fusion of large language model fine tuning and traditional time sequence analysis, and medium

Abstract

The invention discloses a power grid load prediction method, device and equipment based on fusion of large language model fine tuning and traditional time sequence analysis, and a medium. The method comprises the steps of generating multi-dimensional original features; obtaining a first load prediction value; obtaining a large language fine tuning model and generating a second load prediction value; splicing the multi-dimensional original features, the first load prediction value and the second load prediction value to obtain a fusion vector, and generating a plurality of third load prediction values based on a fusion model; determining a load prediction mean value, a load prediction variance and a load prediction confidence interval according to the plurality of third load prediction values; jointly updating the fusion model, the traditional time sequence prediction model and the large language fine tuning model; and generating prediction information of the power grid based on the jointly updated large language fine tuning model, the load prediction mean value, the load prediction variance and the load prediction confidence interval. The invention belongs to the field of power grid load prediction. The power grid load prediction precision can be improved.

Description

Technical Field

[0001] The present invention relates to the field of power grid load forecasting, and in particular to a power grid load forecasting method, apparatus, device and medium based on the fusion of large language model fine-tuning and traditional time series analysis. Background Art

[0002] Load forecasting is a core component of power grid planning and operation. Traditional time series methods (such as ARIMA, regression models, and LSTM) rely primarily on historical load data sequences and are effective at learning temporal dependencies and seasonal patterns. However, purely linear or single-depth models often struggle to accurately capture load fluctuations when new energy sources are integrated, electricity demand fluctuates, and emergencies occur.

[0003] Large language models offer advantages in understanding text and capturing contextual semantic information. However, directly outputting numerical predictions often suffers from large errors and poor generalization. Improving prediction accuracy by integrating the numerical accuracy of traditional time series predictions with the semantic understanding capabilities of large language models is a pressing issue. Summary of the Invention

[0004] The present invention solves the technical problem of low grid load prediction accuracy in the prior art by providing a grid load prediction method, device, equipment and medium based on the fusion of large language model fine-tuning and traditional time series analysis, and achieves the technical effect of improving the grid load prediction accuracy.

[0005] In a first aspect, the present invention provides a power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis, comprising:

[0006] Preprocess the multi-source data of the power grid to generate multi-dimensional original features;

[0007] Modeling historical load data based on a traditional time series forecasting model to obtain a first load forecast value;

[0008] Fine-tune the large language model based on LoRA to obtain a large language fine-tuning model and generate a second load prediction value;

[0009] The multi-dimensional original features, the first load prediction value, and the second load prediction value are concatenated to obtain a fusion vector, and a plurality of third load prediction values ​​are generated based on the fusion model;

[0010] determining a load forecast mean, a load forecast variance, and a load forecast confidence interval based on the plurality of third load forecast values;

[0011] Jointly update the fusion model, traditional time series prediction model, and large language fine-tuning model;

[0012] The prediction information of the power grid is generated based on the jointly updated large language fine-tuning model, load forecast mean, load forecast variance and load forecast confidence interval.

[0013] Furthermore, the multi-source data of the power grid is preprocessed to generate multi-dimensional original features, including:

[0014] Historical load data, historical meteorological data, historical holiday data, historical event data and geographic information data are normalized, denoised and interpolated to obtain the multi-dimensional original characteristics of the power grid, where the multi-source data of the power grid are represented in the form of natural language text.

[0015] Furthermore, fine-tuning the large language model based on LoRA to obtain a large language fine-tuning model and generate a second load prediction value includes:

[0016] The fixed pre-trained weights of the large language model are retained, and low-rank matrix factorization terms are introduced to fine-tune the large language model, including:

[0017] W=W0+BA

[0018] Where W is the weight matrix of the large language model, W0 is the fixed pre-training weight of the large language model, and B and A are both low-rank matrix decomposition items;

[0019] Based on historical load data and multi-source data in the form of natural language text, the large language model is fine-tuned again to obtain a large language fine-tuned model and output to generate a second load prediction value.

[0020] Furthermore, the multi-dimensional original features, the first load prediction value, and the second load prediction value are concatenated to obtain a fusion vector, and a plurality of third load prediction values ​​are generated based on the fusion model, including:

[0021]

[0022] Among them, z t is the fusion vector, is the multidimensional original feature, is the first load forecast value, is the second load forecast value, R d is a d-dimensional real vector space, where d = d0 + 2 and d0 is Dimensions;

[0023] Construct a fusion model, which is a multi-layer nonlinear neural network, including an input layer and a feature extraction layer:

[0024]

[0025] in, is the output of the first hidden layer, φ is the nonlinear activation function, W1 is the weight matrix of the first layer, is a matrix space with d1 rows and d columns, is a real vector space of dimension d1, b1 is the bias of the first layer;

[0026] It also includes multiple hidden layers, where the multiple hidden layers are used to perform layer-by-layer nonlinear mapping on the input, including:

[0027]

[0028] in, is the output of the lth hidden layer, W l is the weight matrix of the lth layer, is the output of the l-1th hidden layer, b l is the bias of the lth layer, k is the number of biases;

[0029] The method further includes a multiple candidate output layer, wherein the multiple candidate output layer is used to output and generate multiple third load prediction values, including:

[0030]

[0031] in, is the third load forecast value of i at time step t, is the output of the kth hidden layer, is the weight matrix corresponding to the third load forecast value of i, is the bias corresponding to the i-th third load prediction value, and M is the number of third load prediction values.

[0032] Furthermore, a load forecast mean, a load forecast variance, and a load forecast confidence interval are determined based on the plurality of third load forecast values, including:

[0033] Determining a load forecast mean value according to the plurality of third load forecast values ​​includes:

[0034]

[0035] Determining a load forecast variance based on a plurality of third load forecast values ​​includes:

[0036]

[0037] Determine the load forecast confidence interval based on the load forecast mean and load forecast variance, including:

[0038]

[0039] in, is the load forecast mean, σ t is the load forecast variance.

[0040] Furthermore, the fusion model, traditional time series prediction model, and large language fine-tuning model are jointly updated, including:

[0041] The parameters of the fusion model are updated using incremental gradient descent, including:

[0042]

[0043] Among them, θ is the parameter of the fusion model, η is the learning rate, L fusion is the loss training function of the fusion model, F θ is the fusion model, z t is the fusion vector, y t For historical load;

[0044] When the traditional time series forecasting model adopts the ARIMA model, unused multi-source data of the power grid is obtained to update the ARIMA model; when the traditional time series forecasting model adopts the LSTM model, a sliding window or periodic retraining is used to update the LSTM model;

[0045] Update the low-rank matrix factorization term of the large language fine-tuning model based on LoRA to update the large language fine-tuning model, including:

[0046]

[0047] Among them, B and A are low-rank matrix decomposition items, η LLM The learning rate for fine-tuning the model for large languages, is the gradient of the low-rank matrix decomposition term A, is the gradient of the low-rank matrix decomposition term B, L LLM Loss function for fine-tuning models on large languages.

[0048] Furthermore, the fusion model, traditional time series prediction model, and large language fine-tuning model are jointly updated, including:

[0049] The loss function of the fusion model, the traditional time series prediction model, and the large language fine-tuning model are jointly updated, including:

[0050] L total =λ1L TS +λ2L LLM +λ3L fusion

[0051] Among them, L total is the total loss function, λ1, λ2, λ3 are weights, L TS is the loss function of the traditional time series prediction model.

[0052] In a second aspect, the present invention provides a power grid load forecasting device based on the fusion of large language model fine-tuning and traditional time series analysis, comprising:

[0053] The preprocessing module is used to preprocess the multi-source data of the power grid and generate multi-dimensional original features;

[0054] A first prediction module is used to model historical load data based on a traditional time series prediction model to obtain a first load prediction value;

[0055] A second prediction module is used to fine-tune the large language model based on LoRA to obtain a large language fine-tuning model and generate a second load prediction value;

[0056] a third prediction module, configured to concatenate the multi-dimensional original features, the first load prediction value, and the second load prediction value to obtain a fusion vector, and generate a plurality of third load prediction values ​​based on the fusion model;

[0057] A load information determination module, configured to determine a load prediction mean, a load prediction variance, and a load prediction confidence interval based on a plurality of third load prediction values;

[0058] Joint update module, used to jointly update the fusion model, traditional time series prediction model, and large language fine-tuning model;

[0059] The prediction information module is used to generate prediction information of the power grid based on the jointly updated large language fine-tuning model, load prediction mean, load prediction variance and load prediction confidence interval.

[0060] In a third aspect, the present invention provides an electronic device, comprising:

[0061] processor;

[0062] a memory for storing processor-executable instructions;

[0063] The processor is configured to execute to implement the power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis as provided in the first aspect.

[0064] In a fourth aspect, the present invention provides a non-temporary computer-readable storage medium. When the instructions in the non-temporary computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to implement the power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis as provided in the first aspect.

[0065] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0066] This solution of the present invention designs a fusion model and utilizes a multi-layer nonlinear neural network structure to combine multidimensional input features (time series, meteorological, event, geographical, and text, etc.), traditional numerical prediction results, and fine-tuned LLM prediction values ​​into a unified vector. After inputting into the fusion model, multiple candidate predictions are output, and the prediction mean and confidence interval are given through statistical calculation. At the same time, a joint incremental learning strategy based on online data feedback is proposed to perform real-time parameter updates on the traditional time series model, the LLM fine-tuning part, and the fusion model itself to adapt to the dynamic changes in power grid load. In addition, the fine-tuned LLM can automatically generate natural language reports containing prediction results, risk warnings, and scheduling recommendations, facilitating power grid scheduling decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 A flow chart of a power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis provided by the present invention;

[0069] Figure 2 A flow chart of another power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis provided by the present invention;

[0070] Figure 3 This is a structural diagram of the power grid load forecasting device provided by the present invention based on the fusion of large language model fine-tuning and traditional time series analysis. DETAILED DESCRIPTION

[0071] The embodiment of the present invention solves the technical problem of low grid load prediction accuracy in the prior art by providing a grid load prediction method based on the fusion of large language model fine-tuning and traditional time series analysis.

[0072] The technical solution of the present invention is to solve the above technical problems, and the overall idea is as follows:

[0073] A power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis includes: preprocessing multi-source data of the power grid to generate multi-dimensional original features; modeling historical load data based on the traditional time series forecasting model to obtain a first load forecast value; fine-tuning the large language model based on LoRA to obtain a large language fine-tuning model and generate a second load forecast value; splicing the multi-dimensional original features, the first load forecast value and the second load forecast value to obtain a fusion vector, and generating several third load forecast values ​​based on the fusion model; determining the load forecast mean, load forecast variance and load forecast confidence interval according to the several third load forecast values, and jointly updating the fusion model, the traditional time series forecasting model and the large language fine-tuning model; generating power grid forecast information based on the jointly updated large language fine-tuning model, load forecast mean, load forecast variance and load forecast confidence interval.

[0074] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0075] First, the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0076] The present invention provides Figure 1 The power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis includes steps S11-S17:

[0077] Step S11 : pre-processing the multi-source data of the power grid to generate multi-dimensional original features.

[0078] Specifically, it includes normalizing, denoising and interpolating historical load data, historical meteorological data, historical holiday data, historical event data and geographic information data to obtain the multi-dimensional original characteristics of the power grid, where the multi-source data of the power grid is represented in the form of natural language text.

[0079] There are data from various sources in the power grid system, and these data reflect the operating status of the power grid and external influencing factors from different perspectives. Historical load data refers to the record of changes in the power consumption or power of the power system over a certain period of time in the past; historical meteorological data refers to the historical records of weather conditions such as temperature, humidity, wind speed, and precipitation, which affect power consumption behavior. Historical holiday data refers to the identification of whether a certain day is a holiday, weekend, or special holiday. Historical event data refers to records of events such as equipment maintenance, failures, power outages, and scheduling plans. Geographic information data refers to information such as the geographical location, topography, and surrounding environment of power grid nodes (such as substations and load centers). Data can be structured (tables), semi-structured (JSON, XML), or unstructured (such as natural language text).

[0080] Raw data usually has the following problems: inconsistent data format, missing values, outliers, inconsistent units, noise, etc.

[0081] Therefore, for subsequent modeling and analysis, data of different dimensions are scaled to the same numerical range, outliers or random fluctuations in the data are removed to make the data more stable and reliable, and missing data points are filled to make the time series complete.

[0082] At the same time, multi-source data is converted into natural language descriptions, such as "At 2:00 PM on March 24, 2025, the temperature was 24°C, the humidity was 60%, it was not a holiday, and the current load was 500 MW; the loads in the past three hours were 480 MW, 500 MW, and 520 MW, respectively."

[0083] Step S12: Modeling the historical load data based on a traditional time series forecasting model to obtain a first load forecast value.

[0084] Traditional time series models are methods specifically designed for modeling and forecasting time series data. They are typically based on time-dependent modeling, where current values ​​are related to values ​​over a period of time. In power grid load forecasting, historical load data can be used to learn its changing patterns and predict future loads.

[0085] Traditional time series prediction models can use ARIMA (Autoregressive Integrated Moving Average Model), regression models or LSTM (Deep Time Series Network).

[0086] ARIMA is a statistical time series model that can handle non-stationary time series (making them stationary through differencing); regression models can include linear regression, multivariate regression, etc.; LSTM is an improved version of RNN (recurrent neural network) that can capture long-term dependencies and is suitable for time series with periodicity and trends such as load forecasting.

[0087] By inputting historical load data into the traditional time series forecasting model, the corresponding load forecast value can be obtained.

[0088] Step S13: Fine-tune the large language model based on LoRA to obtain a large language fine-tuning model and generate a second load prediction value.

[0089] A large language model (LLM) refers to a neural network model with a large number of parameters (such as billions, tens of billions, or even more), such as GPT, BERT, LLaMA, etc. It is usually pre-trained on a large corpus and then fine-tuned according to specific tasks.

[0090] Specifically, we retain the fixed pre-trained weights of the large language model and introduce low-rank matrix factorization terms to fine-tune the large language model, including:

[0091] W=W0+BA

[0092] Where W is the weight matrix of the large language model, W0 is the fixed pre-training weight of the large language model, and B and A are both low-rank matrix decomposition items;

[0093] Based on historical load data and multi-source data in the form of natural language text, the large language model is fine-tuned again to obtain a large language fine-tuned model and output to generate a second load prediction value.

[0094] For large language models, LoRA technology is used to retain the main parameters of W0 while only updating A and B. This ensures training efficiency while retaining most of the model's learned semantic capabilities, thereby producing a second load prediction value.

[0095] In step S14, the multi-dimensional original features, the first load prediction value, and the second load prediction value are concatenated to obtain a fusion vector, and a plurality of third load prediction values ​​are generated based on the fusion model.

[0096] Specifically include:

[0097]

[0098] Among them, z t is the fusion vector, is the multidimensional original feature, is the first load forecast value, is the second load forecast value, R d is a d-dimensional real vector space, where d = d0 + 2 and d0 is Dimensions;

[0099] Construct a fusion model, which is a multi-layer nonlinear neural network, including an input layer and a feature extraction layer:

[0100]

[0101] in, is the output of the first hidden layer, φ is the nonlinear activation function, W1 is the weight matrix of the first layer, is a matrix space with d1 rows and d columns, is a real vector space of dimension d1, b1 is the bias of the first layer;

[0102] It also includes multiple hidden layers, where the multiple hidden layers are used to perform layer-by-layer nonlinear mapping on the input, including:

[0103]

[0104] in, is the output of the lth hidden layer, W l is the weight matrix of the lth layer, is the output of the l-1th hidden layer, b l is the bias of the lth layer, k is the number of biases;

[0105] The method further includes a multiple candidate output layer, wherein the multiple candidate output layer is used to output and generate multiple third load prediction values, including:

[0106]

[0107] in, is the third load forecast value of i at time step t, is the output of the kth hidden layer, is the weight matrix corresponding to the third load forecast value of i, is the bias corresponding to the i-th third load prediction value, and M is the number of third load prediction values.

[0108] Step S15: determining a load prediction mean, a load prediction variance, and a load prediction confidence interval according to a plurality of third load prediction values.

[0109] Specifically include:

[0110] Determining a load forecast mean value according to the plurality of third load forecast values ​​includes:

[0111]

[0112] Determining a load forecast variance based on a plurality of third load forecast values ​​includes:

[0113]

[0114] Determine the load forecast confidence interval based on the load forecast mean and load forecast variance, including:

[0115]

[0116] in, is the load forecast mean, σ t is the load forecast variance.

[0117] The method of outputting multiple third-party load forecast values ​​has both forecast accuracy and uncertainty assessment capabilities.

[0118] Step S16: jointly update the fusion model, the traditional time series prediction model, and the large language fine-tuning model.

[0119] In order to ensure the system's adaptability to the time-varying characteristics of the grid load, the model will be updated online incrementally after deployment.

[0120] Specifically include:

[0121] By real-time acquisition of unused multi-source data and corresponding input vector z t , construct small batch training samples, and use incremental gradient descent to update the parameters of the fusion model, including:

[0122]

[0123] Among them, θ is the parameter of the fusion model, η is the learning rate, L fusion is the loss training function of the fusion model, F θ is the fusion model, z t is the fusion vector, y t For historical load;

[0124] A sliding window technique may also be used to periodically retrain unused multi-source data (the unused multi-source data may all be the latest data) to prevent parameter drift.

[0125] When the traditional time series forecasting model adopts the ARIMA model, unused multi-source data of the power grid is obtained to update the ARIMA model; when the traditional time series forecasting model adopts the LSTM model, a sliding window or periodic retraining is used to update the LSTM model to ensure that the latest time series dynamics are captured.

[0126] Update the low-rank matrix factorization term of the large language fine-tuning model based on LoRA to update the large language fine-tuning model, including:

[0127]

[0128] Among them, B and A are low-rank matrix decomposition items, η LLM is the learning rate for fine-tuning the model for large languages, is the gradient of the low-rank matrix decomposition term A, is the gradient of the low-rank matrix decomposition term B, L LLM Loss function for fine-tuning models on large languages.

[0129] Also includes:

[0130] The loss function of the fusion model, the traditional time series prediction model, and the large language fine-tuning model are jointly updated, including:

[0131] L total =λ1L TS +λ2L LLM +λ3L fusion

[0132] Among them, L total is the total loss function, λ1, λ2, λ3 are weights, L TS is the loss function of the traditional time series prediction model.

[0133] Step S17: Generate prediction information of the power grid based on the jointly updated large language fine-tuning model, the load prediction mean, the load prediction variance, and the load prediction confidence interval.

[0134] By combining the large language fine-tuning model with the load forecast mean, confidence interval and feature importance output by the fusion model, a natural language report (i.e., forecast information) containing forecast trends, risk warnings and scheduling recommendations is automatically generated and displayed through a visualization module to facilitate decision-making by power dispatchers and managers. The power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis provided by the present invention can also be referred to. Figure 2 .

[0135] In summary, the present invention provides a power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis, including: preprocessing multi-source data of the power grid to generate multi-dimensional original features; modeling historical load data based on the traditional time series prediction model to obtain a first load prediction value; fine-tuning the large language model based on LoRA to obtain a large language fine-tuning model and generate a second load prediction value; splicing the multi-dimensional original features, the first load prediction value and the second load prediction value to obtain a fusion vector, and generating several third load prediction values ​​based on the fusion model; determining the load prediction mean, load prediction variance and load prediction confidence interval based on the several third load prediction values; jointly updating the fusion model, the traditional time series prediction model and the large language fine-tuning model; generating power grid prediction information based on the jointly updated large language fine-tuning model, load prediction mean, load prediction variance and load prediction confidence interval. This solution of the present invention designs a fusion model and utilizes a multi-layer nonlinear neural network structure to combine multidimensional input features (time series, meteorological, event, geographical, and text, etc.), traditional numerical prediction results, and fine-tuned LLM prediction values ​​into a unified vector. After inputting into the fusion model, multiple candidate predictions are output, and the prediction mean and confidence interval are given through statistical calculation. At the same time, a joint incremental learning strategy based on online data feedback is proposed to perform real-time parameter updates on the traditional time series model, the LLM fine-tuning part, and the fusion model itself to adapt to the dynamic changes in power grid load. In addition, the fine-tuned LLM can automatically generate natural language reports containing prediction results, risk warnings, and scheduling recommendations, facilitating power grid scheduling decisions.

[0136] Based on the same inventive concept, the present invention provides Figure 3 The power grid load forecasting device shown here, based on the fusion of large language model fine-tuning and traditional time series analysis, includes:

[0137] A preprocessing module 31 is used to preprocess the multi-source data of the power grid and generate multi-dimensional original features;

[0138] A first prediction module 32 is configured to model historical load data based on a traditional time series prediction model to obtain a first load prediction value;

[0139] A second prediction module 33 is configured to fine-tune the large language model based on LoRA to obtain a large language fine-tuning model and generate a second load prediction value;

[0140] The third prediction module 34 is configured to concatenate the multi-dimensional original features, the first load prediction value, and the second load prediction value to obtain a fusion vector, and generate a plurality of third load prediction values ​​based on the fusion model;

[0141] A load information determination module 35 is configured to determine a load prediction mean, a load prediction variance, and a load prediction confidence interval based on a plurality of third load prediction values;

[0142] A joint updating module 36 is used to jointly update the fusion model, the traditional time series prediction model, and the large language fine-tuning model;

[0143] The prediction information module 37 is used to generate prediction information of the power grid based on the jointly updated large language fine-tuning model, the load prediction mean, the load prediction variance and the load prediction confidence interval.

[0144] Based on the same inventive concept, the present invention further provides an electronic device, comprising:

[0145] processor;

[0146] a memory for storing processor-executable instructions;

[0147] The processor is configured to execute to implement the power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis as provided above.

[0148] Based on the same inventive concept, the present invention also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of an electronic device, the electronic device is enabled to execute the power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis as provided above.

[0149] Since the electronic device described in this embodiment is an electronic device used to implement the information processing method in the embodiment of the present invention, based on the information processing method described in the embodiment of the present invention, those skilled in the art will be able to understand the specific implementation of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present invention will not be described in detail here. As long as the electronic device used by those skilled in the art to implement the information processing method in the embodiment of the present invention falls within the scope of protection of the present invention.

[0150] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0154] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0155] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis, characterized by: include: Preprocess the multi-source data of the power grid to generate multi-dimensional original features; Modeling historical load data based on a traditional time series forecasting model to obtain a first load forecast value; Fine-tune the large language model based on LoRA to obtain a large language fine-tuning model and generate a second load prediction value; The multi-dimensional original features, the first load prediction value, and the second load prediction value are concatenated to obtain a fusion vector, and a plurality of third load prediction values ​​are generated based on the fusion model; determining a load forecast mean, a load forecast variance, and a load forecast confidence interval based on the plurality of third load forecast values; Jointly update the fusion model, traditional time series prediction model, and large language fine-tuning model; Prediction information of the power grid is generated based on the jointly updated large language fine-tuning model, the load prediction mean, the load prediction variance, and the load prediction confidence interval.

2. The power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis according to claim 1 is characterized in that: Preprocess the multi-source data of the power grid to generate multi-dimensional original features, including: Historical load data, historical meteorological data, historical holiday data, historical event data and geographic information data are normalized, denoised and interpolated to obtain the multi-dimensional original characteristics of the power grid, where the multi-source data of the power grid are represented in the form of natural language text.

3. The power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis according to claim 2 is characterized in that: Fine-tune the large language model based on LoRA to obtain the large language fine-tuned model and generate a second load prediction value, including: The fixed pre-trained weights of the large language model are retained, and a low-rank matrix factorization term is introduced to fine-tune the large language model, including: W=W0+BA Where W is the weight matrix of the large language model, W0 is the fixed pre-training weight of the large language model, and B and A are both low-rank matrix decomposition items; Based on historical load data and multi-source data in the form of natural language text, the large language model is fine-tuned again to obtain a large language fine-tuned model and output to generate a second load prediction value.

4. The power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis according to claim 3 is characterized in that: The multi-dimensional original features, the first load prediction value, and the second load prediction value are concatenated to obtain a fusion vector, and a plurality of third load prediction values ​​are generated based on the fusion model, including: Among them, z d is the fusion vector, is the multidimensional original feature, is the first load forecast value, is the second load forecast value, R t is a d-dimensional real vector space, where d = d0 + 2 and d0 is Dimensions; Construct a fusion model, which is a multi-layer nonlinear neural network, including an input layer and a feature extraction layer: in, is the output of the first hidden layer, φ is the nonlinear activation function, W1 is the weight matrix of the first layer, is a matrix space with d1 rows and d columns, is a real vector space of dimension d1, b1 is the bias of the first layer; It also includes multiple hidden layers, where the multiple hidden layers are used to perform layer-by-layer nonlinear mapping on the input, including: in, is the output of the lth hidden layer, W l is the weight matrix of the lth layer, is the output of the l-1th hidden layer, b l is the bias of the lth layer, k is the number of biases; The method further includes a multiple candidate output layer, wherein the multiple candidate output layer is used to output and generate multiple third load prediction values, including: in, is the third load forecast value of i at time step t, is the output of the kth hidden layer, W i (out) is the weight matrix corresponding to the third load forecast value of i, is the bias corresponding to the i-th third load prediction value, and M is the number of third load prediction values.

5. The power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis according to claim 4 is characterized in that: Determining a load forecast mean, a load forecast variance, and a load forecast confidence interval based on the plurality of third load forecast values ​​includes: Determining a load forecast mean value according to the plurality of third load forecast values ​​includes: Determining a load forecast variance based on a plurality of third load forecast values ​​includes: Determine the load forecast confidence interval based on the load forecast mean and load forecast variance, including: in, is the load forecast mean, σ t is the load forecast variance.

6. The power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis according to claim 1 is characterized in that: The fusion model, traditional time series prediction model, and large language fine-tuning model are jointly updated, including: The parameters of the fusion model are updated using incremental gradient descent, including: Among them, θ is the parameter of the fusion model, η is the learning rate, L fusion is the loss training function of the fusion model, F θ is the fusion model, z t is the fusion vector, y t For historical load; When the traditional time series forecasting model adopts the ARIMA model, unused multi-source data of the power grid is obtained to update the ARIMA model; when the traditional time series forecasting model adopts the LSTM model, a sliding window or periodic retraining is used to update the LSTM model; Update the low-rank matrix factorization term of the large language fine-tuning model based on LoRA to update the large language fine-tuning model, including: Among them, B and A are low-rank matrix decomposition items, η LLM The learning rate for fine-tuning the model for large languages, is the gradient of the low-rank matrix decomposition term A, is the gradient of the low-rank matrix decomposition term B, L LLM Loss function for fine-tuning models on large languages.

7. The power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis according to claim 6 is characterized in that: The fusion model, traditional time series prediction model, and large language fine-tuning model are jointly updated, including: The loss function of the fusion model, the traditional time series prediction model, and the large language fine-tuning model are jointly updated, including: L total =λ1L TS +λ2L LLM +λ3L fusion Among them, L total is the total loss function, λ1, λ2, λ3 are weights, L TS is the loss function of the traditional time series prediction model.

8. A power grid load forecasting device based on the fusion of large language model fine-tuning and traditional time series analysis, characterized by: include: The preprocessing module is used to preprocess the multi-source data of the power grid and generate multi-dimensional original features; A first prediction module is used to model historical load data based on a traditional time series prediction model to obtain a first load prediction value; A second prediction module is used to fine-tune the large language model based on LoRA to obtain a large language fine-tuning model and generate a second load prediction value; a third prediction module, configured to concatenate the multi-dimensional original features, the first load prediction value, and the second load prediction value to obtain a fusion vector, and generate a plurality of third load prediction values ​​based on the fusion model; A load information determination module, configured to determine a load prediction mean, a load prediction variance, and a load prediction confidence interval based on a plurality of third load prediction values; Joint update module, used to jointly update the fusion model, traditional time series prediction model, and large language fine-tuning model; A prediction information module is used to generate prediction information of the power grid based on the jointly updated large language fine-tuning model, the load prediction mean, the load prediction variance, and the load prediction confidence interval.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute to implement the power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that When the instructions in the non-transitory computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to implement the power grid load forecasting method based on the fusion of large language model fine-tuning and traditional time series analysis as described in any one of claims 1 to 7.

Images