A method for power demand forecasting based on large language model

Abstract

The invention discloses a method for predicting electricity demand based on a large language model, which belongs to the technical field of electricity prediction, and specifically comprises: collecting and filtering electricity demand data; dividing the electricity demand data into a training set, a validation set and a test set; setting seasonal parameters to integrate the electricity demand data according to time sequence; jointly constructing a large language electricity model based on CNN, BERT and LSTM, setting model parameters, training the data of the training set, and using numerical electricity data as labels, calculating the root mean square error between the electricity demand trend data output by the model and the label as a loss function, calculating the gradient of the loss function relative to the model parameters, and updating the parameters, and repeating the above process for multiple cycles until convergence; using the validation set to verify the performance of the model, and using an independent test set to evaluate the final performance of the model, and calculating the model error; the invention provides effective technical support for the optimal scheduling of the power system.

Description

Technical Field

[0001] The present invention relates to the technical field of power forecasting, and in particular to a method for forecasting power demand based on a large language model. Background Art

[0002] Electricity demand forecasting is an important part of power system planning and operation, and it is of great significance to ensure the stability and economy of power supply. Traditional electricity demand forecasting methods mainly rely on statistical methods such as time series analysis and regression analysis. These methods have certain effects when dealing with linear relationships and simple nonlinear relationships, but when faced with complex and changeable electricity demand data, their prediction accuracy is often difficult to meet actual needs.

[0003] With the development of big data and artificial intelligence technology, power demand forecasting methods based on large language models have gradually become a research hotspot. This method can better capture the complex characteristics and changing patterns of power demand by utilizing large-scale text data and combining it with deep learning technology. However, the existing power demand forecasting methods based on large language models still have some problems and challenges.

[0004] Traditional electricity demand forecasting methods mainly rely on statistical methods such as time series analysis and regression analysis. Although these methods can capture the cyclical changes in electricity demand to a certain extent, they often ignore the impact of unstructured information such as meteorological, socio-economic and text data on electricity demand, resulting in limited prediction accuracy. Summary of the invention

[0005] The purpose of the present invention is to provide a method for predicting power demand based on a large language model to solve the following technical problems:

[0006] Traditional electricity demand forecasting methods ignore the impact of unstructured information such as meteorological, socio-economic and text data on electricity demand, resulting in limited forecasting accuracy.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A method for predicting power demand based on a large language model comprises the following steps:

[0009] Collecting power demand data, the power demand data including numerical power data, meteorological data, socio-economic data and text data, and filtering the power demand data;

[0010] Mapping the text data into text vectors, encoding and sorting the text vectors through position embedding, and outputting power text features through a text extraction model; dividing the power demand data processed above into a training set, a validation set, and a test set; setting seasonal parameters and integrating the power demand data according to time sequence through an ARIMA model;

[0011] A large language power model is jointly constructed based on CNN, BERT and LSTM, model parameters are set, the data of the training set is input into the large language power model for training, and numerical power data is used as labels. The root mean square error between the power demand trend data output by the model and the label is calculated as the loss function, the gradient of the loss function relative to the model parameters is calculated, and the parameters are updated. The above process is repeated for multiple cycles until convergence;

[0012] Use the validation set to verify the performance of the model until the model performance no longer improves, and use an independent test set to evaluate the final performance of the model and calculate the model error.

[0013] As a further solution of the present invention: the numerical power data includes power consumption records of the past several years, arranged according to hours, days, weeks and months; the meteorological data includes temperature, humidity, wind speed and precipitation in the selected area; the socio-economic data includes population data, industrial data, holidays, and major event information in the selected area; the text data includes social media comments, news reports and user feedback.

[0014] As a further solution of the present invention: the filtering process is:

[0015] Outliers are removed by quartile method, missing values ​​are supplemented by interpolation method, and data from different data sources are normalized. The formula is:

[0016] ;

[0017] Z represents any numerical data, Z0 represents the normalized value of the numerical data, and Z max and Z min Indicates the maximum and minimum values ​​of the same type of numerical data.

[0018] As a further solution of the present invention: the process of outputting power text features through the text extraction model is:

[0019] The text data is segmented by a standard dictionary, and the text data is split into several words. The text extraction model includes a BERT layer, a BiLSTM layer and a CRF layer. The words are input into BERT and mapped into word vectors. The word vectors are input into the BiLSTM layer, and the context feature information of the word vectors is deeply learned. The probability of each text information corresponding to a label is output, and an entity sequence is generated. The entity sequence is input into the CRF layer for sorting to obtain an entity sequence that optimizes the objective function; and the optimized entity sequence is screened based on a preset power field dictionary to screen out power text features.

[0020] As a further solution of the present invention: sorting the power demand data according to time sequence to generate a time series of power demand data, treating the matrix formed by the time series as an image, and inputting the image into the BERT network of the large language power model in sequence after convolution through a CNN network. The BERT network includes several Transformer layers, each Transformer layer includes several filters. Whenever the matrix image passes through a Transformer layer, the filter slides and scans the matrix and generates a multidimensional array, reducing the size of the matrix image and increasing the dimension of the matrix image. The increased dimension is the same as the number of filters in the Transformer layer. Each Transformer layer reduces the image size in turn and increases the dimension of the matrix image. The multidimensional array finally generated in BERT is flattened and converted into a vector, and the vector is used as the input of LSTM.

[0021] As a further solution of the present invention: the formula for extracting power demand characteristics by the CNN network and the BERT network is:

[0022] ;

[0023] Among them, K is the convolution kernel, B is the bias term, σ is the activation function ReLU, which performs nonlinear mapping on the convolution result, and P ij is the matrix image obtained after convolution, i and j represent the position index in the matrix image, where i represents the row index, j represents the column index, X represents the input data, and m and n represent the elements in the input data that are offset by m rows and n columns relative to the current position (i, j).

[0024] As a further solution of the present invention: the dimension of the matrix image output by BERT remains unchanged, the matrix image is flattened in the time dimension and feature vector, a new matrix is ​​generated from the final multidimensional arrays of all Transformer layers, the new matrix vector is used as the input of LSTM, the flattened vector generated by the flattening is used as the time dimension of LSTM, the new matrix is ​​trained by the time dimension and feature vector, and the corresponding relationship between the power demand trend and time or season is output.

[0025] As a further solution of the present invention: the time scale of each type of electricity demand data is the same, and weights are set for the number of samples in each quarter or year based on their relationship with the current time from near to far, and the weights are distributed in order from near to far.

[0026] As a further solution of the present invention: the integrated power demand data includes a month-on-month time series and a year-on-year time series, and the trends of the power demand data of the two series are trained respectively.

[0027] Beneficial effects of the present invention:

[0028] The present invention integrates numerical power data, meteorological data, socio-economic data and text data, achieves comprehensive coverage of factors affecting power demand, and improves the generalization ability and accuracy of the prediction model; adopts a text extraction model combining BERT, BiLSTM and CRF, effectively solves the problem of unstructured text data, improves the efficiency and quality of text feature extraction, enables the model to make full use of implicit knowledge in text information such as social media comments and news reports, and enhances the depth and breadth of prediction; combines CNN, BERT and LSTM to build a large language power model, and utilizes the spatial feature extraction ability of CNN and the time series extraction ability of LSTM. The advantages of column analysis are integrated with BERT's contextual understanding ability to form a powerful sequence-to-sequence prediction framework, which can better understand and predict the complex patterns of electricity demand, especially when dealing with seasonal fluctuations and the impact of emergencies; by setting seasonal parameters and applying the ARIMA model to integrate electricity demand data, it can flexibly adapt to the characteristics of electricity consumption in different seasons, and improve the model's sensitivity and response speed to seasonal changes; setting sample weights according to the relationship from near to far in time reflects the principle of data timeliness, so that the model pays more attention to the impact of recent data, while taking into account historical trends, and balancing the prediction needs of short-term fluctuations and long-term trends. A complete model evaluation system has been established to ensure the effectiveness of the model training process and the objective evaluation of the final prediction performance; in summary, the present invention provides strong technical support for the optimal scheduling and resource planning of power systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be further described below in conjunction with the accompanying drawings.

[0030] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] See also Figure 1 As shown, the present invention is a method for predicting power demand based on a large language model, comprising the following steps:

[0033] Collecting power demand data, the power demand data including numerical power data, meteorological data, socio-economic data and text data, and filtering the power demand data;

[0034] Mapping the text data into text vectors, encoding and sorting the text vectors through position embedding, and outputting power text features through a text extraction model; dividing the power demand data processed above into a training set, a validation set, and a test set; setting seasonal parameters and integrating the power demand data according to time sequence through an ARIMA model;

[0035] A large language power model is jointly constructed based on CNN, BERT and LSTM, model parameters are set, the data of the training set is input into the large language power model for training, and numerical power data is used as labels. The root mean square error between the power demand trend data output by the model and the label is calculated as the loss function, the gradient of the loss function relative to the model parameters is calculated, and the parameters are updated. The above process is repeated for multiple cycles until convergence;

[0036] Use the validation set to verify the performance of the model until the model performance no longer improves, and use an independent test set to evaluate the final performance of the model and calculate the model error.

[0037] The present invention proposes a method for predicting power demand based on a large language model. The method collects multi-source heterogeneous power demand data, including numerical power data, meteorological data, socio-economic data and text data, and filters and integrates them; then, CNN, BERT and LSTM are used to jointly construct a large language power model, which is trained by the data of the training set, and the numerical power data is used as the label, and the root mean square error between the power demand trend data output by the model and the label is calculated as the loss function, and the gradient of the loss function relative to the model parameters is calculated, and the parameters are updated, and the above process is repeated for multiple cycles until convergence; finally, the performance of the model is evaluated using the validation set until the model performance is no longer improved, and the final performance of the model is evaluated using an independent test set, and the model error is calculated. This comprehensive method can not only capture the various factors affecting power demand more comprehensively, but also deeply mine the potential patterns in the data through deep learning technology, especially the complex information of text data, so as to provide more accurate and reliable prediction results. In addition, the introduction of seasonal parameters and the integration of time series further enhance the model's adaptability to seasonal changes and long-term trends, making the prediction sensitive to short-term fluctuations and in line with long-term trends, providing strong technical support for the optimal scheduling and resource planning of power systems.

[0038] In a preferred embodiment of the present invention, the numerical power data includes power consumption records of the past several years, arranged according to hours, days, weeks and months; the meteorological data includes temperature, humidity, wind speed and precipitation in a selected area; the socio-economic data includes population data, industrial data, holidays, and major event information in a selected area; and the text data includes social media comments, news reports and user feedback.

[0039] By introducing meteorological data (including temperature, humidity, wind speed, and precipitation, etc.) and analyzing the correlation between these data and electricity demand, the accuracy of electricity demand forecasting has been significantly improved. Meteorological factors have a direct impact on electricity demand. For example, hot weather may lead to increased use of air conditioners, thereby increasing electricity consumption. By incorporating meteorological data into the forecasting model, the impact of changes in meteorological conditions on electricity demand can be more accurately reflected, making the forecast results closer to the actual situation and improving the reliability and practicality of the forecast.

[0040] In another preferred embodiment of the present invention, the filtering process is:

[0041] Outliers are removed by quartile method, missing values ​​are supplemented by interpolation method, and data from different data sources are normalized. The formula is:

[0042] ;

[0043] Z represents any numerical data, Z0 represents the normalized value of the numerical data, and Z max and Z min Indicates the maximum and minimum values ​​of the same type of numerical data.

[0044] In another preferred embodiment of the present invention, the process of outputting power text features through the text extraction model is:

[0045] The text data is segmented by a standard dictionary, and the text data is split into several words. The text extraction model includes a BERT layer, a BiLSTM layer and a CRF layer. The words are input into BERT and mapped into word vectors. The word vectors are input into the BiLSTM layer, and the context feature information of the word vectors is deeply learned. The probability of each text information corresponding to a label is output, and an entity sequence is generated. The entity sequence is input into the CRF layer for sorting to obtain an entity sequence that optimizes the objective function; and the optimized entity sequence is screened based on a preset power field dictionary to screen out power text features.

[0046] By adopting a text extraction model that combines BERT, BiLSTM and CRF, the accuracy and efficiency of text data processing in power demand forecasting are effectively improved. The model first uses a standard dictionary to segment text data, splits the text into several words, and then maps these words into word vectors through the BERT layer. Next, these word vectors are input into the BiLSTM layer to capture contextual feature information in a deep learning manner, and output the probability of each text information corresponding to the label to generate an entity sequence. Finally, these entity sequences are input into the CRF layer for sorting to obtain the entity sequence that optimizes the objective function. Based on the preset power field dictionary, these optimized entity sequences are screened to accurately extract text features related to power demand. This process not only improves the efficiency of text feature extraction, but also ensures the accuracy and relevance of the extracted features, providing a more reliable data foundation for subsequent power demand forecasting.

[0047] In another preferred embodiment of the present invention, the power demand data is sorted according to the time sequence to generate the power demand data time series, the matrix formed by the time series is regarded as an image, and the image is convolved through the CNN network and then sequentially input into the BERT network of the large language power model, the BERT network includes several Transformer layers, each Transformer layer includes several filters, and whenever the matrix image passes through a Transformer layer, the filter slides and scans the matrix and generates a multidimensional array, reducing the size of the matrix image and increasing the dimension of the matrix image, the increased dimension being the same as the number of filters in the Transformer layer, each Transformer layer sequentially reduces the image size and increases the dimension of the matrix image, the multidimensional array finally generated in BERT is flattened and converted into a vector, and the vector is used as the input of LSTM.

[0048] By treating the time series of electricity demand data as an image and using the CNN network for convolution processing, the spatial features of the electricity demand data are effectively extracted. This method can not only capture the local patterns and trends of electricity demand data in the time dimension, but also gradually reduce the image size and increase the dimension through sliding scanning of the filter, thereby enriching the representation of the data and providing more refined feature input for subsequent deep learning models. Subsequently, these convolution-processed features are flattened and converted into vectors and used as inputs to LSTM, further utilizing the advantages of LSTM in time series analysis to model long-term dependencies on electricity demand. This method of combining CNN and LSTM not only retains the ability of CNN in spatial feature extraction, but also gives full play to the advantages of LSTM in time series analysis, achieving a comprehensive and in-depth analysis of electricity demand data, and significantly improving the accuracy and efficiency of electricity demand forecasting.

[0049] In a preferred case of this embodiment, the formula for extracting power demand features by the CNN network and the BERT network is:

[0050] ;

[0051] Among them, K is the convolution kernel, B is the bias term, σ is the activation function ReLU, which performs nonlinear mapping on the convolution result, and P ij is the matrix image obtained after convolution, i and j represent the position index in the matrix image, where i represents the row index, j represents the column index, X represents the input data, and m and n represent the elements in the input data that are offset by m rows and n columns relative to the current position (i, j).

[0052] The present invention extracts power demand features by combining CNN networks and BERT networks, thereby realizing efficient and accurate processing of power demand data. Specifically, the CNN network slides the convolution kernel K on the input data through a convolution operation, and calculates in combination with the bias term B, and then applies the activation function ReLU for nonlinear mapping to obtain a feature map after convolution. This process can not only capture local patterns and features in the input data, but also enhance the expressiveness and generalization capabilities of the model through nonlinear mapping. Subsequently, these feature maps are used as inputs to the BERT network, and the powerful capabilities of BERT in natural language processing are used to further extract and understand the deep semantic information in the power demand data. This method of combining CNN and BERT not only retains the advantages of CNN in spatial feature extraction, but also gives play to the capabilities of BERT in text understanding and semantic analysis, providing a richer and more sophisticated feature representation for power demand forecasting, significantly improving the accuracy and reliability of the forecast.

[0053] It is worth noting that the dimension of the matrix image output by BERT is kept unchanged, the matrix image is flattened in the time dimension and feature vector, and the final multidimensional arrays of all Transformer layers are used to generate a new matrix. The new matrix vector is used as the input of LSTM, and the flattened vector is used as the time dimension of LSTM. The new matrix is ​​trained by the time dimension and feature vector, and the correspondence between the power demand trend and time or season is output.

[0054] By keeping the dimension of the matrix image output by BERT unchanged and flattening it in the time dimension and feature vector, a new matrix is ​​generated as the input of LSTM. This method not only retains the feature information extracted by BERT, but also models the long-term dependencies of power demand data through the advantages of LSTM in time series analysis. Specifically, the flat vector generated by flattening is used as the time dimension of LSTM, and the new matrix is ​​trained in combination with the feature vector, so that the law of power demand changes over time or seasons can be accurately captured, and the corresponding corresponding relationship can be output. This method of combining BERT and LSTM not only retains the ability of BERT in feature extraction, but also gives play to the advantages of LSTM in time series analysis, realizes a comprehensive and in-depth analysis of power demand data, and significantly improves the accuracy and efficiency of power demand forecasting.

[0055] In another preferred embodiment of the present invention, the time scale of each type of electricity demand data is the same, and the weight is set for the number of samples in each quarter or year according to the relationship with the current time from near to far. When the collected data is in the past 4 years, the data volume ratio of samples in different years is 15%, 20%, 30% and 35% respectively.

[0056] In another preferred embodiment of the present invention, the integrated power demand data includes a month-on-month time series and a year-on-year time series, and the trends of the power demand data of the two series are trained respectively.

[0057] The integrated electricity demand data can more comprehensively reflect the dynamic changes in electricity demand by considering both the month-on-month time series and the year-on-year time series. The month-on-month time series focuses on the changes in electricity demand between adjacent time periods (such as months or quarters), while the year-on-year time series compares the electricity demand in the same time period (such as the same month or quarter of each year). Training the trends of these two series can capture the characteristics of electricity demand in short-term fluctuations and long-term trends, thereby more accurately predicting future electricity demand. This approach helps to improve the model's sensitivity to the impact of seasonality, cyclicality and emergencies, thereby improving the accuracy and reliability of electricity demand forecasts.

[0058] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A method for predicting power demand based on a large language model, characterized in that: The following steps are involved: Collecting power demand data, the power demand data including numerical power data, meteorological data, socio-economic data and text data, and filtering the power demand data; The numerical power data includes power consumption records for the past several years, arranged by hour, day, week and month; the meteorological data includes temperature, humidity, wind speed and precipitation in the selected area; the socio-economic data includes population data, industrial data, holidays, and major events in the selected area; the text data includes social media comments, news reports and user feedback; Mapping the text data into text vectors, encoding and sorting the text vectors through position embedding, and outputting power text features through a text extraction model; dividing the power demand data processed above into a training set, a validation set, and a test set; Set seasonal parameters and integrate electricity demand data according to time sequence through ARIMA model; A large language power model is jointly constructed based on CNN, BERT and LSTM, model parameters are set, the data of the training set is input into the large language power model for training, and numerical power data is used as labels. The root mean square error between the power demand trend data output by the model and the label is calculated as the loss function, the gradient of the loss function relative to the model parameters is calculated, and the parameters are updated. The above process is repeated for multiple cycles until convergence; Use the validation set to verify the performance of the model until the model performance stops improving, and use an independent test set to evaluate the final performance of the model and calculate the model error; The process of jointly building a large language power model based on CNN, BERT and LSTM is as follows: The power demand data is sorted according to the time sequence to generate the power demand data time series, the matrix formed by the time series is regarded as an image, and the image is convolved through the CNN network and then sequentially input into the BERT network of the large language power model, wherein the BERT network includes a plurality of Transformer layers, each Transformer layer includes a plurality of filters, and whenever the matrix image passes through a Transformer layer, the filter slides and scans the matrix and generates a multidimensional array, reduces the size of the matrix image, and increases the dimension of the matrix image, and the increased dimension is the same as the number of filters in the Transformer layer, and each Transformer layer sequentially reduces the image size and increases the dimension size of the matrix image, and flattens the multidimensional array finally generated in BERT into a vector, and uses the vector as the input of the LSTM; The formula for extracting power demand features by CNN network and BERT network is: ； Among them, K is the convolution kernel, B is the bias term, σ is the activation function ReLU, which performs nonlinear mapping on the convolution result, and P ij is the matrix image obtained after convolution, i and j represent the position index in the matrix image, where i represents the row index, j represents the column index, X represents the input data, and m and n represent the elements in the input data that are offset by m rows and n columns relative to the current position (i, j); The dimension of the matrix image output by BERT remains unchanged, and the matrix image is flattened in the time dimension and feature vector. The final multidimensional arrays of all Transformer layers are used to generate a new matrix. The new matrix vector is used as the input of LSTM, and the flattened vector generated by flattening is used as the time dimension of LSTM. The new matrix is ​​trained by the time dimension and feature vector, and the correspondence between the power demand trend and time or season is output.

2. The method for predicting power demand based on a large language model according to claim 1, characterized in that: The filtering process is: Outliers are removed by quartile method, missing values ​​are supplemented by interpolation method, and data from different data sources are normalized. The formula is: ； Z represents any numerical data, Z0 represents the normalized value of the numerical data, and Z max and Z min Indicates the maximum and minimum values ​​of the same type of numerical data.

3. The method for predicting power demand based on a large language model according to claim 1, characterized in that: The process of outputting power text features through the text extraction model is as follows: The text data is segmented by a standard dictionary, and the text data is split into several words. The text extraction model includes a BERT layer, a BiLSTM layer and a CRF layer. The words are input into BERT and mapped into word vectors. The word vectors are input into the BiLSTM layer, and the context feature information of the word vectors is deeply learned. The probability of each text information corresponding to a label is output, and an entity sequence is generated. The entity sequence is input into the CRF layer for sorting to obtain an entity sequence that optimizes the objective function; and the optimized entity sequence is screened based on a preset power field dictionary to screen out power text features.

4. The method for predicting power demand based on a large language model according to claim 1, characterized in that: The time scale of each type of electricity demand data is the same, and weights are set for the number of samples in each quarter or year based on their relationship with the current time from near to far, and the weights are distributed in order from near to far.

5. The method for predicting power demand based on a large language model according to claim 1, characterized in that: The integrated electricity demand data includes month-on-month time series and year-on-year time series, and the trends of the electricity demand data of the two series are trained respectively.

Images