Power load forecasting method based on large language model and socio-demographic information

Abstract

The present invention discloses a method for power load forecasting based on a large language model and social demographic information, relates to the technical field of power load forecasting, and solves the technical problem that the power load forecasting method is difficult to combine social demographic information, difficult to process unstructured text information, and affects the accuracy and generalization ability of power load forecasting. The method includes obtaining historical power load information based on a time series conversion module; obtaining structured demographic information based on a social adapter module; fusing historical power load information and structured demographic information as input data of a power load forecasting model; and training by minimizing the prediction error of the power load forecasting model to obtain a power load forecasting model that is ultimately used for forecasting. The present invention combines a large language model with social demographic information, breaks through the limitations of traditional load forecasting models in data utilization, can process unstructured text information, and improves the accuracy of predictions.

Description

Technical Field

[0001] The present invention relates to the technical field of power load forecasting, and in particular to a power load forecasting method based on a large language model and social demographic information. Background Art

[0002] With the continuous increase in global energy demand and the improvement of power system reliability requirements, power load forecasting has become an important part of power system planning and operation. Traditional load forecasting methods mainly rely on time series data and historical load data. Typical forecasting models include regression, support vector machine, long short-term memory network (LSTM), etc. These methods can capture the temporal pattern of power load changes to a certain extent.

[0003] Since socio-demographic factors, such as household income, household size, and employment status, have a significant impact on residents' electricity consumption behavior, this information is often crucial to understanding the characteristics of power loads. Traditional machine learning and deep learning models have great difficulties in processing such unstructured text information, resulting in insufficient accuracy in load forecasting. In addition, with the diversification and complexity of energy usage patterns, forecasting methods that rely solely on time series data are difficult to adapt to changes in modern electricity demand. Therefore, existing technologies have also attempted to incorporate non-time series information into load forecasting models, such as using graph neural networks (GNNs) to model the relationship between different electricity users, or using multimodal fusion models to integrate information from multiple data sources. However, these methods have difficulty combining socio-demographic information with power load data, and have difficulty processing high-dimensional unstructured text information.

[0004] Large Language Model (LLM) performs well in processing natural language and text information, and can effectively understand and reason about complex text information. Therefore, it is urgent to introduce LLM into power load forecasting and improve the accuracy and generalization ability of power load forecasting by integrating social demographic information.

[0005] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0006] Power load forecasting methods are difficult to incorporate socio-demographic information and to process high-dimensional unstructured text information, which affects the accuracy and generalization ability of power load forecasting. Summary of the invention

[0007] The purpose of the present invention is to provide a method for power load forecasting based on a large language model and social demographic information, so as to solve the technical problem that the power load forecasting method in the prior art is difficult to combine social demographic information, difficult to process high-dimensional unstructured text information, and affects the accuracy and generalization ability of power load forecasting. The preferred technical solution among the many technical solutions provided by the present invention can produce many technical effects as described below.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] The present invention provides a method for power load forecasting based on a large language model and social demographic information, comprising the following steps: S100: based on a time series conversion module, standardizing and instance normalizing the time series data of the historical power load to obtain the historical power load information; S200: based on a social adapter module, converting the social demographic information into an embedding representation suitable for processing by a large language model to obtain structured demographic information; S300: based on a social perception reprogramming layer, fusing the historical power load information and the structured demographic information as input data for an initial power load forecasting model; S400: training by minimizing the prediction error of the initial power load forecasting model to obtain the power load forecasting model ultimately used for forecasting.

[0010] Preferably, in the step S100, the time series conversion module performs standardization processing on the input time series data by reversible instance normalization, and the standardization processing is performed by the following formula:

[0011]

[0012] in, Represents the total number of time series data blocks, Indicates the horizontal sliding stride, Indicates the time step length of the historical load data power consumption. The time series data representing overlapping or non-overlapping data blocks are lengthened, and number 2 handles the boundary problem by adding extra blocks at the starting point and the end point, so that the time series data of the power historical load is completely covered.

[0013] Preferably, in step S200, the social adapter module converts the socio-demographic information into a text format understandable to the large language model through a pre-built prompt template, and generates structured demographic information using pre-trained knowledge of the large language model.

[0014] Preferably, the workflow of the social adapter module is expressed by the formula Conduct, among which, Represents structured demographic information, represents a large language model, Represents a pre-built prompt template.

[0015] Preferably, in step S300, the social perception reprogramming layer adopts a multi-head cross-attention mechanism to enable interaction between historical power load information and structured demographic information.

[0016] Preferably, the cross attention mechanism is expressed by the following formula:

[0017]

[0018]

[0019]

[0020] in, represents the query matrix projected from the historical power load information, denote the key matrix and value matrix projected from the structured demographic information, respectively. The function represents the linear embedding layer of the embedding representation, Indicates that the historical power load information is input as a query item. Represents a word embedding matrix that is smaller than the pre-trained one A smaller set of prototype embeddings, Represents the feature matrix.

[0021] Preferably, the large language model uses low-rank adaptation LoRA to adjust some layers of the model to adapt to the specific needs of power load forecasting.

[0022] Preferably, the low-rank adaptation LoRA is expressed by the following formula:

[0023]

[0024]

[0025] in and Respectively represent the last Layer Transformer Query Sum Projection applies low-rank adaptation LoRA update, Representation layer Low-rank adaptive LoRA update of the query and value matrices in , is the learned low-rank matrix.

[0026] Preferably, the Prompt-as-Prefix strategy is introduced to further enhance the reasoning ability of the large language model, and the historical power load information is input into the large language model as a prefix.

[0027] Preferably, training is performed by minimizing the mean square error between the predicted load value and the actual load value, and the mean square error loss function is The expression is:

[0028]

[0029] in, Indicates the historical power load information, indicating the backtracking time is The power consumption in a time step is Indicates Socio-demographic information about the household, represents the parameters of the power load forecasting model, Indicates the predicted power load value.

[0030] Implementing one of the above technical solutions of the present invention has the following advantages or beneficial effects:

[0031] The present invention outperforms the prediction accuracy of traditional methods on multiple real load data sets, especially in the load forecasting scenario during peak electricity consumption. The model's responsiveness and prediction accuracy are significantly improved by integrating social demographic information. In addition, due to the introduction of the social perception reprogramming layer, the present invention can more flexibly adapt to the electricity consumption behavior of different user groups, providing a more effective decision support tool for the power sector. By combining a large language model with social demographic information, the limitations of traditional load forecasting models in data utilization have been broken through. It can not only process unstructured text information and improve the accuracy of predictions, but also excel in data privacy protection, providing strong support for future energy management and power system planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. It is obvious that the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0033] Figure 1 is a flow chart of a method for power load forecasting based on a large language model and social demographic information according to an embodiment of the present invention;

[0034] Figure 2It is an overall framework diagram of the power load forecasting method based on a large language model and social demographic information according to an embodiment of the present invention;

[0035] Figure 3 is a load forecast comparison diagram of LLMSA and different socio-demographic data according to an embodiment of the present invention;

[0036] Figure 4 Schematic diagram of attention score of a sample in LLMSA according to an embodiment of the present invention;

[0037] Figure 5 Schematic diagram of evaluating the number of training layers U of LLMSA in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present invention clearer, the various exemplary embodiments to be described below will refer to the corresponding drawings, which constitute a part of the exemplary embodiments, wherein various exemplary embodiments that may be used to implement the present invention are described. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present disclosure. It should be understood that they are only examples of processes, methods, devices, etc. that are consistent with some aspects of the present disclosure as detailed in the attached claims, and other embodiments may also be used, or the embodiments listed herein may be modified in structure and function without departing from the scope and essence of the present invention.

[0039] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", etc. indicate the orientation or positional relationship based on the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the elements referred to must have a specific orientation, be constructed and operated in a specific orientation. The terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. The term "multiple" means two or more. The terms "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, an integral connection, a mechanical connection, an electrical connection, a communication connection, a direct connection, an indirect connection through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0040] In order to illustrate the technical solution of the present invention, a specific embodiment is used below for description, and only the parts related to the embodiment of the present invention are shown.

[0041] Embodiment 1:

[0042] like Figure 1 , Figure 2 As shown, the present invention provides a method for predicting power load based on a large language model and social demographic information, characterized in that it includes the following steps: S100: Based on the time series conversion module, the time series data of the power historical load is standardized and instance normalized to eliminate the distribution drift problem in the time series data, enhance the stability of the model on data in different time periods, and capture the time series characteristics of the power historical load changes, thereby obtaining the power historical load information. S200: Based on the social adapter module, the social demographic information is converted into an embedded representation suitable for large language model processing, which can be effectively combined with the historical load data to obtain structured demographic information; the social demographic information includes family income, population size, work status, etc., which are of great significance in predicting load. S300: Based on the social perception reprogramming layer, the power historical load information and the structured demographic information are fused as input data of the initial power load prediction model, and the initial power load prediction model is a large language model, thereby capturing complex social and time dependencies. This process can effectively enhance the accuracy of load prediction, especially in scenarios with strong social background influence. S400: The prediction error of the initial power load forecasting model is minimized by optimizing the function for training. The final power load forecasting model used for prediction can be obtained by training the initial power load prediction, thereby realizing the prediction of the power load. The present invention is superior to the prediction accuracy of traditional methods on multiple real load data sets, especially in the load forecasting scenario during peak power consumption. The responsiveness and prediction accuracy of the model are significantly improved by integrating social demographic information. In addition, due to the introduction of the social perception reprogramming layer, the present invention can more flexibly adapt to the power consumption behavior of different user groups, providing a more effective decision support tool for the power sector. By combining the large language model with social demographic information, the limitations of traditional load forecasting models in data utilization are broken through. Not only can unstructured text information be processed to improve the accuracy of prediction, but also data privacy protection is excellent, providing strong support for future energy management and power system planning.

[0043] As an optional implementation, in step S100, the time series conversion module normalizes the input time series data through reversible instance normalization (Reversible Instance Normalization, RevIN), and the normalization is performed by the following formula:

[0044]

[0045] in, Represents the total number of time series data blocks, Indicates the horizontal sliding stride, Indicates the time step length of the historical load data power consumption. The time series data representing overlapping or non-overlapping data blocks are length, and number 2 handles the boundary problem by adding additional blocks at the starting point and the end point, so that the time series data of the power historical load is completely covered. Thus, the power historical load data is processed into a standardized form with zero mean and unit variance, ensuring that the time series pattern of the data can be effectively captured.

[0046] As an optional implementation, in step S200, the social adapter module converts the social demographic information into a text format understandable by the large language model through a pre-built prompt template. Specifically, the social information is organized into text, and then the large language model is used to generate social information embeddings. These embeddings contain rich semantic information of social factors and can be used in subsequent reprogramming layers. After conversion, the pre-trained knowledge of the large language model is used to generate structured demographic information. The workflow of the social adapter module is shown in the formula Conduct, among which, Represents structured demographic information, represents a large language model, Represents a pre-built prompt template.

[0047] As an optional implementation, in step S300, the social perception reprogramming layer adopts a multi-head cross-attention mechanism to combine time series data with social information embedding, so that the historical power load information and structured demographic information interact with each other. Specifically, the historical power load information of the time series is first segmented into local patches, and each patch generates a feature embedding, which is input into the multi-head cross-attention module together with the structured demographic information embedding. This module can capture the complex dependencies between time series and demographic information, thereby providing more comprehensive contextual information for the large language model. The cross-attention mechanism is expressed by the following formula:

[0048]

[0049]

[0050]

[0051] in, represents the query matrix projected from the historical power load information, denote the key matrix and value matrix projected from the structured demographic information, respectively. The function represents the linear embedding layer of the embedding representation, Indicates that the historical power load information is input as a query item. Represents a word embedding matrix that is smaller than the pre-trained one A smaller set of prototype embeddings, Represents the feature matrix.

[0052] As an optional implementation, the large language model uses low-rank adaptation LoRA to adjust some layers of the model to adapt to the specific needs of power load forecasting. Low-rank adaptation LoRA parameterizes some layers of the large language model to reduce computational overhead and adapt to the specific needs of load forecasting tasks. Low-rank adaptation LoRA is expressed by the following formula:

[0053]

[0054]

[0055] in and Respectively represent the last Layer Transformer Query Sum Projection applies low-rank adaptation LoRA update, Representation layer Low-rank adaptive LoRA update of the query and value matrices in , is the learned low-rank matrix.

[0056] As an optional implementation, the Prompt-as-Prefix strategy is introduced to further enhance the reasoning ability of the large language model, and the historical power load information is input into the large language model as a prefix. In Patch-as-Prefix, some prior information of the time series data set is used as a prefix in natural language, and is concatenated with the aligned time series features and fed to the large language model to improve the prediction effect. The large language model is prompted to predict the subsequent values ​​in the time series in natural language, which improves the adaptability of the large prediction model to downstream tasks and supplements the reprogramming of the patch.

[0057] As an optional implementation, training is performed by minimizing the mean square error between the predicted load value and the actual load value. The calculation is relatively simple and reliable. This joint training is not only based on time series information, but also combines social demographic information, making the model more robust when dealing with the load characteristics of different social groups. Mean square error loss function The expression is:

[0058]

[0059] in, Indicates the historical power load information, indicating the backtracking time is The power consumption in a time step is Indicates Socio-demographic information about the household, represents the parameters of the power load forecasting model, Represents the predicted power load value. The joint training method based on historical load and social demographic information is adopted to improve the prediction ability of the power load forecasting model for different social groups.

[0060] In this embodiment, the power load forecasting method (LLMSA) proposed in this embodiment is verified by using the data set of the Irish Energy Regulatory Commission. The data set is derived from the smart metering power user behavior experiment from July 14, 2009 to December 31, 2010. The data set contains the electricity records every 30 minutes for 18 months, collected from 4232 Irish residential users. The information collected through the questionnaire survey reveals the energy consumption behavior and habits of Irish residential customers. The data were deeply analyzed to dig out the patterns and correlations between various factors and classify the customers. The key tags extracted include employment status, house ownership and number of appliances, aiming to summarize the user's electricity consumption characteristics. As shown in Table 1, these tags are further divided into two or three categories.

[0061] Table 1 Family characteristic labels

[0062] The experimental indicators mainly use mean square error (MSE), mean absolute error (MAE) and determination coefficient ( ) to measure the prediction accuracy of the model. At the same time, the performance of load forecasting during peak power consumption is analyzed to verify the reliability of the model under high demand conditions. The calculation formula is as follows:

[0063]

[0064]

[0065]

[0066] in, Respectively represent The actual and predicted values ​​for each instance.

[0067] To evaluate the performance, the dataset is divided into 80% training set and 20% test set in chronological order. The learning rate α is set to 0.0001, the lookback length L is set to 672, and the prediction step T is set to 336. GPT2 is used as the pre-trained large language model, which is chosen because of its light weight and good performance, and low-rank adaptation LoRA is applied to its last 12 layers because U=12. To verify the prediction accuracy of the method of this embodiment, the method of this embodiment is run and the results are compared with the following three different methods: PatchTST, a channel-independent patch time series Transformer based on time series segmentation; iTransformer, applying attention and feedforward networks on the inverted dimension; TimeLLM, a reprogramming framework is proposed to reuse the large language model for time series prediction, including load prediction.

[0068] Table 2 shows that the method of this embodiment outperforms other models in most cases across all sociodemographic factors, showing lower MSE and MAE values ​​and higher score. It is noteworthy that the method of this embodiment outperforms the baseline model by up to 4.96%. This improved performance is due to the unique ability of the method of this embodiment to integrate socio-demographic factors into the prediction process, thereby capturing the subtle relationship between the factors and energy consumption patterns. The connection between social information embedding and the social perception reprogramming layer enables the method of this embodiment to generate more intelligent and context-aware representations, ultimately leading to more accurate predictions. Compared with the method of this embodiment, PatchTST and iTransformer perform poorly. PatchTST focuses only on extracting temporal features while ignoring socio-demographic factors, which limits its ability to capture user-specific consumption behaviors. Similarly, while iTransformer employs advanced temporal feature extraction, it also lacks socio-demographic integration, which weakens its adaptability to different socio-demographic factors. Therefore, the performance of these two models is consistently poor compared to the method of this embodiment. TimeLLM shows better performance than PatchTST and iTransformer due to the inherent generalization ability of the pre-trained large language model. Although TimeLLM shows a slightly better MAE in some cases, this metric balances the average error without severely penalizing outliers and does not perform significantly better than the method of this embodiment. Furthermore, TimeLLM lacks the socially relevant adapters necessary to effectively integrate socio-demographic factors, which limits its ability to fully capture the relationship between user characteristics and energy consumption, resulting in lower accuracy compared to the method of this embodiment.

[0069] Table 2 Comparison of different methods

[0070]

[0071] Figure 3 The load forecasting performance of different models under socio-demographic factors is shown. All models are trained to predict 336 data points in the future, but only the last 96 forecast points are shown in the figure, focusing on the differences in model performance as the forecast range increases, resulting in an increase in error. The comparison results clearly show that the method of this embodiment closely follows the real data in all socio-demographic groups, showing its superior performance in predicting power load consumption. The method of this embodiment always has more accurate predictions at the peak of the real data, indicating that socio-demographic factors have a significant impact on the peak behavior of energy consumption. The components related to social factors in the method of this embodiment are specifically designed to capture these effects, so that accurate adjustments can be made in the forecast to better reflect the real consumption peak. PatchTST and iTransformer show larger gaps in the real data, especially during periods of significant fluctuations in energy consumption. The relatively simple model structures of PatchTST and iTransformer limit their ability to capture complex relationships, resulting in delayed responses to peak demand and failure to fully capture the amplitude of consumption peaks and valleys. Although TimeLLM performs better than PatchTST and iTransformer, it is not as good as the method of this embodiment. Due to the lack of social-related adapters and social-aware reprogramming layers, TimeLLM cannot effectively integrate sociodemographic factors, resulting in reduced adaptability to changes in different factors. Overall, the method of this embodiment showed consistent and more accurate prediction capabilities under all sociodemographic factors. This shows that the method of this embodiment successfully combines time characteristics and sociodemographic factors through its unique components, thereby providing more reliable and robust energy consumption predictions.

[0072] In order to evaluate whether the method in this embodiment truly understands and effectively utilizes the sociodemographic factors in text form, a comparative analysis was conducted using "age" as one of the sociodemographic factors. Specifically, the method in this embodiment was compared with three methods that changed the sociodemographic context: NoFactors, which excludes the input of any sociodemographic factors with the purpose of analyzing the benefits brought by sociodemographic factors; Random Factors, in which the sociodemographic information is randomly shuffled, which actually destroys the connection between each household load data and its sociodemographic characteristics. This test evaluates whether it can still gain advantages even when the sociodemographic data of the model does not match; Factors ID, which replaces the sociodemographic description in text form and uses ID embedding. In this case, the model cannot use prior knowledge about sociodemographic text and must learn from scratch, limiting the advantages of using pre-trained knowledge of large language models.

[0073] The results are summarized in Table 3. Across all age groups, the method of this embodiment consistently outperforms the other methods, demonstrating its effectiveness in accurately capturing consumption patterns influenced by sociodemographic factors. In contrast, the “no factor” and “random factor” settings perform worse than the method of this embodiment, indicating that a lack of sociodemographic information or information mismatch can deteriorate model performance. In the “random factor” setting, random sociodemographic inputs lead to inconsistencies between household load data and sociodemographic information, introducing noise into model training. The models are unable to learn the true relationship between user attributes and energy use, but are forced to learn from contradictory or erroneous relationships, which severely affects their ability to make accurate predictions. However, historical load data still enables these models to make relatively effective predictions, emphasizing the importance of time series inputs. The “factor ID” method also consistently performs worse than the method of this embodiment, but is similar to the performance of the “no factor” setting. This similarity suggests that using identifier embeddings without actual textual context does not allow the model to effectively exploit the rich prior knowledge embedded in the large language model. In fact, “Factor ID” does not provide any explicit semantic information that the model can leverage, so the model can only learn correlations based on raw data, but cannot gain advantages by understanding user-specific sociodemographic factors. The use of identifiers fails to leverage the general understanding capabilities of large language models, which could better account for sociodemographic details. Due to the lack of detailed sociodemographic context, the “Factor ID” approach only has a small improvement over “No Factor”, which shows that providing rich text descriptions is crucial for further effective predictions.

[0074] Table 3 Comparison of different methods for the sociodemographic factor “age”

[0075] In order to understand in detail how the method framework of this embodiment understands sociodemographic factors in text form, we focus on analyzing the attention mechanism of the last layer of the transformer of the large language model. Figure 4 The attention score of a certain sample is shown. In this heat map, the y-axis represents the tag tokens reprogrammed by the user's historical load data, and the x-axis corresponds to the tag tokens from the sociodemographic factors in text form. It is worth noting that the highest attention weight is assigned to specific tag tokens that play a key role in defining the sociodemographic context. Specifically, the model significantly focuses on the initial and end tokens that are "social" tag tokens, indicating the boundaries of the sociodemographic input, and the type of sociodemographic information, such as "age". In addition, significant attention allocation is also observed for the tag token corresponding to "age is 46-55 years old", indicating that the model understands the importance of this specific attribute value. The attention score shows that the method model of this embodiment can effectively focus on the relevant parts of the sociodemographic input in text form, which include both the category of information (such as "age") and the specific details in the category (such as "46-55 years old"). This shows that the model does not regard the sociodemographic information as general text, but actively utilizes the semantic meaning in the text, enabling it to make more informative predictions about energy consumption. Overall, the results consistently show that the method of the present embodiment, which incorporates and understands sociodemographic factors in text form, outperforms other methods in all age groups, suggesting that text-based sociodemographic information effectively leverages the capabilities of pre-trained large language models to encode richer insights, thereby achieving better load prediction performance.

[0076] Evaluated a key hyperparameter , i.e. the number of transformer layers in the large language model updated using low-rank adaptive LoRA. It aims to understand the extent to which the adaptability of the large language model is affected by its ability to effectively learn from socio-demographic factors in text form and temporal load data. While increasing the number of trainable transformer layers for low-rank adaptive LoRA applications can improve the adaptability of the model to the load prediction task, this also increases the risk of overfitting, potentially leading to the loss of generalizable prior knowledge encoded in the large language model. It is therefore crucial to find an optimal balance to fully exploit the capabilities of the large language model while ensuring effective adaptability for task-specific learning. Therefore, the goal is to determine an optimal , to improve adaptability without compromising the generalization ability of the pre-trained large language model. Performance between 4 and 32 layers. Figure 5 Shows the difference The evaluation results under the value include MSE, MAE and R². As can be seen from the figure, initially Increasing from 4 to 12 improves the performance of the model, indicating an improvement in the model’s ability to adapt to domain-specific needs and learn from sociodemographic information. This trend is due to the increased flexibility of the last few layers of the large language model, which makes fine-tuning load prediction more effective. However, when When it continues to increase to more than 20, the MSE value begins to fluctuate slightly and eventually begins to rise, indicating that the benefits of updating more transformer layers are gradually diminishing, suggesting that the model begins to lose the generalization knowledge captured during pre-training and begins to fit the training data too closely. Therefore, increasing the number of training layers for low-rank adaptation LoRA can improve the accuracy of load prediction by enhancing the adaptability of the model, but excessive fine-tuning may lead to the loss of generalization characteristics, thereby reducing performance.

[0077] The embodiment is only a specific example and does not represent only one implementation mode of the present invention.

[0078] The above description is only the preferred embodiment of the present invention. It is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the protection scope of the present invention.

Claims

1. A method for power load forecasting based on a large language model and social demographic information, characterized in that: The following steps are involved: S100: Based on the time series conversion module, the time series data of the power historical load is standardized and instance normalized to obtain the power historical load information; S200: Based on the social adapter module, the social demographic information is converted into an embedding representation suitable for processing by a large language model to obtain structured demographic information; S300: Based on the social perception reprogramming layer, the power historical load information and the structured demographic information are integrated as input data of an initial power load forecasting model; S400: training by minimizing the prediction error of the initial power load prediction model to obtain a final power load prediction model for prediction; In the step S300, the social perception reprogramming layer adopts a multi-head cross attention mechanism to enable interaction between the power historical load information and the structured demographic information; The cross-attention mechanism is expressed by the following formula: in, represents the query matrix projected from the historical power load information, denote the key matrix and value matrix projected from the structured demographic information, respectively. The function represents the linear embedding layer of the embedding representation, Indicates that the historical power load information is input as a query item. Represents a word embedding matrix that is smaller than the pre-trained one A smaller set of prototype embeddings, represents the feature matrix; The large language model uses low-rank adaptation LoRA to adjust some layers of the model to meet the specific needs of power load forecasting; The low-rank adaptation LoRA is expressed by the following formula: in and Respectively represent the last Layer Transformer Query Sum Projection applies low-rank adaptation LoRA update, Representation layer Low-rank adaptive LoRA update of the query and value matrices in , is the learned low-rank matrix.

2. The method for power load forecasting based on large language model and social demographic information according to claim 1, characterized in that: In the step S100, the time series conversion module performs standardization processing on the input time series data by reversible instance normalization, and the standardization processing is performed by the following formula: in, Represents the total number of time series data blocks, Indicates the horizontal sliding stride, Indicates the time step length of the historical load data power consumption. The time series data representing overlapping or non-overlapping data blocks are lengthened, and number 2 handles the boundary problem by adding extra blocks at the starting point and the end point, so that the time series data of the power historical load is completely covered.

3. The power load forecasting method based on large language model and social demographic information according to claim 1 is characterized in that: In the step S200, the social adapter module converts the social demographic information into a text format understandable by the large language model through a pre-built prompt template, and generates structured demographic information using the pre-trained knowledge of the large language model.

4. The method for power load forecasting based on large language model and social demographic information according to claim 3 is characterized in that: The workflow of the social adapter module is expressed as Conduct, among which, Represents structured demographic information, represents a large language model, Represents a pre-built prompt template.

5. The method for power load forecasting based on large language model and social demographic information according to claim 1, characterized in that: By introducing the Prompt-as-Prefix strategy, the reasoning ability of the large language model is further enhanced, and the historical power load information is input into the large language model as a prefix.

6. The method for power load forecasting based on large language model and social demographic information according to claim 1, characterized in that: The training is performed by minimizing the mean square error between the predicted load value and the actual load value. The mean square error loss function The expression is: in, Indicates the historical power load information, indicating the backtracking time is The power consumption in a time step is Expressed as Socio-demographic information about the household, represents the parameters of the power load forecasting model, Indicates the predicted power load value.

Images