Electricity load prediction method enhanced by large language model

Abstract

The application discloses a power load prediction method enhanced by a large language model, which comprises the following steps: dividing a time sequence according to a certain step length, extracting multi-scale feature information through convolution kernels of different sizes, processing the time sequence after convolution through a channel attention mechanism, enabling the channel attention mechanism to complete feature learning in the channel aspect, strengthening the extraction of feature information, adding weights in the channel aspect to the time sequence, concentrating attention on the key part, and suppressing attention on unimportant features, and converting the time sequence information into text information that can be understood by the large language model through a cross-attention mechanism. The application strengthens feature extraction by introducing the channel attention mechanism, and improves the prediction accuracy with little increase in calculation amount.

Description

Technical Field

[0001] This invention relates to the fields of data processing and power system automation technology, specifically to a power load forecasting method enhanced by a large language model. Background Technology

[0002] Power system load forecasting predicts changes in power system load over a future period, taking into full account historical load data, economic conditions, meteorological conditions, and social events. Load forecasting is crucial for power system planning, dispatching, and management, helping power companies allocate resources rationally and ensuring the stability and reliability of power supply. Recent research and application development in power system load forecasting has yielded significant results, proposing numerous load forecasting methods.

[0003] Currently, a convenient approach in practical applications is to utilize large language models similar to Time-LLM for power load forecasting. Time-LLM is a reprogrammed large language model framework that leverages large language models to perform time-series forecasting while maintaining the integrity of the backbone model. Specifically, the raw time-series data collected by transformer field sensors (current, voltage, oil temperature, ambient temperature, etc.) is first divided into several patches with a fixed step size; then, a single-scale 1D convolution is performed on each patch to extract local features; simultaneously, the large language model compresses its vocabulary into a subset relevant to the time-series information, and uses a cross-attention mechanism to map the convolutional features into text embeddings; finally, guided by cue prefixes, the frozen large language model outputs future load values, which are then converted back to a time series through a projection layer. However, in practice, the prediction accuracy still falls short of requirements. The main reason for this is that using large language models for power load forecasting presents the following problems:

[0004] (1) Fixed step size segmentation cannot distinguish between abrupt change periods and stable periods in the load curve, resulting in insufficient feature extraction of key periods or wasted computational resources in redundant periods.

[0005] (2) Single-scale convolution cannot simultaneously capture short-term shocks (such as current spikes caused by heavy load switching) and long-term trends (such as the overall shape of the daily load curve), resulting in a large prediction error.

[0006] (3) Each sensor channel is assigned the same weight during feature extraction, which cannot be dynamically adjusted according to real-time operating conditions, resulting in redundant information entering the model and reducing accuracy. Summary of the Invention

[0007] The present invention aims to at least partially solve one of the technical problems existing in the related art.

[0008] The purpose of this invention is to provide a power load forecasting method enhanced by a large language model, which adaptively weights and fuses local shocks and long-term trends during abrupt changes in time series data, thereby significantly improving forecasting accuracy without retraining the large language model and with almost no increase in computing power.

[0009] To achieve the above objectives, the present invention provides a power load forecasting method enhanced by a large language model, comprising the following steps:

[0010] Acquire time-series data from the power transformer monitoring system and perform preprocessing;

[0011] The time series data is divided into several time segments of fixed length according to a set length, and each time segment is passed through a convolutional layer to obtain the initial convolutional features.

[0012] The obtained initial convolution features are then subjected to secondary convolution using small and large convolution kernels respectively, resulting in two types of convolution features after secondary convolution.

[0013] The two convolutional features after the second convolution are subjected to global average pooling and global max pooling respectively, and then passed through a fully connected layer to obtain two sets of channel weights. The two sets of channel weights are then weighted and summed to generate a comprehensive channel weight. Based on this comprehensive channel weight, the two convolutional features after the second convolution are weighted and fused to obtain a fused feature.

[0014] The fused features are input into a frozen large language model, mapped to text embeddings via a cross-attention mechanism, and output load prediction results for future periods under the guidance of cue prefixes.

[0015] A further preferred embodiment of the present invention is that the acquisition of time-series data from the power transformer monitoring system, followed by preprocessing, specifically involves:

[0016] Time-series data from various monitoring systems of power transformers, including oil temperature, load, current, and voltage data, are collected at a sampling rate of once per hour or every 15 minutes. The acquired time-series data are then processed by imputing missing values, removing outliers, normalizing, and standardizing.

[0017] As a preferred option, when passing each time segment through a convolutional layer to obtain the initial convolutional features, the time segment is input into a 1*1 convolutional layer to upgrade the input to a high-dimensional feature map.

[0018] Then, the feature map is convolved twice using both large and small convolution kernels. The small convolution kernel is used to capture local feature changes, while the large convolution kernel is used to capture the global shape of the time sequence.

[0019] Preferably, the two convolutional features after the second convolution are subjected to global average pooling and global max pooling, respectively; specifically:

[0020] The feature map after the second convolution is subjected to global average pooling. For each channel, the spatial dimension of that channel, i.e., the time length, is compressed into a single value. The feature map of each channel is compressed into a global descriptor, which represents the average response of that channel in the entire space.

[0021] At the same time, the feature map is subjected to global max pooling, which also generates a global descriptor representing the maximum response of the channel in the entire space;

[0022] The input feature map is T*C*N, where T is the time series length, C is the number of channels, and N is the batch size. The output is a 1*C*N vector.

[0023] Preferably, after global average pooling and global max pooling, the feature map is input into a shared fully connected layer.

[0024] The shared connection layer consists of two fully connected layers. The first fully connected layer compresses the number of channels C of the feature map to the C / r dimension through a compression operation, where r is a hyperparameter representing a scaling factor. The second fully connected layer restores the number of channels C of the feature map to its original value. A ReLU activation function is added between the two fully connected layers to learn the non-linear relationship of the features. Finally, a Sigmoid activation function is added to the two fully connected layers to adjust the weight of each channel to a value between 0 and 1, representing the saliency of that channel.

[0025] Preferably, after obtaining the weights of each channel in the feature map, the two sets of channel weights are summed to generate a comprehensive channel weight. Based on this comprehensive channel weight, the two convolutional features after the second convolution are then weighted and fused to obtain the fused features; specifically:

[0026] After processing the feature maps using both large and small convolutional kernels, they pass through a channel attention module to obtain the channel weights of the two sets of feature maps. Then, a weighted sum is performed to obtain a channel importance weight vector with a comprehensive effect. This weight vector is then multiplied channel by channel with the original input feature map, and finally, a fused feature with channel-dimensional weights is output.

[0027] Preferably, the fused features are input into a frozen large language model, mapped to text embeddings via a cross-attention mechanism, and the load prediction results for future time periods are output under the guidance of cue prefixes; specifically:

[0028] The fused features with channel-dimensional weights are combined with time-related vocabulary groups compressed from a large language model. A cross-attention approach is used to convert time series data into vocabulary. The segmented time series blocks serve as input information Q, while the compressed vocabulary groups serve as K and V. The final output V represents the text information. The conversion formula is as follows:

[0029] ;

[0030] The converted text information and the prompt prefix are input into the large language model. After the large language model makes inference and prediction, the prediction result is obtained. The prompt part is removed, and the predicted information is projected into a time series as the load prediction result for the future period.

[0031] Preferably, the prompt prefix includes statistical information on the maximum, minimum, and average values ​​of the task description and time segment, which is used to guide the large language model to complete the load prediction for the next time period.

[0032] In another aspect, the present invention provides a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to execute the above-described large language model-enhanced power load forecasting method.

[0033] In another aspect, the present invention provides an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus, and the processor calls logical instructions in the memory to execute the above-mentioned large language model-enhanced power load forecasting method.

[0034] In another aspect, the present invention provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer executes the above-described large language model-enhanced power load forecasting method.

[0035] Beneficial Effects: This invention's power load forecasting method enhanced by a large language model divides the time series into segments with a certain step size. Multi-scale feature information is extracted through convolution with kernels of different sizes. Simultaneously, the convolved time series undergoes channel attention processing to enable channel-level feature learning, strengthening feature extraction and adding channel weights to the time series. This focuses attention on key parts while suppressing attention to unimportant features. The processed time series is then transformed into text information understandable by the large language model using a cross-attention mechanism. Furthermore, to help the large language model better understand the content represented by the time series, a cue prefix is ​​introduced to aid understanding of the input time series information. The cue prefix and the transformed text information are input into a frozen large model. Freezing maintains all parameters and weights of the original large model without updating, preventing catastrophic forgetting and preserving the powerful general inference and pattern recognition capabilities of the LLM. Backpropagation is unnecessary to calculate the gradient of the entire large model, reducing computational load. The inference capability of the large model is used to complete the time series prediction. The output result is a truncated version of the predicted time series content, and finally, the output result is mapped back to the time series output to complete the time series prediction. This invention enhances feature extraction by introducing a channel attention mechanism, thereby improving prediction accuracy with minimal increase in computation. Attached Figure Description

[0036] Figure 1 This is the overall flowchart of the power load forecasting method enhanced by the large language model of the present invention;

[0037] Figure 2 This is a flowchart illustrating the overall operation of the time series prediction model in this invention.

[0038] Figure 3 This is a schematic diagram of the cross-membrane state transition process of the large language model in this invention;

[0039] Figure 4 This is a flowchart illustrating how the feature extraction process is enhanced by introducing a channel attention mechanism in this invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0041] The following is combined with Figures 1-4 This invention describes the power load forecasting method enhanced by a large language model.

[0042] Example 1: This example provides a power load forecasting method enhanced by a large language model. The most important objective of this method is to improve prediction accuracy by introducing a channel attention mechanism to enhance feature extraction, thereby increasing the computational cost significantly.

[0043] The channel attention mechanism used in this invention is a technique focused on adjusting the weights of different channels in a feature map to highlight important features. To learn channel-dimensional features, the main idea is to convolve the image and then pass it through a pooling layer. Average pooling or max pooling is used to allow the machine learning to focus on the average or maximum value of the channel dimension. Specifically, this compresses the spatial features of the feature map into a single value, preserving the information between channels. This reduces the model's parameters and computational cost, improving computational efficiency. The fully connected layer processes the input feature layer, allowing it to learn the weights of each channel. These weights reflect the channel's contribution to the task, making the model focus more on features that contribute to the task. Multiplying these weights by the original feature layer allows the network to focus more on useful features, or selectively emphasize useful features and suppress less useful features by learning global information. During convolution, the convolution of each channel involves summation, so the channel feature information and the spatial relationships learned by the convolution kernel are mixed together. Therefore, it is necessary to separate these two feature information sets so that the model can learn the feature relationships between the channels.

[0044] The specific steps of the method in this embodiment are as follows: Figure 1 As shown, it includes:

[0045] S1. Obtain the time-series data of the power transformer monitoring system and perform preprocessing.

[0046] In this embodiment, the time series datasets ETTh1, ETTh2, ETTm1, and ETTm2 from the power sector are used as datasets. These belong to the ETT (Electricity Transformer Temperature) series and are mainly used for power load forecasting, transformer temperature forecasting, and related time series analysis. The data source is the monitoring system of power transformers (such as oil temperature, load, current, and voltage). The time granularity is typically hourly or every 15 minutes of sampling data, usually covering 1 to 2 years of continuous monitoring data, which is suitable for long-term trend analysis.

[0047] The acquired time-series data is processed by imputing missing values, removing outliers, normalizing and standardizing to facilitate subsequent feature extraction.

[0048] This embodiment uses the Time-LLM time series prediction model as a basis for improvement. Before processing the above dataset, the time series prediction model is first introduced. Figure 2 As shown, Time-LLM is a reprogrammed large language model framework that leverages large language models to perform time-series predictions while maintaining the integrity of the core model. From Figure 3 As we can see, the large language model compresses its vocabulary, retaining only word groups relevant to time-series information. The segmented time series and the compressed word groups are then input into a cross-attention mechanism to obtain textual information reflecting the time series. The large language model uses this textual information for the next prediction, and then projects the predictions back into a time series output. Furthermore, to enable the large language model to better understand the transformed text and thus improve inference performance, a hint as a prefix (pap) is introduced. This is a novel idea that enriches the input time series with additional context and provides task instructions in natural language form. This hint provides declarative guidance to the transformed text.

[0049] This technology has the following main features: (1) It introduces a new concept, using a frozen large language model for inference without changing the training backbone of the large language model, thus saving time while completing the task. (2) Since the large language model still performs well in scenarios with few or no samples, it has great potential to be used as a time series learner for time series prediction. (3) It provides a scalable paradigm for the reprogramming framework of the large language model, injecting new functions beyond its original training into the large language model. (4) It completes cross-modal conversion between time series and text, enabling the feature information of the time series to be understood and used by the large language model.

[0050] As the introduction shows, Time-LLM extracts information from time series by segmenting the time series, with each segment containing changes at multiple points. However, it cannot perform detailed segmentation and analysis on complex parts, while segmenting stable parts multiple times is computationally wasteful. It also cannot allocate attention to important features. Therefore, it is necessary to add weights to the feature extraction of time series so that the computer can better focus on important features to better grasp the changes in the time series. Thus, this invention constructs a new model called DCATime. The main idea is to extract local features and long-term trends from the segmented time series using different convolutional kernels before time series to text conversion. These two feature layers are then input into a dynamic channel attention mechanism module that processes time series data to learn the channel dimension weights, and finally, feature fusion is performed. This allows the model to concentrate more resources on important parts, providing more accurate feature information when converted to text. This enables large language models to better understand the changing characteristics and trends of time series, thereby achieving better predictions. Specific methods are as follows... Figure 4 As shown, it includes:

[0051] S2. Divide the time-series data into several fixed-length time segments according to a set length. Input each time segment into a 1*1 convolutional layer to upscale the input to a high-dimensional feature map, obtaining the initial convolutional features. Then, perform secondary convolutions on the obtained initial convolutional features using small and large convolutional kernels respectively, obtaining two types of convolutional features after secondary convolution.

[0052] The segmented time patches are convolved with kernels of different sizes to extract features. Small kernels are used to focus on prominent details in the short term, while large kernels are used to focus on dependencies that influence long-term trends. By using convolution operations with kernels of different sizes to capture feature information from time patches, the model can effectively fuse features from different time scales, improving its ability to process complex time-series data and increasing its diversity, helping it make better choices among different types of features. This approach reduces the risk of overfitting to features at a single scale, thereby improving generalization ability—a multi-scale feature processing method.

[0053] S3. Perform global average pooling and global max pooling on the two convolutional features after the second convolution to reduce the overall complexity.

[0054] The feature map after the second convolution is subjected to global average pooling. For each channel, the spatial dimension of that channel, i.e., the time length, is compressed into a single value. The feature map of each channel is compressed into a global descriptor, which represents the average response of that channel in the entire space.

[0055] At the same time, the feature map is subjected to global max pooling, which also generates a global descriptor representing the maximum response of the channel in the entire space;

[0056] The input feature map is T*C*N, where T is the time series length, C is the number of channels, and N is the batch size. The output is a 1*C*N vector. Keeping N constant does not affect channel variations.

[0057] The convolutional temporal feature map utilizes both average pooling and max pooling operations. Average pooling provides global information about the feature distribution, compressing the feature map of each channel into a single value while preserving statistical information between channels. Global average pooling reduces the number of model parameters and computational cost, improving computational efficiency. Max pooling provides information about the most salient features, selecting the maximum value within each channel to retain the most prominent feature information. This helps the model identify important features and allows it to ignore irrelevant or noisy features, improving generalization ability. By combining these two methods, the channel attention module can better understand which feature channels are more important in a given input time series.

[0058] S4. Input the results of average pooling and max pooling into a shared fully connected layer, perform two full connections and one activation, then add the two output results to obtain two sets of channel weights, then sum the two sets of channel weights to generate a comprehensive channel weight, and use this comprehensive channel weight to perform weighted fusion of the two convolutional features after the second convolution to obtain the fused feature.

[0059] The shared connection layer consists of two fully connected layers. The first fully connected layer compresses the number of channels C of the feature map to the C / r dimension through a compression operation, where r is a hyperparameter representing a scaling factor, such as 16. The second fully connected layer restores the number of channels C of the feature map to its original value. A ReLU activation function is added between the two fully connected layers to learn the non-linear relationship of the features. Finally, a Sigmoid activation function is added to the two fully connected layers to adjust the weight of each channel to a value between 0 and 1, representing the saliency of that channel.

[0060] The results from both pooling operations are fed into a shared fully connected layer, followed by two more fully connected layers (which can be replaced by 1x1 convolutions). The first fully connected layer has fewer neurons, while the second has the same number of neurons as the input feature layer. The fully connected layer automatically generates weights for each channel through parameter learning. These weights reflect the channel's contribution to the task. By learning these channel weights, the fully connected layer enhances the feature representation capability, enabling the model to better understand the input data. The fully connected layer allows the model to focus more on features that contribute to the task, improving the quality of feature representation. After the two fully connected layers, the results from the two pooling operations are weighted and summed, then a sigmoid function is applied to fix the value between 0 and 1. Finally, this weighted sum is multiplied by the original input feature layer. This weighted feature map highlights important features and suppresses unimportant features, improving the model's prediction accuracy and generalization ability.

[0061] Then, the feature maps processed using both large and small convolutional kernels are processed and passed through the channel attention module to obtain the channel weights of the two sets of feature maps. The weighted sums are then performed to obtain a channel importance weight vector with a comprehensive effect. This weight vector is then multiplied channel by channel with the original input feature map, and finally, the fused feature with channel-dimensional weights is output.

[0062] To address the information gap and differences in information capture capabilities caused by the varying receptive fields of different convolutional kernels, a feature fusion approach is employed. This involves convolving the convolutional layers of time-series segments with different kernels, processing each layer through a channel attention mechanism, and then weighted summing the results. The fused features ensure that each time-series segment corresponds to a single output while incorporating the processing effects of both convolutional kernels. This allows the output time series to learn the feature relationships between channels, selectively emphasizing useful features or suppressing useless ones, thereby improving the effectiveness of the final feature acquisition.

[0063] S5. Input the fused features into the frozen large language model, map them into text embeddings through a cross-attention mechanism, and output the load prediction results for future periods under the guidance of prompt prefixes.

[0064] The fused features with channel-dimensional weights are combined with time-related vocabulary groups compressed from a large language model. A cross-attention approach is used to convert time series data into vocabulary. The segmented time series blocks serve as input information Q, while the compressed vocabulary groups serve as K and V. The final output V represents the text information. The conversion formula is as follows:

[0065] ;

[0066] The converted text information is not one patch corresponding to one or a few words, but all word groups related to this patch occupy a certain proportion. The more important the word, the larger the proportion. Therefore, improving the extraction of temporal features is equivalent to changing the proportion of words after conversion, so that the information can better reflect the trend of temporal change.

[0067] S6. Input the converted text information and prompt prefix into the large language model. The prompt prefix contains the task description and statistical information of the maximum, minimum and average values ​​of the time segment, which is used to guide the large language model to complete the load prediction for the next time period.

[0068] After inference and prediction using a large language model, the predicted results are derived. The hints are removed, and the predicted information is projected into a time series as the load forecast for future periods. Tables 1 and 2 show the results of long-term and zero-sample forecasts using the method of this embodiment, respectively.

[0069] Table 1 Long-term forecast results

[0070]

[0071] Table 2 Zero-sample prediction results

[0072]

[0073] Experimental results show that the improved model (DCATime) significantly improves performance compared to other prediction models in both long-term and zero-sample predictions. The overall prediction results are extremely close to the true values ​​in both trend and numerical terms, with smoother curves, less drastic fluctuations, and more stable predictions. It improves the accuracy of time-series predictions with minimal increase in computation, namely by reducing the MSE and MAE between predicted and true values.

[0074] Example 2: This example provides a non-transitory computer-readable storage medium storing computer instructions that cause a computer to execute a large language model-enhanced power load forecasting method, which includes the following steps:

[0075] Acquire time-series data from the power transformer monitoring system and perform preprocessing;

[0076] The time series data is divided into several time segments of fixed length according to a set length, and each time segment is passed through a convolutional layer to obtain the initial convolutional features.

[0077] The obtained initial convolution features are then subjected to secondary convolution using small and large convolution kernels respectively, resulting in two types of convolution features after secondary convolution.

[0078] The two convolutional features after the second convolution are subjected to global average pooling and global max pooling respectively, and then passed through a fully connected layer to obtain two sets of channel weights. The two sets of channel weights are then weighted and summed to generate a comprehensive channel weight. Based on this comprehensive channel weight, the two convolutional features after the second convolution are weighted and fused to obtain a fused feature.

[0079] The fused features are input into a frozen large language model, mapped to text embeddings via a cross-attention mechanism, and output load prediction results for future periods under the guidance of cue prefixes.

[0080] Example 3: This example provides an electronic device that may include a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor can call logical instructions from the memory to execute a large language model-enhanced power load forecasting method. This method includes the following steps:

[0081] Acquire time-series data from the power transformer monitoring system and perform preprocessing;

[0082] The time series data is divided into several time segments of fixed length according to a set length, and each time segment is passed through a convolutional layer to obtain the initial convolutional features.

[0083] The obtained initial convolution features are then subjected to secondary convolution using small and large convolution kernels respectively, resulting in two types of convolution features after secondary convolution.

[0084] The two convolutional features after the second convolution are subjected to global average pooling and global max pooling respectively, and then passed through a fully connected layer to obtain two sets of channel weights. The two sets of channel weights are then weighted and summed to generate a comprehensive channel weight. Based on this comprehensive channel weight, the two convolutional features after the second convolution are weighted and fused to obtain a fused feature.

[0085] The fused features are input into a frozen large language model, mapped to text embeddings via a cross-attention mechanism, and output load prediction results for future periods under the guidance of cue prefixes.

[0086] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0087] Example 4: This example provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a large language model-enhanced power load forecasting method, which includes the following steps:

[0088] Acquire time-series data from the power transformer monitoring system and perform preprocessing;

[0089] The time series data is divided into several time segments of fixed length according to a set length, and each time segment is passed through a convolutional layer to obtain the initial convolutional features.

[0090] The obtained initial convolution features are then subjected to secondary convolution using small and large convolution kernels respectively, resulting in two types of convolution features after secondary convolution.

[0091] The two convolutional features after the second convolution are subjected to global average pooling and global max pooling respectively, and then passed through a fully connected layer to obtain two sets of channel weights. The two sets of channel weights are then weighted and summed to generate a comprehensive channel weight. Based on this comprehensive channel weight, the two convolutional features after the second convolution are weighted and fused to obtain a fused feature.

[0092] The fused features are input into a frozen large language model, mapped to text embeddings via a cross-attention mechanism, and output load prediction results for future periods under the guidance of cue prefixes.

[0093] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A power load forecasting method enhanced by a large language model, characterized in that, Includes the following steps: Acquire time-series data from the power transformer monitoring system and perform preprocessing; The time series data is divided into several time segments of fixed length according to a set length, and each time segment is passed through a convolutional layer to obtain the initial convolutional features. The obtained initial convolution features are then subjected to secondary convolution using small and large convolution kernels respectively, resulting in two types of convolution features after secondary convolution. The two convolutional features after the second convolution are subjected to global average pooling and global max pooling respectively, and then passed through a fully connected layer to obtain two sets of channel weights. The two sets of channel weights are then weighted and summed to generate a comprehensive channel weight. Based on this comprehensive channel weight, the two convolutional features after the second convolution are weighted and fused to obtain a fused feature. The fused features are input into a frozen large language model, mapped to text embeddings via a cross-attention mechanism, and output load prediction results for future periods under the guidance of cue prefixes.

2. The power load forecasting method enhanced by a large language model according to claim 1, characterized in that, The acquisition and preprocessing of time-series data from the power transformer monitoring system are as follows: Time-series data from various monitoring systems of power transformers, including oil temperature, load, current, and voltage data, are collected at a sampling rate of once per hour or every 15 minutes. The acquired time-series data are then processed by imputing missing values, removing outliers, normalizing, and standardizing.

3. The power load forecasting method enhanced by a large language model according to claim 1, characterized in that, Each time segment is passed through a convolutional layer to obtain the initial convolutional features. Then, the time segments are input into a 1*1 convolutional layer to upscale the input to a high-dimensional feature map. Then, the feature map is convolved twice using both large and small convolution kernels. The small convolution kernel is used to capture local feature changes, while the large convolution kernel is used to capture the global shape of the time sequence.

4. The power load forecasting method enhanced by a large language model according to claim 3, characterized in that, The two convolutional features after the second convolution are then subjected to global average pooling and global max pooling, respectively; specifically: The feature map after the second convolution is subjected to global average pooling. For each channel, the spatial dimension of that channel, i.e., the time length, is compressed into a single value. The feature map of each channel is compressed into a global descriptor, which represents the average response of that channel in the entire space. At the same time, the feature map is subjected to global max pooling, which also generates a global descriptor representing the maximum response of the channel in the entire space; The input feature map is T*C*N, where T is the time series length, C is the number of channels, and N is the batch size. The output is a 1*C*N vector.

5. The power load forecasting method enhanced by a large language model according to claim 4, characterized in that, After global average pooling and global max pooling, the feature map is input into a shared fully connected layer; The shared connection layer consists of two fully connected layers. The first fully connected layer compresses the number of channels C of the feature map to the C / r dimension through a compression operation, where r is a hyperparameter representing the scaling factor. The second fully connected layer restores the number of channels in the feature map to the original number C. A ReLU activation function is added between the two fully connected layers to learn the non-linear relationship of the features. Finally, a Sigmoid activation function is added to the two fully connected layers to adjust the weight of each channel to a value between 0 and 1, representing the significance of that channel.

6. The power load forecasting method enhanced by a large language model according to claim 5, characterized in that, After obtaining the weights of each channel in the feature map, the two sets of channel weights are summed to generate a comprehensive channel weight. Based on this comprehensive channel weight, the two convolutional features after the second convolution are weighted and fused to obtain the fused features. Specifically: After processing the feature maps using both large and small convolutional kernels, they pass through a channel attention module to obtain the channel weights of the two sets of feature maps. Then, a weighted sum is performed to obtain a channel importance weight vector with a comprehensive effect. This weight vector is then multiplied channel by channel with the original input feature map, and finally, a fused feature with channel-dimensional weights is output.

7. The power load forecasting method enhanced by a large language model according to claim 6, characterized in that, The fused features are input into a frozen large language model, mapped to text embeddings via a cross-attention mechanism, and output load prediction results for future time periods under the guidance of cue prefixes; specifically: The fused features with channel-dimensional weights are combined with time-related vocabulary groups compressed from a large language model. A cross-attention approach is used to convert time series data into vocabulary. The segmented time series blocks serve as input information Q, while the compressed vocabulary groups serve as K and V. The final output V represents the text information. The conversion formula is as follows: ； The converted text information and the prompt prefix are input into the large language model. After the large language model makes inference and prediction, the prediction result is obtained. The prompt part is removed, and the predicted information is projected into a time series as the load prediction result for the future period.

8. The power load forecasting method enhanced by a large language model according to claim 7, characterized in that, The prompt prefix contains statistical information on the task description and the maximum, minimum, and average values ​​of the time segment, which is used to guide the large language model to complete the load prediction for the next time period.

9. A non-transitory computer-readable storage medium, characterized in that, It stores computer instructions that cause the computer to execute the large language model-enhanced power load forecasting method as described in any one of claims 1-8.

10. An electronic device, characterized in that, include: The system includes a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor calls logical instructions from the memory to execute the large language model-enhanced power load forecasting method according to any one of claims 1-8.

Images