Short-term power consumption prediction method and device based on LLM decomposition for different types of users and storage medium

Abstract

The application provides a short-term power consumption prediction method and device based on LLM for different types of users, and a storage medium, and belongs to the field of power system power consumption prediction; the method comprises the following steps: collecting power consumption and meteorological data of multiple types of users, and preprocessing the data; dividing the preprocessed data in proportion; building an LLM large language model, inputting meteorological data and power consumption data for self-supervised pre-training, and determining the input and output dimensions according to the input and output data characteristics; then performing a classification task module in the LLM large language model to divide user types; inputting the user classification results and user power consumption data into a Transformer model as input, and training the Transformer model; the application has high prediction accuracy, solves the problem that the prior art cannot effectively combine climate data to realize high-precision power consumption prediction, and is suitable for short-term user power consumption prediction.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of power system electricity consumption prediction, in particular to a short-term electricity consumption prediction method and device for decomposing different types of users based on an LLM and a storage medium. BACKGROUND

[0002] In recent years, with the acceleration of global energy structure transformation, the influence of climate variability on the power system has become increasingly significant, bringing many challenges and opportunities to the power industry.

[0003] At present, for the problem of medium and long-term electricity decomposition under new energy access, many studies have proposed relevant strategies and methods, but most of them ignore the nested optimization of the connection between the medium and long-term market and the spot market transaction. In addition, climate variability covers all aspects of the power industry, time scales and geographical ranges, and affects the electricity consumption of multiple types of users, including changes in time series characteristics and climate characteristics, which poses a great challenge to user electricity consumption analysis and prediction.

[0004] Therefore, in order to accurately depict the characteristics of user electricity consumption, explore the correlation between user types, time series characteristics and climate characteristics of electricity consumption, and improve the short-term prediction accuracy of user electricity consumption, it is necessary to carry out research on user electricity consumption data analysis and prediction considering climate variability. SUMMARY

[0005] In order to solve the above technical problems, the application proposes a short-term electricity consumption prediction method for decomposing different types of users based on an LLM, a device and a storage medium, which uses artificial intelligence large models and other algorithms to include climate variability factors in user electricity consumption data analysis, and realizes accurate short-term electricity consumption prediction.

[0006] The technical scheme adopted by the application is as follows: a short-term electricity consumption prediction method for decomposing different types of users based on an LLM, comprising the following steps:

[0007] Step 1, data acquisition and preprocessing: collect multi-type user electricity consumption data and meteorological data, and perform data preprocessing, divide the preprocessed data by proportion to obtain a training set, a validation set and a test set;

[0008] Step 2, build an LLM large language model, use the meteorological data and electricity consumption data in the training set to pre-train the LLM large language model in a self-supervised manner, determine the input and output dimensions according to the input and output data characteristics, and then execute the classification task module in the LLM large language model to divide user types and output user type results;

[0009] Step 3, input the user type results and user electricity consumption data into the Transformer model together, and train the Transformer model.

[0010] Furthermore, the electricity consumption data and meteorological data of various types of users in step 1 are obtained through the electricity consumption information collection system and the National Meteorological Data Network. Data preprocessing includes labeling and encoding of non-numerical data, as well as handling of missing values ​​and outliers.

[0011] Furthermore, in step 2, during the self-supervised pre-training of the LLM large language model using meteorological and electricity consumption data from the training set, a series of forward and backward propagation steps are employed to adjust the model parameters. In the forward propagation stage, the model receives the input sequence and uses the embedding layer and multi-layer network to predict the value for the next time step. Then, the error between the predicted value and the actual value is calculated using the loss function. In the backward propagation stage, based on the error calculated above, the gradient of the loss function relative to the model parameters is calculated, and then the optimization algorithm is used to update the model parameters.

[0012] Furthermore, the classification task module in the LLM large language model focuses on handling classification tasks, and the process is as follows:

[0013] First, the original classification scores are mapped to category probability vectors through a linear layer. Then the probability vector The input is fed into a softmax layer, where it is transformed into a probability distribution for each class. Finally, the output yields the probability distribution, i.e., the predicted probabilities of each category. The formula for the above process is expressed as follows:

[0014] ;

[0015] In the formula: The weight matrix of the linear layer. It is a high-dimensional feature vector. For bias terms, For category The probability, For category Attention score This represents the total number of categories.

[0016] Furthermore, the Transformer model consists of two parts: an encoder and a decoder. Each part contains several functionally identical layers. The layers have similar structures and are independent of each other. Each layer of the encoder has a multi-head self-attention mechanism and a feedforward neural network. Each layer of the decoder has a masked multi-head self-attention mechanism, an encoder-decoder attention mechanism, and a feedforward neural network.

[0017] Further, the user type result output in step 2 is n data sets of different user types, and the training of the Transformer model in step 3 is to put the n data sets of different user types generated in step 2 into the Transformer model respectively for training. The trained Transformer model will prioritize the classification of user types when predicting, and then assign weights to different data for accurate prediction.

[0018] Further, the feedforward neural network includes two linear transformations and an activation function.

[0019] Further, it further includes step 4: retraining the Transformer model on the combined data set of the training set and the validation set to obtain the model with the optimal performance.

[0020] A computer device comprising a memory, a processor and a computer program stored on the memory, the processor executing the computer program to implement the steps of the method.

[0021] A computer readable storage medium having stored thereon a computer program / instructions which, when executed by a processor, implement the steps of the method.

[0022] The application has the beneficial effects of the prior art: the core of the application is to use the LLM large language model to classify different user types, and then use the Transformer model to judge the user type and then make a prediction, thereby improving the prediction accuracy. First, the collected data may contain non-numeric data, such as user type, weather condition description, etc. First, label encoding is performed on it, and different categories are assigned a unique numeric code so that it can be identified and processed by the subsequent model, completing data preprocessing. Next, the preprocessed data is divided into training set, validation set and test set according to the proportion, the training set is used to let the model learn the data pattern rule, the validation set is used to evaluate the model performance during training and adjust the hyperparameters to prevent overfitting, and the test set is used to finally test the generalization ability of the model. Then build the LLM large language model, determine the input and output dimensions of the model according to the feature quantities of the input meteorological data and power consumption data, and use the training set data to pre-train the model. Self-supervised learning as an unsupervised learning method can let the model automatically generate labels for training by learning the internal structure of the data, so that it can master the potential relationship between meteorological data and power consumption data. After completing the pre-training, execute the classification task module of the LLM large language model, and comprehensively analyze the user's power consumption behavior and meteorological data according to the input data, and divide the users into different types. Then, put the user type results divided by the LLM large language model and the user power consumption data into the Transformer model together. The Transformer model is based on the attention mechanism and has strong sequence modeling capability. In the training process, the performance is evaluated using the validation set and the learning rate, batch size and other hyperparameters are adjusted to improve the prediction accuracy. Finally, the data of the training set and the validation set are combined, and the Transformer model is retrained with more data after merging to let the model learn more data features and patterns. After multiple iterations of training, the model with the best performance is obtained to ensure that the user power consumption prediction can maintain high accuracy at different time scales. BRIEF DESCRIPTION OF DRAWINGS

[0023] The application will be further described below in conjunction with the drawings:

[0024] Figure 1 The method flowchart provided for the embodiments of the application;

[0025] Figure 2 The calculation process diagram of the self-attention mechanism layer provided for the embodiments of the application;

[0026] Figure 3 The Transformer model structure diagram provided for the embodiments of the application;

[0027] Figure 4 The structural block diagram of the computer device provided for the embodiments of the application. DETAILED DESCRIPTION

[0028] As Figures 1-4 shown, the present application provides a method for short-term prediction of different types of user electricity consumption based on LLM. The basic idea is: after obtaining the multi-type user electricity consumption data in the electricity information collection system and the meteorological data in the national meteorological data network, explore the user electricity consumption short-term prediction optimization mechanism of fusing LLM large language model and Transformer model, consider the variability of climate and the diversity of user electricity consumption behavior. When carrying out data analysis and model construction work, it is very important to pretreat the collected data. Because there may be non-numeric data such as user type and weather condition description in the data, the model cannot directly process it. Therefore, the method of label encoding will be used to assign a unique numeric code to each different category, so that these data can be identified by the subsequent model. After completing the pretreatment, the data is divided into training set, validation set and test set according to the proportion. The training set is used to provide the pattern and rule of model learning data; the validation set is used to evaluate the performance of the model during training, and the hyperparameters are adjusted accordingly to prevent the model from overfitting; the test set is used to finally test the generalization ability of the model. Then, the LLM large language model is built, and the input and output dimensions of the model are determined according to the characteristic quantities of the input meteorological data and electricity consumption data. Self-supervised pre-training is carried out on the model using the training set data. Self-supervised learning, as an unsupervised learning method, can let the model automatically generate labels for training by mining the internal structure of the data, and thus grasp the potential relationship between meteorological data and electricity consumption data. After pre-training, the classification task module of the LLM large language model is started, which will comprehensively analyze the input meteorological data and user electricity consumption data, divide the users into different types, and output the user type results. Then, the user type results divided by the LLM large language model and the user electricity consumption data are input into the Transformer model. The Transformer model is based on attention mechanism and has strong sequence modeling ability. During training, the performance of the model is evaluated by means of the validation set, and the learning rate, batch size and other hyperparameters are adjusted to improve the prediction accuracy. Finally, in order to make the model learn more data features and patterns, the training set and validation set data are merged, and the Transformer model is retrained with the merged larger data set. After multiple iterations of training, the performance of the model reaches the optimal state, so as to ensure that the user electricity consumption prediction has high accuracy under different time scales.

[0029] Based on the above technical concept, the implementation steps of the method of the present application can be summarized as:

[0030] Step 1, data acquisition and preprocessing: multiple types of user power consumption data and meteorological data can be collected from the power information collection system and the national meteorological data network, non-numerical data is labeled and encoded, missing values and outliers are processed and normalized, and data preprocessing is completed; then, the preprocessed data is divided in proportion to obtain the training set, validation set and test set;

[0031] Step 2, build LLM large language model, input meteorological data and power consumption data for self-supervised pre-training, determine input and output dimensions according to input and output data characteristics; then execute the classification task module in the LLM large language model to divide user types and output user type results;

[0032] Step 3, then input the user type results and user power consumption data into the Transformer model, train the Transformer model to achieve higher accuracy prediction;

[0033] Step 4, retrain the Transformer model on the combined data set of the training set and the validation set to obtain the model with the best performance.

[0034] The principle of model self-supervised pre-training in step 2 is as follows:

[0035] In the self-supervised pre-training process of the LLM large language model, a series of forward propagation and back propagation steps are used to adjust the model parameters. In the forward propagation stage, the model receives the input sequence, uses the embedding layer and the multi-layer network in it to make predictions for the next time step value. Then, the error between the predicted value and the actual value is calculated by the loss function. In the back propagation stage, the gradient of the loss function with respect to the model parameters is calculated based on the error, and the optimization algorithm is used to update the model parameters. This process can be represented by the formula:

[0036] ;

[0037] In the formula: is a given sequence, is the log-likelihood value of the sequence, is the maximum likelihood function, is the expectation under different permutations, represents the generation probability function of the model parameters, is the th randomly permuted text span, represents the tokenized text after replacing the missing sequence in the original sequence, is the set of all possible permutations, is the length of the span . Through pre-training, the internal rules of the data can be learned, thereby improving the accuracy of the output.

[0038] The classification task module in the LLM large language model is a logical mapping function unit that maps data to predefined categories, relying on the understanding ability of pre-training data and guided by prompt engineering or labeled data fine-tuning. Specifically, when processing regression tasks, the LLM large language model calculates the predicted value through the following steps: integrating multi-dimensional information in the input sequence and combining the context, outputting a high-dimensional feature vector . This vector completely represents the integrated features at time step . Then, the high-dimensional feature vector is input into a linear layer (fully connected layer), and finally mapped to the prediction value space. The role of the linear layer is to convert the high-dimensional feature vector into a scalar, i.e., the predicted value . The specific formula is as follows:

[0039] ;

[0040] In the formula: is the feedforward neural network; represents the attention mechanism; is the input sequence feature matrix; is the activation function; is the weight matrix of the linear layer; is the bias term.

[0041] The classification task module in the LLM large language model focuses on processing classification tasks, so the output layer of the LLM large language model needs to be modified. First, the original classification scores are mapped to a class probability vector through a linear layer; then the probability vector is input into the softmax layer to convert it into a probability distribution of the class; finally, the probability distribution is output, i.e., the predicted probability of each class. The specific formula is as follows:

[0042] ;

[0043] In the formula: is the weight matrix of the linear layer, is the high-dimensional feature vector, is the bias term, is the probability of class , is the attention score of class , and is the total number of classes.

[0044] In the classification task of the LLM (Large Language Model), obtaining the original classification score involves two core steps. First, text semantic encoding: the input text is first processed through the Transformer model structure of the LLM, using self-attention and feedforward networks to transform each token into a hidden state containing contextual information. Then, by extracting the hidden states of the tokens, or averaging or pooling all the hidden states, a global semantic vector representing the overall meaning of the text is obtained. Next is the mapping from the global semantic vector to the original classification score. This global semantic vector is input into a task-specific linear layer (classification head). This linear layer transforms the high-dimensional semantic vector into an N-dimensional original score vector, consistent with the number of predefined categories, through matrix multiplication of the weight matrix and the global semantic vector, plus a bias term. The value of each dimension corresponds to the model's original preference for the text belonging to that category. The values ​​have no probabilistic meaning but reflect category priority, forming the basis for subsequent probability distribution generation.

[0045] The Transformer model consists of two main parts: an encoder and a decoder. Each part contains several functionally identical layers, which are structurally similar but independent of each other. Within each layer of the encoder, there are two key sub-modules. The first is a multi-head self-attention mechanism, which simultaneously focuses on multiple positions in the input sequence to capture and encode the complex relationships between elements within the sequence. This multi-head design allows the model to learn information in parallel across different subspaces, thereby improving the model's understanding of the sequence. The second is a feedforward neural network, a densely connected network layer that further processes the output of the self-attention layer, typically involving two linear transformations and an activation function.

[0046] Specifically, the self-attention mechanism works by processing the input data. Linear transformation to generate query ,key Sum matrix:

[0047] ;

[0048] in, , , This is the weight matrix learned by the model, used to transform the input data. The core computation of the self-attention mechanism can be represented by the formula for scaled dot product attention:

[0049] ;

[0050] In the above formula, The dot product calculation determines the similarity between different elements, while the scaling factor... The scale of the dot product is adjusted to ensure the stability of the computation. The function converts these similarities into a probability distribution, assigning a weight to each value.

[0051] In addition, the Transformer model employs a multi-head attention mechanism to enhance the processing capacity of the model:

[0052] ;

[0053] In the formula: is the output of the th attention head, is the weight matrix.

[0054] Each head is an independent attention mechanism that focuses on different information subspaces, and the expression is as follows:

[0055] .

[0056] This step linearly transforms the results of multi-head attention through an output weight matrix, to integrate the information captured by different heads and form a comprehensive output representation.

[0057] The output of the self-attention layer is passed to a feed-forward neural network, which includes two linear transformations and a ReLU activation function:

[0058] ;

[0059] In the formula: is the feed-forward neural network function; represents the input data from the previous layer (or the self-attention layer); and are the weight matrices of the first and second layers of the network, respectively; and are the bias vectors of the first and second layers of the network, respectively.

[0060] In each layer of the decoder, there are three main sub-modules. First, there is a masked multi-head self-attention mechanism, which is similar to the self-attention mechanism in the encoder, but with an additional masking mechanism. The masked multi-head self-attention mechanism ensures that only previous outputs can be relied upon when generating outputs at each time step, preventing information leakage and preventing the use of future information when making predictions. Second, there is an encoder-decoder attention mechanism, which allows each decoder layer to focus on the output results of the encoder and combine them with its own input to generate the final output sequence. Finally, the decoder is equipped with a feed-forward neural network, which has a similar network structure to the feed-forward neural network in the encoder, for further processing of the data.

[0061] The encoder-decoder attention mechanism of the Transformer model (also known as cross-attention) is a core component that connects the encoder and the decoder in the model, and mainly functions to establish semantic association between the input text and the output text.

[0062] When the decoder generates each token of the target text, the mechanism allows the decoder to simultaneously focus on two key pieces of information: one is the context generated by the decoder itself (handled by the masked self-attention), and the other is the semantic representation of the complete input text output by the encoder. It calculates the similarity (i.e., the attention weight) between the current hidden state of the decoder and the hidden state of all positions of the encoder, and then performs weighted summation on the encoder output to finally obtain a context vector that has fused the input information, thereby providing support for the decoder to generate accurate and input-semantic-compliant content.

[0063] In the Transformer model architecture, to compensate for the inherent defect of the self-attention mechanism that is not sensitive to token order, the present application proposes a position encoding method based on positive-sine and negative-sine oscillation. This method generates a unique position representation for an arbitrary length sequence without introducing additional trainable parameters, thereby explicitly injecting topological order information into the input embedding space. The position encoding uses a set of sine and cosine functions to generate, and its expression is as follows:

[0064] ;

[0065] In the formula: is the position index, is the dimension index, is the dimension of the embedding in the model.

[0066] The encoder-decoder attention mechanism allows the decoder to focus on the output of the encoder:

[0067] ;

[0068] Finally, the decoder generates the target sequence step by step based on the output of the encoder and its own output at the previous step.

[0069] Based on this, the present embodiment gives specific implementations of the method as follows:

[0070] First, collect data and perform data preprocessing. Collect multi-type user power consumption data from the power information collection system, and obtain climate data from the national weather data network, including load, weather, wind direction, wind power, and daily temperature data, etc. For non-numeric data such as weather and wind direction, convert them into specific numerical values using label encoding, find and process missing values and outliers, and then normalize the data to provide high-quality and suitable training data for the model.

[0071] The second step is to allocate and divide the data set. The preprocessed data is divided into training, validation, and test sets according to a certain ratio (70% training set, 15% validation set, and 15% test set) to adjust the model's hyperparameters and determine the model's prediction accuracy.

[0072] The third step is to build the LLM large language model. Initialize the model's basic parameters, such as the number of network layers, the number of neurons in each layer, batch size, and number of training epochs. Feed the data into the model for self-supervised pre-training to identify potential relationships and patterns between data points, thereby reducing the training difficulty of the LLM large language model.

[0073] The fourth step is to execute the classification task module of the LLM large language model. In this method, the LLM large language model is mainly used to classify different user types. Therefore, the input dimension is determined by the number of features in the input data, and the output dimension is obtained as part of the model training parameters in the form of training results.

[0074] The fifth step involves feeding the n datasets generated for different user types into the Transformer model for training. The trained Transformer model will no longer rely solely on meteorological and electricity consumption data for prediction; instead, it will prioritize classifying user types, assign weights to different data types, and then perform accurate predictions.

[0075] Step 6, Model Validation. The Transformer model trained on the training set is retrained on a combined training and validation dataset to fully utilize the data and train the best-performing Transformer model.

[0076] Figure 4 A structural block diagram of a computer device according to a specific embodiment of this application is shown. Figure 4 As shown, the computer device includes a memory and a processor, the memory storing instructions executable on the processor. When the processor executes the instructions, it implements the methods described in the above embodiments. The number of memories and processors can be one or more. This computer device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The computer device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0077] The computer device can also include a communication interface to facilitate communications with other devices. The various devices can be connected by various buses, and can be mounted on a common motherboard or in other manners as appropriate. The processor can process instructions for execution within the computer device, including instructions stored in the memory or on the storage device to display graphical information for a GUI on an external input / output device, such as a display device coupled to the interface. In other implementations, multiple processors and / or multiple buses can be employed as appropriate, as well as multiple memories and types of memory. Also, multiple electronic devices can be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). The buses can be divided into address buses, data buses, command buses, etc. As will be appreciated, however, that are shown in this figure, the bus connections can be implemented in a variety of manners, as appropriate, given the logic flows of this disclosure. Figure 4 In reality, only one bus is used to represent the buses, although multiple buses can be present in the computer device. Also, while the first computer system 1000 and the second computer system 1050 are shown with buses connecting the internal components, the bus implementations used to communicate data and control information between the components can be implemented in many ways.

[0078] Optionally, if the memory, the processor and the communication interface are integrated on a chip, the memory, the processor and the communication interface can communicate with each other through an internal interface.

[0079] It should be understood that the processor can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. It should be noted that the processor can be a processor supporting an advanced RISC machine (ARM) architecture.

[0080] The embodiment of the present application provides a computer readable storage medium (such as the memory described above), which stores computer instructions, and the program is executed by the processor to realize the method provided in the embodiment of the present application.

[0081] Optionally, the memory can include a program storage area and a data storage area, where the program storage area can store an operating system, application programs required by at least one function; the data storage area can store data created according to the use of the mapping computer device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-transient memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transient solid-state memory device. In some embodiments, the memory can optionally include a memory that is remotely arranged relative to the processor, and these remote memories can be connected to the mapping computer device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0082] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A short-term forecasting method for user electricity consumption based on LLM decomposition of different types, characterized in that: Includes the following steps: Step 1: Data Acquisition and Preprocessing: Collect electricity consumption data and meteorological data from various types of users, perform data preprocessing, and divide the preprocessed data into training set, validation set and test set according to the proportion; Step 2: Build an LLM large language model. Use meteorological data and electricity consumption data from the training set to conduct self-supervised pre-training on the LLM large language model. Determine the input and output dimensions based on the input and output data features. Then, execute the classification task module in the LLM large language model to divide user types and output the user type results. In Step 2, during the self-supervised pre-training of the LLM large language model using meteorological data and electricity consumption data from the training set, a series of forward and backward propagation steps will be used to adjust the model parameters. In the forward propagation stage, the model receives the input sequence and uses the embedding layer and multi-layer network to predict the value of the next time step. Next, the error between the predicted and actual values ​​is calculated using the loss function. In the backpropagation stage, the gradient of the loss function with respect to the model parameters is calculated based on the error calculated above, and then the model parameters are updated using an optimization algorithm. Step 3: Input the user type results and user electricity consumption data into the Transformer model and train the Transformer model. The Transformer model consists of two parts: an encoder and a decoder. Each part contains several layers with the same function. The layers have similar structures and are independent of each other. Each layer of the encoder has a multi-head self-attention mechanism and a feedforward neural network. Each layer of the decoder has a masked multi-head self-attention mechanism, an encoder-decoder attention mechanism, and a feedforward neural network. In the Transformer model architecture, a position encoding method based on sine and cosine oscillations is used. This method generates a unique position representation for a sequence of arbitrary length without introducing additional trainable parameters, thereby explicitly injecting topological sequence information into the input embedding space. The position encoding is generated using a set of sine and cosine functions.

2. The method for short-term forecasting of electricity consumption of different types of users based on LLM decomposition according to claim 1, characterized in that: The electricity consumption data and meteorological data of various types of users in step 1 are obtained through the electricity consumption information collection system and the National Meteorological Data Network. Data preprocessing includes labeling and encoding of non-numerical data, as well as handling of missing and outlier values.

3. The method for short-term forecasting of electricity consumption of different types of users based on LLM decomposition according to claim 1, characterized in that: The classification task module in the LLM large language model focuses on handling classification tasks, and its process is as follows: First, the original classification scores are mapped to category probability vectors through a linear layer. Then the probability vector The input is fed into a softmax layer, which transforms it into a probability distribution for each class. Finally, the output yields the probability distribution, i.e., the predicted probabilities of each category. The formula for the above process is expressed as follows: ； In the formula: The weight matrix of the linear layer. It is a high-dimensional feature vector. For bias terms, For category The probability, For category Attention score This represents the total number of categories.

4. The method for short-term forecasting of user electricity consumption based on LLM decomposition of different types as described in claim 1, characterized in that: The user type results output in step 2 are n datasets of different user types. In step 3, training the Transformer model involves feeding the n datasets of different user types generated in step 2 into the Transformer model for training. The trained Transformer model will prioritize classifying user types during prediction, and then assign weights to different data to make accurate predictions.

5. A short-term forecasting method for user electricity consumption based on LLM decomposition of different types, as described in claim 4, is characterized in that: The feedforward neuron Varro consists of two linear transformations and an activation function.

6. A short-term forecasting method for user electricity consumption based on LLM decomposition of different types, as described in claim 4, is characterized in that: It also includes step 4: retraining the Transformer model on the combined training and validation datasets to obtain the best-performing model.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-6.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-6.

Images