A power load prediction method based on multi-scale hypergraph neural network and large language model alignment

Abstract

The application discloses a power load prediction method based on multi-scale supergraph neural network and large language model alignment, comprising: mapping training samples and word embeddings of a large language model into multi-scale time feature representations and multi-scale text prototype representations respectively in a multi-scale feature extraction module; aggregating the multi-scale time feature representations into multi-scale superedge feature representations in a superedge feature extraction module; aligning the multi-scale text prototype representations and the multi-scale superedge feature representations in a cross-modal alignment module; inputting the multi-scale mixed prompts and the aligned multi-scale feature representations into a large language model for prediction after splicing; and training a power load prediction model comprising the multi-scale feature extraction module, the superedge feature extraction module, the cross-modal alignment module and the large model prediction module. The application can enhance the understanding ability of the large language model for the multi-scale mode of the power load sequence, thereby improving the accuracy and robustness of the model prediction.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of power load prediction, and particularly relates to a power load prediction method based on multi-scale supergraph neural network and large language model alignment. BACKGROUND

[0002] With the intelligent transformation and upgrading of modern social industries, the global energy consumption structure is undergoing profound changes. Under the dual driving of new urbanization construction and digital development of manufacturing industry, the urban power load is gradually increasing, which has become an important problem to be solved. Accurate prediction of future power load changes not only can realize precise coordination of power generation, transmission and distribution links, but also can provide dynamic decision support for the flexibility optimization of smart grid. Therefore, power load prediction has become one of the key technical breakthroughs in the digital evolution of modern energy systems.

[0003] Power load prediction in real life is a very challenging task: on the one hand, due to the influence of extreme events and newly built scenes, the power load sequence often has few samples or even zero samples, which brings great challenges to accurate power load prediction; on the other hand, due to the influence of human activities, the power load sequence presents different change patterns at different time scales, such as household electricity consumption usually decreases in the morning and increases at night at the day scale; at the week scale, it decreases during the week and increases on weekends. Considering the multi-scale pattern of power load sequence, more accurate prediction results can be obtained than considering only a single scale.

[0004] In order to make accurate power load prediction, many classic network structures have appeared. Convolutional Neural Networks (CNNs) have strong feature extraction ability and perform well in local feature capture; Graph Neural Networks (GNNs) are good at processing data with graph structure and can effectively mine the association relationship between data; Recurrent Neural Networks (RNNs) have natural advantages in processing sequence data and can capture the time sequence dependence relationship in the data; Transformers exhibit excellent performance in processing long sequence data and parallel computing with their self-attention mechanism. However, these methods are to train special models for specific scenarios, which are difficult to effectively migrate to different scenarios and difficult to handle few samples or even zero samples.

[0005] In recent years, with the rise of the application of large language models (LLMs) in natural language processing and computer vision, people have tried to apply large language models to power load forecasting. However, the existing methods based on large language models have obvious shortcomings. On the one hand, they ignore the differences between natural language and power load sequences in the multi-scale semantic space. Natural language focuses on semantic understanding and logical reasoning, while power load sequences emphasize numerical changes and pattern recognition in the time dimension. On the other hand, they lack the design of special prompts (Prompts) for power load forecasting, which cannot fully exploit the potential of large language models in handling complex data and tasks. These two key issues pose challenges to the successful application of large language models to power load forecasting and need to be further studied and effectively addressed. SUMMARY

[0006] In view of the above, the purpose of the present application is to provide a power load forecasting method based on multi-scale hypergraph neural network and large language model alignment, to align the multi-scale feature representations of natural language and power load sequences, and to enhance the understanding ability of large language models for multi-scale patterns of power load sequences by introducing a multi-scale hybrid prompt mechanism and utilizing multiple different prompts to enrich context information, thus having broad application prospects in the fields of power system operation, energy planning and energy efficiency management.

[0007] To achieve the above-mentioned purpose of the application, the technical solutions provided by the present application are as follows:

[0008] The power load forecasting method based on multi-scale hypergraph neural network and large language model alignment provided by the embodiments of the present application comprises the following steps:

[0009] Pretreat the power load sequence and construct training samples;

[0010] In the multi-scale feature extraction module, map the training samples and the word embedding based on the pre-trained large language model into multi-scale time feature representations and multi-scale text prototype representations, respectively;

[0011] In the hyperedge feature extraction module, aggregate the multi-scale time feature representations into multi-scale hyperedge feature representations based on the hyperedge mechanism of the multi-scale hypergraph neural network;

[0012] In the cross-modal alignment module, align the multi-scale text prototype representations and the multi-scale hyperedge feature representations based on cross-attention to obtain aligned multi-scale feature representations, and construct multi-scale hybrid prompts;

[0013] In the large model prediction module, the multi-scale hybrid prompts and aligned multi-scale feature representations are concatenated and then input into the pre-trained large language model for power load prediction. The power load prediction model, which includes a multi-scale feature extraction module, a hyperedge feature extraction module, a cross-modal alignment module and a large model prediction module, is trained based on the prediction loss.

[0014] The new power load sequence is input into the trained power load prediction model to obtain the prediction result.

[0015] Preferably, in the multi-scale feature extraction module, mapping the training samples and word embeddings based on the pre-trained large language model into multi-scale temporal feature representations and multi-scale text prototype representations respectively includes:

[0016] The training samples are normalized to obtain the normalized power load sequence. T represents the time window size. Based on the subsequence at scale s-1, a subsequence at scale s-th is generated through different aggregation windows. The subsequence at scale s-th is represented as... in Let D be the power load sequence at the s-th scale and time step t, where D is the feature dimension. Let N be the sequence length at the s-th scale. s-1 Let l be the length of the subsequence at the (s-1)th scale. s-1 Let be the size of the aggregation window at the (s-1)th scale. The calculation formula for the aggregation process is as follows:

[0017]

[0018] Where Agg(·) is an aggregation function, θ s-1 Let be the learnable parameters of the aggregation function at the (s-1)th scale. These are the input training samples after instance normalization.

[0019] Word embeddings based on pre-trained large language models Where V is the vocabulary size and P is the hidden layer dimension of the pre-trained large language model, the word embeddings U are first transformed into the initial text prototype representation through linear mapping. Where V′ << V, the multi-scale text prototype representation is then obtained through linear mapping. The calculation formula for the linear mapping process is as follows:

[0020]

[0021] Where Linear(·) is a linear mapping function, U s For the text prototype representation at the s-th scale, U s-1 For the text prototype representation at the (s-1)th scale, λ s-1V represents the learnable parameters of the linear mapping function at the (s-1)th scale. s The number of text prototypes represented at the s-th scale;

[0022] Finally, the time feature representations at S scales generated by the multi-scale feature extraction module are obtained {X}. 1 ,…,X s ,…,X S} and text prototype representation {U 1 ,…,U s ,…,U S}, where X s and U s These are the temporal feature representation and text prototype representation at the s-th scale (1≤s≤S), respectively.

[0023] Preferably, in the hyperedge feature extraction module, the hyperedgeization mechanism based on a multi-scale hypergraph neural network aggregates multi-scale temporal feature representations into multi-scale hyperedge feature representations, including:

[0024] Multi-scale temporal features are represented as {X} 1 ,…,X s ,…,X S Treat it as a node and initialize two parameters: hyperedge embedding. and node embedding Among them, X s For the time feature representation of the s-th scale (1≤s≤S), M s Let D be the number of hyperedges at the s-th scale, and D be the feature dimension. The point-edge association matrix H at the s-th scale is obtained through similarity calculation. s The calculation formula is as follows:

[0025]

[0026] in, and For learnable parameters, the tanh(·) activation function is used to perform nonlinear transformations, the ReLU(·) activation function is used to eliminate weak connections, and Linear(·) is a linear mapping function. The superscript T indicates the transpose operation. Finally, we obtain the point-edge affinity matrix {H} at S scales. 1 ,…,H s ,…,H S};

[0027] The i-th hyperedge at scale s hyperedge features Based on the vertex-edge association matrix H s The calculation formula is as follows, obtained through information aggregation:

[0028]

[0029] Where Avg(·) is the averaging operation. H is the point-edge incidence matrix at scale s. s The image shown is obtained through the hyperedge. Connected neighbor nodes, For the j-th node at scale s The temporal feature representation is that the set of all hyperedge features at scale s is ε. s Finally, we obtain hyperedge feature representations at S scales {ε 1 ,…,ε s ,…,ε S}

[0030] Preferably, a sparsity strategy is used to calculate the point-edge association matrix at the s-th scale, and the calculation formula is as follows:

[0031]

[0032] Where η∈[0,M s [] is the threshold of the TopK(·) function, representing the maximum number of neighbor hyperedges connected to the node, where n is the node index and m is the hyperedge index. To connect all the hyperedges to the nth node, Let be the sparsed vertex-edge incidence matrix at the s-th scale, based on The final point-edge affinity matrices {H} at S scales are obtained. 1 ,…,H s ,…,H S}

[0033] Preferably, in the cross-modal alignment module, the alignment of the multi-scale text prototype representation and the multi-scale hyperedge feature representation based on cross-attention alignment to obtain the aligned multi-scale feature representation includes:

[0034] Hyperedge feature representation at a mid-scale s based on multi-scale hyperedge feature representation ε s And multi-scale text prototype representation, text prototype representation at scale s, U s First, map it to the corresponding query. key Sum in It is the index of the number of heads in the attention. This represents the total number of heads. and Let be the learnable mapping matrix at scale s, D be the feature dimension, and P be the hidden layer dimension of the pre-trained large language model. Then, cross-attention is calculated to align the hyperedge feature representation and the text prototype representation. The calculation formula is as follows:

[0035]

[0036] Where Attn(·) is the cross-attention mechanism, softmax(·) is the activation function, and the superscript T denotes the transpose operation. For the scale s, the first The outputs of each attention head are aggregated to obtain the aligned feature representation Z of the multi-head attention outputs at scale s (1≤s≤S). s Finally, the aligned multi-scale feature representations {Z} at S scales are obtained. 1 ,…,Z s ,…,Z S}

[0037] Preferably, the multi-scale mixed cue includes:

[0038] Learnable tips Represented as in, The learnable cue at the s-th scale is obtained from the learnable initial embedding, 1≤s≤S, where S is the number of scales, and L s is the length of the learnable cue at scale s, and D is the feature dimension;

[0039] Data-related tips This includes data description hints π that provide basic background information on the input power load sequence for the pre-trained large language model, task description hints τ that guide the pre-trained large language model to understand and perform specific power load prediction-related tasks, and data statistical hints μ that include the statistical features of the power load sequence, including the input power load sequence and subsequences at different scales.

[0040] Ability Enhancement Tips These include logical thinking-based prompts (σ) that guide pre-trained large language models to solve problems step by step, and emotion manipulation-based prompts that simulate the influence of emotions on human decision-making. And inference-related class hints μ that guide pre-trained large language models for power load prediction.

[0041] Preferably, in the large model prediction module, the step of concatenating the multi-scale hybrid cue and the aligned multi-scale feature representation and inputting it into the large language model for power load prediction includes:

[0042] After obtaining multi-scale mixed cues, the learnable cues {P} at S scales are first processed. 1 ,...,P s ,...,P S} and the aligned multi-scale feature representation {Z 1 ,…,Z s ,…,Z SThe data is then concatenated, and further concatenated with data-related cues and capability-enhancing cues. The concatenated result is then input into a pre-trained large language model to obtain the output result, calculated using the following formula:

[0043]

[0044] Where [·,·] represents the concatenation operation, Z s Let be the aligned feature representation at the s-th scale (1≤s≤S). This is the output of the pre-trained large language model LLMs(·).

[0045] Preferably, the parameters of the main part of the pre-trained large language model are kept frozen during the training process.

[0046] Preferably, the output of the large language model is input into a linear layer for linear transformation, and the transformed result is then subjected to inverse instance normalization to obtain the predicted power load value for the next H steps. As a result of power load forecasting.

[0047] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0048] (1) The present invention designs a hyperedge feature extraction module, which extracts multi-scale hyperedge feature representation from the multi-scale time feature representation of multi-scale power load sequence based on the hyperedgeization mechanism, thereby enhancing the multi-scale semantic information in the semantic space of power load sequence, enabling the model to more comprehensively and deeply understand the changing patterns and internal connections of power load data at different time scales. Thus, when facing complex and ever-changing power load scenarios, the model can more accurately capture key features, providing richer and more accurate information support for subsequent prediction tasks, and effectively improving the accuracy and reliability of power load prediction.

[0049] (2) This invention designs a cross-modal alignment module that can align the multi-scale feature representations of natural language and power load sequences. In the process of alignment, a multi-scale hybrid prompting mechanism is introduced to generate diverse and targeted prompting information, which guides the large language model to explore the semantic connotation and potential rules behind the multi-scale patterns of power load sequences more deeply. This enhances the large language model's ability to understand the multi-scale patterns of power load sequences. As a result, when dealing with complex prediction scenarios such as few samples or even zero samples, the model can better transfer existing knowledge with its enhanced understanding ability, achieve more accurate and robust power load prediction, and significantly improve the versatility and practicality of power load prediction technology in different application scenarios. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart of an overall method for predicting power load based on multi-scale hypergraph neural network and large language model alignment provided by an embodiment of the present invention.

[0052] Figure 2 This is a general framework diagram of an electricity load prediction method based on multi-scale hypergraph neural network and large language model alignment provided by an embodiment of the present invention;

[0053] Figure 3 This is a detailed schematic diagram of the multi-scale feature extraction module provided in an embodiment of the present invention;

[0054] Figure 4 This is a detailed schematic diagram illustrating the hyperedgeization mechanism provided in an embodiment of the present invention;

[0055] Figure 5 This is a detailed schematic diagram illustrating the multi-scale hybrid prompting mechanism provided in this embodiment of the invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0057] The inventive concept of this invention is as follows: Addressing the technical problem in existing technologies where the multi-scale feature representations of natural language and power load sequences cannot be effectively aligned when using large language models for power load prediction, thus affecting prediction accuracy, this invention provides a power load prediction method based on the alignment of a multi-scale hypergraph neural network and a large language model. First, power load data is preprocessed and a training dataset is constructed. Second, a multi-scale feature extraction module is introduced, mapping the input power load sequence and word embeddings based on the pre-trained large language model into multi-scale temporal feature representations and multi-scale textual prototype representations, respectively. Then, a hyperedgeization mechanism is introduced to aggregate the multi-scale temporal feature representations into multi-scale hyperedge feature representations, thereby enhancing the semantic information in the multi-scale semantic space of the power load sequence. Next, a cross-modal alignment module is introduced to align the multi-scale feature representations of natural language and power load sequences, and a multi-scale hybrid prompting mechanism is introduced during the alignment process to provide multi-scale contextual information, enhancing the large language model's understanding of the multi-scale patterns of the power load sequence. Finally, the multi-scale hybrid prompting and the aligned multi-scale feature representations are fused and fed into the frozen main body of the pre-trained large language model to achieve prediction of the power load sequence.

[0058] Figure 1 This is a flowchart illustrating the overall process of a power load prediction method based on multi-scale hypergraph neural network and large language model alignment, as provided in this embodiment of the invention. Figure 2 This is a general framework diagram of an electricity load prediction method based on multi-scale hypergraph neural network and large language model alignment provided by an embodiment of the present invention. Figure 1 and Figure 2 As shown in the embodiment, an electricity load prediction method based on multi-scale hypergraph neural network and large language model alignment is provided, including the following steps:

[0059] The power load forecasting task is defined as follows: given the power load sequence observations of a certain site or region at T previous times. Predict the power load values ​​at the next H times

[0060] Step 1: Perform outlier and missing value removal on the power load, and divide the processed data into training datasets by using a sliding window in chronological order.

[0061] Outliers and missing values ​​in a given power load sequence are eliminated. Then, based on experience, a time window size T is manually set, and the normalized data is divided using a fixed-length sliding step to obtain the training dataset.

[0062] Step 2: Divide the training dataset into batches of a fixed batch size, with a total batch size of B.

[0063] The training dataset is divided into batches based on an empirically set batch size P, with a total batch size of B. The calculation formula is as follows:

[0064]

[0065] Where, N Samples This represents the total number of samples in the training dataset.

[0066] Step 3: Select a batch of training samples with index b sequentially from the training dataset, where b∈{1,2,…,B}. Repeat steps 4-11 for each training sample in this batch.

[0067] Step 4: Perform instance normalization on the training samples to make them have zero mean and unit variance in the feature dimension.

[0068] The power load sequences in the training samples are subjected to instance normalization to transform the data into a standard normal distribution with zero mean and unit variance in the feature dimensions. The calculation formula is as follows:

[0069]

[0070] Where, x′ t,d To represent the value of the d-th dimension (1≤d≤D) at time step t in the power load sequence after outlier elimination, μ d σ is the average value of the d-th dimension in this power load sequence. d Let be the variance of the d-th dimension in this power load sequence. This represents the power load value at time step t in the d-th dimension of the power load sequence instance after normalization.

[0071] Step 5: Normalize the power load sequence of the instances. The word embeddings U, based on a pre-trained large language model, are input into the multi-scale feature extraction module to extract multi-scale temporal feature representations {X}. 1 ,…,X s ,…,X S} and multi-scale text prototype representation {U 1 ,…,U s ,…,U S}. Where X s and U s These represent the temporal feature representation and text prototype representation at the s-th scale (1≤s≤S), respectively.

[0072] like Figure 3 As shown in (a), the normalized power load sequence for a given instance The multi-scale temporal feature extraction module generates S subsequences at different scales through different aggregation windows. Here, scale refers to time scale, such as hour, day, week, etc. The subsequence at the s-th scale is represented as... in Let D be the power load sequence at the s-th scale and time step t, where D is the feature dimension. Let N be the sequence length at the s-th scale. s-1 Let l be the length of the subsequence at the (s-1)th scale. s-1 This represents the size of the aggregation window at the (s-1)th scale. The specific calculation formula for the aggregation process is as follows:

[0073]

[0074] Where Agg(·) is an aggregation function, such as convolution or pooling, θ s-1 Let be the learnable parameters of the aggregation function at the (s-1)th scale. This is the input sample sequence after instance normalization.

[0075] like Figure 3 As shown in (b), given word embeddings based on a pre-trained large language model Where V is the vocabulary size and P is the hidden layer dimension of the pre-trained large language model. First, word embeddings U are transformed into initial text prototype representations through linear mapping. Where V′ << V. Then, the multi-scale text prototype extraction module obtains the multi-scale text prototype representation through linear mapping. The specific calculation formula for the mapping process is as follows:

[0076]

[0077] Where Linear(·) is a linear mapping function, U s For the text prototype representation at the s-th scale, U s-1 Let λ be the text prototype representation at the (s-1)th scale, where scale refers to the text scale corresponding to the time scale, such as word, sentence, paragraph, etc. s-1 V represents the learnable parameters of the linear mapping function at the (s-1)th scale. s Let be the number of text prototypes represented at the s-th scale.

[0078] Finally, the time feature representations {X} at S scales generated by the multi-scale feature extraction module are obtained. 1 ,…,X s ,…,X S} and text prototype representation {U 1 ,…,U s ,…,U S}, where X s and U sThese represent the temporal feature representation and text prototype representation at the s-th scale (1≤s≤S), respectively.

[0079] Step 6: In the hyperedge feature extraction module, the multi-scale temporal feature representation is aggregated into a multi-scale hyperedge feature representation through the hyperedgeization mechanism.

[0080] like Figure 4 As shown, the multi-scale temporal features are first represented as {X}. 1 ,…,X s ,…,X S Treat it as a node and initialize two parameters: hyperedge embedding. and node embedding Where M s This is a hyperparameter representing the number of hyperedges at the s-th scale. Then, the vertex-edge association matrix H at the s-th scale is calculated using similarity. s The calculation formula is as follows:

[0081]

[0082] in, and The parameters are learnable. The tanh(·) activation function is used to perform nonlinear transformations, the ReLU(·) activation function is used to eliminate weak connections, and Linear(·) is a linear mapping function. The superscript T denotes the transpose operation. To enhance the robustness of the model and reduce noise interference in subsequent calculations, a sparsity strategy is designed, the calculation formula of which is as follows:

[0083]

[0084] Where η∈[0,M s [] is the threshold of the TopK(·) function, representing the maximum number of neighbor hyperedges connected to the node, where n is the node index and m is the hyperedge index. To connect all the hyperedges to the nth node, Let H be the sparsed vertex-edge affinity matrix at the s-th scale. After the sparsification strategy, we finally obtain vertex-edge affinity matrices at S scales {H}. 1 ,…,H s ,…,H S}

[0085] The i-th hyperedge at scale s hyperedge features Based on the vertex-edge association matrix H s The calculation formula is as follows, obtained through information aggregation:

[0086]

[0087] Where Avg(·) is the averaging operation. H is the point-edge incidence matrix at scale s. s The image shown is obtained through the hyperedge. Connected neighbor nodes, For the j-th node at scale s The temporal feature representation is that the set of all hyperedge features at the fusion scale s is ε. s Finally, we obtain hyperedge feature representations at S scales {ε 1 ,…,ε s ,…,ε S}

[0088] Step 7, represent the multi-scale hyperedge features {ε 1 ,…,ε s ,…,ε S and multi-scale text prototype representation {U 1 ,…,U s ,…,U S The data is fed into the cross-modal alignment module to obtain the aligned multi-scale feature representation {Z}. 1 ,…,Z s ,…,Z S}. Among them, Z s This represents the feature representation of the s-th scale after alignment (1≤s≤S).

[0089] Hyperedge feature representation ε at a given scale s s and text prototype representation U s First, map it to the corresponding query. key Sum in It is the index of the number of heads in the attention process. This represents the total number of heads. and Let be the learnable mapping matrix at scale s. Then, the hyperedge feature representation and text prototype representation are aligned by calculating cross-attention, as shown in the following formula:

[0090]

[0091] Where Attn(·) is the cross-attention mechanism, softmax(·) is the activation function, and the superscript T denotes the transpose operation. For the scale s, the first The outputs of each attention head can be aggregated to obtain the aligned feature representation Z of the multi-head attention outputs at scale s. s Finally, the aligned feature representations at different scales are represented as {Z}.1 ,…,Z s ,…,Z S}

[0092] Step 8: Implement a multi-scale hybrid cueing mechanism by constructing learnable cues. Data-related tips And ability enhancement prompts Generate multi-scale blended cues.

[0093] like Figure 5 As shown, the multi-scale hybrid cueing mechanism uses different types of cues (i.e., learning cues) Data-related tips And ability enhancement prompts This enriches the contextual information of the input, thereby enhancing the ability of the large language model to understand the multi-scale patterns of power load sequences.

[0094] Learnable Hints: Enhancing the understanding of multi-scale temporal patterns in power load sequences by constructing learnable hints. Multi-scale learnable hints can be represented as... in, It is a learnable cue at the s-th scale, which is obtained from a learnable initialization embedding, L s It is the length of the learnable cue at scale s. Learning is achieved through the supervised loss between the model's output and the true label.

[0095] Data-related prompts: such as Figure 5 As shown in (a), data-related hints are constructed by introducing three components. Specifically, the data description hints are π, the task description hints are τ, and the data statistics hints are μ. The data description hints provide the large language model with basic background information about the input power load sequence; the task description hints guide the large language model to understand and perform specific power load prediction-related tasks; and the data statistics hints provide statistical features of the power load sequence, including the input power load sequence and subsequences at different scales. Finally, the data-related hints... The calculation formula is as follows:

[0096]

[0097] Among them, tokenizer(·) is the tokenizer.

[0098] Ability enhancement prompts: such as Figure 5 As shown in (b), three component building capabilities are introduced to enhance hints. That is, logical thinking-based prompts σ, and emotional manipulation-based prompts. And reasoning-related hints μ. Logical thinking hints guide the large language model to solve problems step by step, thereby improving its multi-step reasoning ability in power load sequences; emotion manipulation hints simulate the influence of emotions on human decision-making, making the model more focused on the current task through "emotional blackmail"; while reasoning-related hints provide usable methods to guide the large language model in predicting power load. Finally, capability enhancement hints. The calculation formula is as follows:

[0099]

[0100] Among them, tokenizer(·) is the tokenizer.

[0101] Step 9: Concatenate the multi-scale hybrid cue and the aligned multi-scale feature representation, and feed the concatenated multi-scale feature representation into the frozen pre-trained large language model body to generate the output of the pre-trained large language model.

[0102] After obtaining multi-scale mixed cues, the learnable cues P at different scales are first... s Aligned multi-scale feature representation Z s Perform the stitching. Then connect it with data-related prompts. And ability enhancement tips The concatenation is performed, and the concatenated result is fed into the frozen pre-trained large language model to obtain the output of the pre-trained large language model. The calculation formula is as follows:

[0103]

[0104] Among them, [·,·] represents the splicing operation. This is the output of the pre-trained large language model LLMs(·).

[0105] Step 10: Input the output of the pre-trained large language model into the linear layer for linear transformation, and then perform inverse instance normalization on the transformed result to obtain the predicted power load value for the next H steps.

[0106] After obtaining the output of the pre-trained large language model, it is input into a linear layer for linear transformation, and the result of the linear transformation is subjected to inverse instance normalization to obtain the final H-step power load prediction value. The formula for calculating reverse instance normalization is as follows:

[0107]

[0108] in, Let o′ be the predicted power load value at time step t in the d-th dimension after reverse instance normalization of power load data. t,d This represents the result of linearly transforming the output of a pre-trained large language model, with the d-th dimension having a time step of t.

[0109] Step 11, Calculate the prediction loss That is, the predicted value of the training sample. and corresponding real tags The error between them.

[0110] In this invention, the squared error is used as the prediction loss to calculate the predicted value of the training sample. and corresponding real tags The error between them is calculated using the following formula:

[0111]

[0112] Step 12, based on the loss of all samples in the batch The learnable network parameters, node embeddings, and scale embeddings in the entire power load prediction model are adjusted.

[0113] Loss of all samples in a batch The calculation formula is as follows:

[0114]

[0115] in, Let B be the loss for the b-th sample in a batch, and B be the number of samples in each batch. Based on the loss... The learnable network parameters, node embeddings, and scale embeddings (all learnable parameters denoted as θ) in the entire power load prediction model are adjusted, and the updated formula is as follows:

[0116]

[0117] Where γ is the learning rate.

[0118] Step 13: Repeat steps 3-12 until all batches of the training dataset have been used to train the power load prediction model.

[0119] Step 14: Repeat steps 3-13 until the specified number of iterations is reached.

[0120] Step 15: Input the power load sequence to be predicted into the trained power load prediction model to obtain the prediction result.

[0121] The power load sequence to be predicted is preprocessed and then input into a pre-trained power load prediction model. The power load prediction model predicts the power load values ​​at the next H times based on the power load data within the historical window, and uses the prediction results for subsequent analysis and decision-making.

[0122] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A power load forecasting method based on multi-scale hypergraph neural network and large language model alignment, characterized in that, Includes the following steps: The power load sequence is preprocessed and training samples are constructed. In the multi-scale feature extraction module, the training samples and word embeddings based on the pre-trained large language model are mapped to multi-scale temporal feature representations and multi-scale text prototype representations, respectively. This includes: performing instance normalization on the training samples to obtain the instance-normalized power load sequence. , The time window size is based on the first... The subsequences at the nth scale are generated by using different aggregation windows. The subsequence at the nth scale, the nth Subsequences of scales are represented as ,in For the first Each scale time step is The power load sequence is represented as follows: For feature dimension, For the first Sequence length at each scale, For the first Subsequence length at each scale For the first The aggregation window size is determined by several scales, and the calculation formula for the aggregation process is as follows: ， in, For aggregate functions, For the aggregation function in the th Learnable parameters at each scale, These are the input training samples after instance normalization. Word embeddings based on pre-trained large language models ,in For vocabulary size, To increase the hidden layer dimension of the pre-trained large language model, word embeddings are first performed using linear mapping. Transformed into the initial text prototype representation ,in Then, a multi-scale text prototype representation is obtained through linear mapping. The calculation formula for the linear mapping process is as follows: ， in, It is a linear mapping function. For the first Text prototype representation at various scales For the first Text prototype representation at various scales For the linear mapping function in the th... Learnable parameters at each scale, For the first The number of text prototypes represented by each scale; Finally, the multi-scale feature extraction module generates the result. Time characteristics at each scale and text prototype representation ,in and are respectively the first ( Temporal feature representation and text prototype representation at multiple scales; In the hyperedge feature extraction module, the hyperedgeization mechanism based on the multi-scale hypergraph neural network aggregates the multi-scale temporal feature representation into a multi-scale hyperedge feature representation; In the cross-modal alignment module, the aligned multi-scale feature representation is obtained by aligning the multi-scale text prototype representation and the multi-scale hyperedge feature representation based on cross-attention alignment, including: the mesoscale based on the multi-scale hyperedge feature representation. Hyperedge feature representation and multi-scale text prototype representation of mesoscale The following text prototype representation First, map it to the corresponding query. ,key Sum ,in It is the index of the number of heads in the attention. This represents the total number of heads. , and For scale Learnable mapping matrix For feature dimension, To determine the hidden layer dimension of a pre-trained large language model, Then, cross-attention is computed to align the hyperedge feature representation and the text prototype representation; and multi-scale hybrid cues are constructed. In the large model prediction module, the multi-scale hybrid prompts and aligned multi-scale feature representations are concatenated and then input into the pre-trained large language model for power load prediction. The power load prediction model, which includes a multi-scale feature extraction module, a hyperedge feature extraction module, a cross-modal alignment module and a large model prediction module, is trained based on the prediction loss. The new power load sequence is input into the trained power load prediction model to obtain the prediction result.

2. The power load prediction method based on multi-scale hypergraph neural network and large language model alignment according to claim 1, characterized in that, In the hyperedge feature extraction module, the hyperedgeization mechanism based on multi-scale hypergraph neural networks aggregates multi-scale temporal feature representations into multi-scale hyperedge feature representations, including: Representing multi-scale temporal features Treat it as a node and initialize two parameters: hyperedge embedding. and node embedding ,in, For the first ( The time features are represented at several scales. For the first The number of hyperedges at each scale As the feature dimension, the first value is obtained through similarity calculation. Point-edge correlation matrix at various scales The calculation formula is as follows: ， ， ， in, and For learnable parameters, Activation functions are used to perform non-linear transformations. Activation functions are used to eliminate weak connections. For linear mapping functions, superscript For the transpose operation, the final result is... Point-edge correlation matrix at various scales ; scale Next super edge hyperedge features Based on the point-edge association matrix The calculation formula is as follows, obtained through information aggregation: ， in, To perform the averaging operation, For scale Lower edge correlation matrix The image shown is obtained through the hyperedge. Connected neighbor nodes, For scale Next Nodes Temporal characteristics, scale The set of all hyperedge features is ,final The hyperedge features at each scale are represented as follows: .

3. The power load prediction method based on multi-scale hypergraph neural network and large language model alignment according to claim 2, characterized in that, Calculate the first using a sparsity strategy The formula for calculating the point-edge incidence matrix at each scale is as follows: ， in, for The threshold of the function represents the maximum number of neighboring superedges connected to the node. For node indexing, For superedge index, To connect the first All superedges of each node, For the first A sparse vertex-edge correlation matrix of several scales, based on Get the final Point-edge correlation matrix at various scales .

4. The power load forecasting method based on multi-scale hypergraph neural network and large language model alignment according to claim 1, characterized in that, The method of aligning hyperedge feature representation and text prototype representation by calculating cross attention is as follows: ， in, For cross-attention mechanism, For activation functions, superscript For transpose operation, For scale Next The scale is obtained by aggregating the outputs of each attention head. Aligned feature representation of multi-head attention output Finally, the aligned Multi-scale feature representation at multiple scales .

5. The power load prediction method based on multi-scale hypergraph neural network and large language model alignment according to claim 1, characterized in that, Multiscale blending cues include: Learnable tips : Represented as ,in, For the first Learnable cues at various scales are obtained from learnable initialization embeddings. , For scale quantity For scale The length of the learnable hints below, For feature dimensions; Data-related tips This includes data description class hints that provide basic background information about the input power load sequence for pre-trained large language models. Task description hints used to guide pre-trained large language models to understand and perform specific power load prediction-related tasks. And data statistics tips including statistical characteristics of the power load sequence, including the input power load sequence and subsequences at different scales. ; Ability Enhancement Tips This includes logical thinking prompts that guide the pre-trained large language model to solve problems step by step. Emotional manipulation prompts that simulate the influence of emotions on human decision-making. And class hints related to inference methods for pre-trained large language models to predict power load. .

6. The power load prediction method based on multi-scale hypergraph neural network and large language model alignment according to claim 5, characterized in that, In the large model prediction module, the step of concatenating multi-scale hybrid prompts and aligned multi-scale feature representations and inputting them into the large language model for power load prediction includes: After receiving multi-scale mixing cues, first... Learnable hints at each scale Aligned multi-scale feature representation The data is concatenated, then further concatenated with data-related cues and capability-enhancing cues. The concatenated result is then input into a pre-trained large language model to obtain the output. The calculation formula is as follows: ， in, For splicing operations, For the first Aligned feature representations at various scales For pre-training large language models The output result.

7. The power load forecasting method based on multi-scale hypergraph neural network and large language model alignment according to claim 1, characterized in that, During training, keep the main parameters of the pre-trained large language model frozen.

8. The power load prediction method based on multi-scale hypergraph neural network and large language model alignment according to claim 1, characterized in that, The output of the large language model is input into a linear layer for linear transformation, and the transformed result is then subjected to inverse instance normalization to obtain the future... Step power load forecast As a result of power load forecasting.

Images