A power price sequence reconstruction and prediction method based on a large language model drive

Abstract

The application belongs to the technical field of electricity price prediction, and discloses a method for electricity price sequence reconstruction and prediction based on a large language model, which comprises the following steps: obtaining and preprocessing the electricity price time sequence of a target electricity market, constructing a missing identification matrix and a missing pattern coding matrix; extracting dynamic prior features; constructing joint input features, extracting forward time features and backward time features through a pre-trained large language model, performing confidence assessment, generating bidirectional fusion features, and obtaining a complete electricity price time sequence after missing value filling; dividing the complete electricity price time sequence into multiple patches according to a preset time window and performing patch reprogramming; constructing a prompt prefix to form a unified sequence representation; inputting the unified sequence representation into the pre-trained large language model to perform forward reasoning, linear projection and inverse normalization processing, and obtaining electricity price prediction results at multiple future time points. The application can effectively fill in missing values and accurately predict electricity prices.

Description

Technical Field

[0001] This invention belongs to the field of electricity price forecasting technology, specifically relating to a method for reconstructing and forecasting electricity price sequences based on a large language model. Background Technology

[0002] In the development of the electricity market and the adjustment of the energy structure, electricity prices, as a core market signal, directly affect the economic benefits of market participants and are crucial for guiding the rational allocation of electricity resources and ensuring the stable operation of the system. Affected by multiple factors such as fuel price fluctuations, unstable renewable energy output, and changes in load demand, electricity price time series exhibit highly nonlinear and volatile characteristics. Building accurate and reliable short-term electricity price forecasting models has significant theoretical and practical value for improving the scientific nature of market decision-making and mitigating operational risks. Current electricity price forecasting methods are mainly divided into three categories: traditional time series methods, machine learning, and deep learning. Traditional methods struggle to handle the multidimensional influence mechanisms of electricity prices; machine learning methods lack the ability to model complex spatiotemporal dynamics; and while deep learning models improve prediction accuracy, they suffer from complex architectures, high computational costs, and difficulties in training and adjustment. Furthermore, all methods are highly dependent on data integrity.

[0003] In the actual power data collection and transmission process, electricity price data is prone to missing due to various factors such as equipment failure, communication interruption, and human error. This severely damages the integrity of electricity price time-series data, thereby significantly reducing the accuracy of prediction models. Therefore, reasonable preprocessing of missing electricity price data is an important prerequisite for building a reliable electricity price prediction model. Existing methods for data imputation mainly include three categories: statistical learning, machine learning, and deep learning. Although statistical learning is simple to operate and widely applicable, it is prone to changing the original data structure and losing discrete features. Machine learning can handle complex feature data, but its performance will significantly decrease in high missing rate scenarios, and its performance is greatly affected by the quality of input features. Deep learning can generate high-precision imputation values, but it is not adaptable to high missing rate scenarios, and it is difficult to balance efficiency, interpretability, and stability. It also lacks a robust fusion framework adapted to different application scenarios.

[0004] Various electricity price forecasting and data filling methods have their own technical limitations. Moreover, in existing technologies, missing value processing and electricity price forecasting are mostly independent processes, without forming a collaborative optimization technical framework. In real-world scenarios with missing data, problems such as feature transfer discontinuity and poor model adaptability are likely to occur, further exacerbating the industry pain points of insufficient forecast accuracy and weak robustness, making it difficult to meet the actual needs of the electricity market for electricity price forecasting.

[0005] Large language models have achieved groundbreaking developments in the field of natural language processing thanks to their superior pattern recognition, long-range dependency capture, and reasoning capabilities. Existing research has also confirmed their significant potential in time series data imputation and prediction, such as imputing missing values ​​in time series data by fine-tuning pre-trained models and improving the accuracy of intraday short-term electricity price forecasts by combining market-related information analysis. However, there is currently no innovative method based on large language models that can simultaneously achieve effective imputation of missing values ​​and high-precision electricity price prediction for incomplete electricity price data. This makes it difficult to effectively overcome the various limitations of existing technologies in scenarios with missing data. There is an urgent need to build a unified technical framework that combines the capabilities of missing value imputation and electricity price prediction. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method for reconstructing and predicting electricity price sequences based on a large language model, which can effectively fill missing values ​​and predict electricity prices with high accuracy.

[0007] To achieve the above objectives, this invention provides a method for reconstructing and predicting electricity price sequences based on a large language model, comprising the following steps: S1. Obtain and preprocess the electricity price time series of the target electricity market, and construct a missing value identifier matrix of the same length as the electricity price time series to identify the location of observations and missing values. Determine the missing value type for the missing value location and construct a missing value pattern encoding matrix. S2. Extract dynamic prior features from the preprocessed electricity price time series, including local fluctuation intensity, time interval between adjacent valid observation points, and length of continuous missing intervals; S3. The preprocessed electricity price time series is constructed into forward input branches and backward input branches according to the forward and reverse time order, respectively. For each time step, the normalized electricity value, missing state, missing type and dynamic prior features are extracted and combined with the forward input branch and the backward input branch to form joint input features. These are mapped to the hidden dimension space of the pre-trained large language model to obtain the missing time embedding. After superimposing the multi-scale time position encoding, it is input into the pre-trained large language model to extract the forward time representation and the backward time representation. S4. Confidence assessment is performed on the forward time representation and the backward time representation. The fusion weight is dynamically calculated based on the confidence assessment results. The forward time representation and the backward time representation are weighted and fused to generate a bidirectional fused representation. The missing positions are numerically estimated based on the bidirectional fused representation to obtain the complete electricity price time series after the missing values ​​are filled. S5. Divide the complete electricity price time series into multiple patches according to a preset time window, map each patch to a unified hidden dimension space, and then perform patch reprogramming operation to make the patch embedding aligned to the semantic space of the pre-trained large language model. S6. Construct a hint prefix and concatenate the hint prefix with the reprogrammed Patch to form a unified sequence representation; S7. The unified sequence representation is fed into the pre-trained large language model for forward inference. The output result is linearly projected and inversely normalized to obtain the electricity price prediction results for multiple future time points.

[0008] As a preferred embodiment of the present invention, in S1, the preprocessing specifically includes: sorting the electricity price time series according to timestamps to construct a unified time index set; merging duplicate timestamps according to preset rules; retaining timestamp gaps and marking the corresponding positions as missing value positions when there are timestamp gaps. Perform outlier detection and processing, identify outliers containing spike values, outliers, or anomalous jump points, and remove, correct, or mark outliers; the missing positions generated after outlier processing are also marked as missing value positions; Min-Max normalization is performed to map non-missing samples in the electricity price time series to the [0,1] interval.

[0009] As a preferred embodiment of the present invention, in S1, a mask matrix of the same length as the electricity price time series is constructed as a missing identifier matrix. When the electricity price corresponding to the time point is an observed valid value, the mask is set to 1; when the electricity price corresponding to the time point is a missing value, the mask is set to 0 and the missing identifier is set to NaN. The missing locations are identified by type. Based on the source of the missing points, the temporal distribution pattern, the length of consecutive missing points, and the characteristics of neighboring observations, the missing value locations are divided into random missing, continuous interrupted missing, abnormally removed missing, and artificially simulated missing. Missing pattern coding information is generated through one-hot coding to form a multi-state missing pattern coding matrix.

[0010] As a preferred embodiment of the present invention, the process of obtaining the forward time representation and the backward time representation in S3 is as follows: Multi-scale temporal location coding includes at least continuous time step location coding, intra-day periodic location coding, and intra-weekly periodic location coding, which are used to jointly describe temporal sequence features and periodic structure features. The forward input branch and its corresponding normalized electrical value, missing state, missing type and dynamic prior features constitute the forward joint feature sequence, and the reverse input branch and its corresponding normalized electrical value, missing state, missing type and dynamic prior features constitute the reverse joint feature sequence. The forward and reverse joint feature sequences are transformed into dimensions and mapped to the same dimension space as the hidden layer of the pre-trained large language model to generate forward missing time embedding sequences and reverse missing time embedding sequences. Multi-scale temporal position encoding is superimposed on forward and reverse missing-aware temporal embedding sequences to generate forward and reverse input representations, respectively. The forward input representation is fed into the forward branch of the pre-trained large language model to learn the trend continuation, periodic evolution and local stationarity features of the electricity price time series, and the forward time representation is extracted. The backward input representation is fed into the backward branch of the pre-trained large language model to learn the missing interval recovery, mutation boundary recognition and future context compensation features, and the backward time representation is extracted.

[0011] As a preferred embodiment of the present invention, in S3, a direction-aware gating module is introduced at the input ends of the forward branch and the reverse branch respectively, and the passing ratio of historical context information and future context information is dynamically adjusted according to the missing rate, the length of the continuous missing interval, the intensity of local fluctuations and the distribution density of effective observation points in the input window. For pre-trained large language models, freeze their multi-head self-attention layers and feedforward network layers, while opening up the positional encoding parameters, layer normalization parameters, and low-rank adapter parameters for updating.

[0012] As a preferred embodiment of the present invention, the process of obtaining the complete electricity price time series after missing value filling in S4 is as follows: S4.1 Construct confidence evaluation branches for forward and backward time representations respectively. Estimate the confidence of forward and backward time representations based on local context integrity, neighborhood observation density, representation stability, context consistency, and the relative position of the current time step in the continuous missing interval. Forward confidence at time step t and backward confidence They are defined as follows: ; ; In the formula, This represents the forward time representation of time step t; represents the backward time representation at time step t; Sim represents the similarity function between the forward and backward time representations. This represents the distance from time step t to the previous valid observation point; This represents the distance from time step t to the next valid observation point; This indicates the local observation density in the forward direction; This indicates the local observation density in the backward direction; This represents the relative position encoding of time step t within a consecutive missing interval; Indicates the length of the consecutive missing interval at time step t; This represents the intensity of local fluctuations within the neighborhood of time step t. , , , , , For preset weighting coefficients, This is the Sigmoid activation function, used to constrain the confidence level to the (0,1) interval; Defined as: ; In the formula, To prevent positive numbers with a denominator of zero; S4.2, based on and The forward fusion weights and backward fusion weights are calculated using normalization: ; ; In the formula, Indicates the forward fusion weights at time step t; This represents the backward fusion weights at time step t; S4.3, to and Perform weighted fusion to generate a bidirectional fusion representation: ; In the formula, Represents the bidirectional fusion characterization of time step t; S4.4, to Perform a linear transformation, to Map back to a dimension space consistent with the original electricity price time series: ; In the formula, To reconstruct the mapping matrix, a learnable linear transformation weight matrix is ​​used to... Linear mapping back to the 1-dimensional electricity price numerical space; For bias terms; The reconstructed electrical value for time step t; S4.5 For missing value locations, fill the corresponding gaps with the reconstructed electricity value after inverse normalization; for valid observation locations, retain their original electricity value; and concatenate the electricity values ​​of all time steps in the original time order to obtain the complete electricity price time series after missing value filling.

[0013] In a preferred embodiment of the present invention, the total number P of the divided patches in S5 is: ; In the formula, T is the number of time steps; The length of each patch; S represents the horizontal sliding step size; The segmented patches are processed through a linear embedding layer. Mapped to a unified hidden input dimension The above yields the embedded representation. Where p is the index of the Patch. It represents the set of real numbers.

[0014] As a preferred embodiment of the present invention, in S5, the Patch reprogramming operation specifically involves using the word embedding matrix of a pre-trained large language model. Reprogramming as the basis of the target semantic space Where V is the vocabulary size and D is the hidden dimension of the pre-trained large language model; A set of text prototypes semantically related to electricity price time series is selected through a learnable fully connected layer, resulting in a selected text prototype matrix. ,in This is the number of text prototypes retained after filtering. ; By using a multi-head cross-attention mechanism, each patch is embedded into a text prototype matrix for alignment and mapping, resulting in a reprogrammed patch representation.

[0015] As a preferred embodiment of the present invention, in S6, the prompt prefix specifically includes: Basic data information, including data source, field background, and sampling frequency; The mission instructions specify the length of the historical observation sequence and the future step size to be predicted. Electricity price time series statistical features are extracted to transform the original electricity price time series into a semantic description with clear statistical meaning, including the maximum, minimum, median and overall trend of the data.

[0016] As a preferred embodiment of the present invention, in S7, the unified sequence representation is fed into the frozen pre-trained large language model for forward inference, the prefix part corresponding to the prompt prefix is ​​discarded, the output representation related to the patch is retained, the output representation is flattened, and mapped to the target prediction step size dimension through the output projection layer to generate normalized electricity price prediction values ​​for multiple future time points. The normalized electricity price prediction value is then subjected to inverse normalization processing to obtain the future electricity price prediction result for the actual electricity price value range.

[0017] The beneficial effects of this invention are: This invention constructs an integrated framework for "missing value imputation-electricity price prediction" driven by a large language model. It is adaptable to various incomplete scenarios, such as random and continuous missing data in electricity prices, achieving efficient imputation of missing values ​​and high-precision prediction of short-term electricity prices. The integrated framework incorporates a bidirectional time representation learning mechanism, fully leveraging the historical contextual dependencies and future time step correlations of electricity price time series data. This allows the model to maintain excellent imputation accuracy and robustness even in scenarios with high missing data rates, ensuring the reliability of subsequent prediction tasks from the data source and solving the core problem of significant performance degradation in existing technologies with incomplete data.

[0018] This invention utilizes a patch partitioning and reprogramming mechanism to accurately map electricity price time series data to the semantic space of a large language model, achieving deep alignment between numerical time series modalities and natural language semantic modalities. Simultaneously, by incorporating domain-specific prompt prefixes, it effectively injects electricity market information, electricity price prediction task instructions, and sequence statistical features into the model, fully activating the deep sequence reasoning capabilities of the pre-trained large language model. This design eliminates the need for complex network structures and tedious parameter tuning, significantly improving electricity price prediction accuracy while drastically reducing model training costs and deployment complexity, making it more suitable for practical engineering applications.

[0019] This invention has been validated across multiple scenarios using publicly available electricity market data. In tasks such as missing value imputation, complete data prediction, and joint prediction after missing value imputation, the core evaluation metrics significantly outperform existing comparative models. Even under high missing value rates, the model maintains high fit and low error. In complete data prediction, the error index is significantly lower than mainstream benchmark models, and in the joint imputation-prediction task, the prediction performance only shows slight fluctuations with increasing missing value rates. Furthermore, the model performs excellently in transfer validation across different electricity markets, demonstrating strong engineering applicability and generalization ability, providing reliable electricity price data support and predictive references for decision-making and risk management by electricity market participants. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the principle of this invention; Figure 2 This is a diagram illustrating patch reprogramming; Figure 3 This is an architecture diagram of the overall prediction process of this invention; Figure 4 This is a comparison chart of model prediction indicators for Region 1 during the verification process of this invention; Figure 5 This is a comparison chart of model prediction indicators for Region 2 during the verification process of this invention. Detailed Implementation

[0021] The embodiments of the present invention will be further described below with reference to the accompanying drawings: Example 1: As Figure 1 As shown, a method for reconstructing and predicting electricity price sequences based on a large language model includes the following steps: S1. Obtain and preprocess the electricity price time series of the target electricity market, and construct a missing value identifier matrix of the same length as the electricity price time series to identify the location of observations and missing values. Determine the missing value type for the missing value location and construct a missing value pattern encoding matrix. S2. Extract dynamic prior features from the preprocessed electricity price time series, including local fluctuation intensity, time interval between adjacent valid observation points, and length of continuous missing intervals; S3. The preprocessed electricity price time series is constructed into forward input branches and backward input branches according to the forward and reverse time order, respectively. For each time step, the normalized electricity value, missing state, missing type and dynamic prior features are extracted and combined with the forward input branch and the backward input branch to form joint input features. These are mapped to the hidden dimension space of the pre-trained large language model to obtain the missing time embedding. After superimposing the multi-scale time position encoding, it is input into the pre-trained large language model to extract the forward time representation and the backward time representation. S4. Confidence assessment is performed on the forward time representation and the backward time representation. The fusion weight is dynamically calculated based on the confidence assessment results. The forward time representation and the backward time representation are weighted and fused to generate a bidirectional fused representation. The missing positions are numerically estimated based on the bidirectional fused representation to obtain the complete electricity price time series after the missing values ​​are filled. S5. Divide the complete electricity price time series into multiple patches according to a preset time window, map each patch to a unified hidden dimension space, and then perform patch reprogramming operation to make the patch embedding aligned to the semantic space of the pre-trained large language model. S6. Construct a hint prefix and concatenate the hint prefix with the reprogrammed Patch to form a unified sequence representation; S7. The unified sequence representation is fed into the pre-trained large language model for forward inference. The output result is linearly projected and inversely normalized to obtain the electricity price prediction results for multiple future time points.

[0022] In this embodiment, the Large Language Model (LLM) is preferably LLaMA.

[0023] In S1, the preprocessing specifically involves: sorting the electricity price time series according to timestamps and constructing a unified time index set; merging duplicate timestamps according to preset rules; and retaining timestamp gaps and marking the corresponding positions as missing value positions when there are timestamp gaps. Perform outlier detection and processing to identify outliers that deviate significantly from the normal distribution, including peak values, outliers, or anomalous jump points. Remove, correct, or mark outliers. The missing positions generated after outlier processing are also marked as missing value positions and participate in the subsequent reconstruction process. Min-Max normalization is performed to map non-missing samples in the electricity price time series to the [0,1] interval.

[0024] Construct a mask matrix of the same length as the electricity price time series as a missing value matrix. When the electricity price corresponding to a time point is an observed valid value, set the mask to 1; when the electricity price corresponding to a time point is a missing value, set the mask to 0 and set the missing value to NaN. The missing locations are classified into different types based on their origin, temporal distribution, length of consecutive missing values, and neighborhood observation characteristics. These types include random missing values, continuous interrupted missing values, abnormally removed missing values, and artificially simulated missing values. Missing pattern coding information is generated through one-hot coding to form a multi-state missing pattern coding matrix, thereby enhancing the model's ability to identify different sources of missing values.

[0025] In S3, the preprocessed electricity price time series is constructed into a forward input sequence according to the original time order to learn the evolutionary impact of historical observations on the current moment; at the same time, the electricity price time series is constructed into a reverse input sequence according to the reverse time order to learn the constraint compensation information of future observations on the current missing position. The process of obtaining the forward and backward time representations is as follows: Multi-scale temporal location coding includes at least continuous time step location coding, intra-day periodic location coding, and intra-weekly periodic location coding, which are used to jointly describe temporal sequence features and periodic structure features. The positive input representation formed by superposition is fed into the positive branch of the pre-trained large language model to learn the features of trend continuation, periodic evolution and local stationarity; the negative input representation formed by superposition is fed into the negative branch of the pre-trained large language model to learn the features of missing interval recovery, mutation boundary recognition and future context compensation, thereby constructing an asymmetric but complementary bidirectional temporal representation learning mechanism. The forward input branch and its corresponding normalized electrical value, missing state, missing type and dynamic prior features constitute the forward joint feature sequence, and the reverse input branch and its corresponding normalized electrical value, missing state, missing type and dynamic prior features constitute the reverse joint feature sequence. The forward and reverse joint feature sequences are transformed into dimensions and mapped to the same dimension space as the hidden layer of the pre-trained large language model to generate forward missing time embedding sequences and reverse missing time embedding sequences. Multi-scale temporal position encoding is superimposed on forward and reverse missing-aware temporal embedding sequences to generate forward and reverse input representations, respectively. The forward input representation is fed into the forward branch of the pre-trained large language model to learn the trend continuation, periodic evolution and local stationarity features of the electricity price time series, and the forward time representation is extracted. The backward input representation is fed into the backward branch of the pre-trained large language model to learn the missing interval recovery, mutation boundary recognition and future context compensation features, and the backward time representation is extracted.

[0026] Direction-aware gating modules are introduced at the input ends of the forward and reverse branches respectively. Based on the missing rate, the length of the continuous missing interval, the intensity of local fluctuations, and the distribution density of effective observation points within the input window, the ratio of historical context information to future context information is dynamically adjusted. When the missing rate is high or the continuous missing interval is long, the information utilization weight of the reverse branch is increased. When the historical observations are continuous and the sequence fluctuations are stable, the information utilization weight of the forward branch is increased. Regarding the parameter update strategy, a combination of partial freezing and lightweight adaptation is adopted for the pre-trained large language model. The multi-head self-attention layer and feedforward network layer are frozen, while the positional encoding parameters, layer normalization parameters and low-rank adapter parameters are opened for updating.

[0027] The update weights of the lightweight adaptation can be adaptively adjusted according to the degree of missing information and the complexity of fluctuation in the input window, so as to balance the ability to retain pre-trained knowledge and the ability to adapt to tasks.

[0028] For any time step, calculate the difference features between the forward time representation and the backward time representation. The difference features include at least the difference in representation amplitude, the difference in direction, the local similarity or the equivalent difference measure, which are used to characterize the consistency of the bidirectional context with the recovery result of the current time step. The process of obtaining the complete electricity price time series after missing value filling is as follows: S4.1 Construct confidence evaluation branches for forward and backward time representations respectively. Estimate the confidence of forward and backward time representations based on local context integrity, neighborhood observation density, representation stability, context consistency, and the relative position of the current time step in the continuous missing interval. Forward confidence at time step t and backward confidence They are defined as follows: ; ; In the formula, This represents the forward time representation of time step t; represents the backward time representation at time step t; Sim represents the similarity function between the forward and backward time representations, preferably cosine similarity. This represents the distance from time step t to the previous valid observation point; This represents the distance from time step t to the next valid observation point; This indicates the local observation density in the forward direction; This indicates the local observation density in the backward direction; This represents the relative position encoding of time step t within a consecutive missing interval; Indicates the length of the consecutive missing interval at time step t; This represents the intensity of local fluctuations within the neighborhood of time step t. , , , , , These are learnable parameters or preset weighting coefficients. This is the Sigmoid activation function, used to constrain the confidence level to the (0,1) interval; Defined as: ; In the formula, To prevent extremely small positive numbers with a denominator of zero; When the missing point is closer to the beginning of a consecutive missing interval, the position bias term ( The position bias term helps to improve forward confidence; when the missing point is closer to the end of the continuous missing interval, the position bias term helps to improve backward confidence, thereby enabling differentiated characterization and evaluation of different positions within the continuous missing interval. S4.2, based on and The forward fusion weights and backward fusion weights are calculated using normalization: ; ; In the formula, Indicates the forward fusion weights at time step t; The weights represent the backward fusion weights at time step t; both satisfy the normalization constraint. For time steps at different locations within a continuous missing interval, differentiated fusion is performed by combining the positional bias term and confidence assessment results: when the missing point is close to the beginning of the missing interval, the weight of the forward time representation is increased; when the missing point is close to the end of the missing interval, the weight of the backward time representation is increased; when the missing point is located in the middle of the missing interval, the degree of balanced fusion of bidirectional representations is increased, so as to achieve fine-grained recovery of different locations within a continuous missing interval. S4.3, to and Perform weighted fusion to generate a bidirectional fusion representation: ; In the formula, Represents the bidirectional fusion characterization of time step t; S4.4, to Perform a linear transformation, to Map back to a dimension space consistent with the original electricity price time series: ; In the formula, To reconstruct the mapping matrix, a learnable linear transformation weight matrix is ​​used to... Linear mapping back to the 1-dimensional electricity price numerical space; For bias terms; The reconstructed electrical value for time step t; S4.5 For missing value locations, fill the corresponding gaps with the reconstructed electricity value after inverse normalization; for valid observation locations, retain their original electricity value; and concatenate the electricity values ​​of all time steps in the original time order to obtain the complete electricity price time series after missing value filling.

[0029] To ensure that the reconstructed electricity price series is consistent with the actual electricity price change pattern in terms of local shape and overall distribution, a reconstruction consistency constraint is introduced during the training process. The consistency constraint includes at least local smoothness constraint, peak and valley shape preservation constraint and statistical distribution consistency constraint, so as to reduce the problem of excessive smoothing or structural distortion under high missing rate conditions. The reconstruction results can be evaluated for reconstruction quality. When the reconstruction error or consistency deviation exceeds the preset threshold, the bidirectional fusion weights or gating parameters are adjusted in reverse, and the missing location estimation is re-executed, thereby improving the reconstruction stability. Finally, the reconstruction results of missing locations are combined with the observed electricity prices to generate a complete electricity price time series, which serves as the input for subsequent patch partitioning, reprogramming mapping, and prediction modules.

[0030] In S5, the patch length and sliding step size are adaptively adjusted based on the local fluctuation characteristics, periodic change characteristics and missing distribution of the electricity price time series. In short-term electricity price forecasting, the original time series is mainly an N-dimensional time series composed of electricity price time series and related characteristic variables (such as weather and fuel prices). Reversible instance normalization is performed on each input channel to eliminate distribution bias, ensuring it has zero mean and unit variance, and it is then divided into several consecutive overlapping or non-overlapping patches. This embodiment adopts a sliding partitioning principle, and according to the partitioning principle, the total number of patches P is: ; In the formula, T is the number of time steps; The length of each patch; S represents the horizontal sliding step size; The segmented patches are processed through a linear embedding layer. Mapped to a unified hidden input dimension The above yields the embedded representation. Where p is the index of the Patch. It represents the set of real numbers.

[0031] like Figure 2 As shown, the Patch reprogramming operation specifically involves using the word embedding matrix of a pre-trained large language model. Reprogramming as the basis of the target semantic space Where V is the vocabulary size and D is the hidden dimension of the pre-trained large language model; To alleviate the problem of excessive search space caused by direct reprogramming in the full vocabulary space, a set of text prototypes semantically related to electricity price time series is selected through a learnable fully connected layer, resulting in a selected text prototype matrix. ,in This is the number of text prototypes retained after filtering. ; By using a multi-head cross-attention mechanism, each patch is embedded into a text prototype matrix for alignment and mapping, resulting in a reprogrammed patch representation.

[0032] In S6, to address the semantic differences between pre-trained large language models and time series data, and to fully leverage their reasoning capabilities, a cue prefix (cue word prefix) is introduced. This prefix consists of a natural language template, which is fed into the model before the reprogrammed time series patch, enabling the LLM to fully clarify task instructions, relevant domain knowledge, and basic statistical information of the input sequence.

[0033] The prompt prefix specifically includes: Basic data information, including data source, domain background, and sampling frequency; this part provides the model with key domain context, enabling it to activate patterns and concepts related to a specific task in pre-trained knowledge, thereby establishing a preliminary semantic understanding of the input data.

[0034] The task instructions specify the length of the historical observation sequence and the future step size to be predicted, so as to ensure that the output format of the model is strictly aligned with the task objective and guide its generation behavior to focus on the specified prediction range. Electricity price time series statistical features, through feature extraction, transform the original electricity price time series into a semantic description with clear statistical meaning, including the maximum, minimum, median and overall trend of data. This can effectively alleviate the limitation of the model in being unable to directly identify statistical patterns from numerical sequences, and significantly enhance its structured perception of data distribution characteristics and dynamic evolution patterns.

[0035] An example prompt prefix is: <Basic Information> This dataset is a time-series electricity price data sourced from a company's official website for a specific region, with a sampling interval of 5 minutes. It records electricity price data for this region over many years. Each data point includes the time point and the corresponding electricity price data.

[0036] <Task Instruction> is currently executing <Task Type>, which predicts future electricity price data based on past <number of steps> of historical electricity price data.

[0037] <Statistical Characteristics> The maximum value of this dataset is <maximum value>, the minimum value is <minimum value>, the median is <median value>, and the overall trend is <overall trend>.

[0038] A dynamic cue prefix is ​​constructed and used to form a unified model input. A cue prefix is ​​constructed to activate the task comprehension ability of the pre-trained large language model, encoding task instructions, data sources, sampling frequency, region information, missing state descriptions, and sequence statistical features into natural language cue. The text prototype is dynamically selected based on the current task type and input sequence features, and consistency constraints between numerical features and semantic descriptions are constructed.

[0039] In S7, the unified sequence representation is fed into the frozen pre-trained large language model for forward inference. The prefix part corresponding to the prompt prefix is ​​discarded, and the output representation related to the patch is retained. The output representation is flattened and mapped to the target prediction step size dimension through the output projection layer to generate normalized electricity price prediction values ​​for multiple future time points. The normalized electricity price prediction value is then denormalized to obtain the future electricity price prediction result for the actual electricity price range, which can be expressed as: ; In the formula, O represents the output representation corresponding to Patch; For flattening operation; W is the output projection matrix; is the bias term; Y is the normalized electricity price forecast value within the future forecast time window. We perform inverse normalization on it to obtain the final forecast result that is consistent with the actual electricity price dimension.

[0040] Based on the above, the overall prediction architecture of this embodiment is as follows: Figure 3 As shown.

[0041] It can simultaneously construct prediction outputs at multiple time scales based on the same output representation, including short-term prediction results and longer-term prediction results, to meet the differentiated needs of different power trading and dispatch scenarios for prediction time domain. In order to improve the interpretability and risk assessment capability of prediction results in engineering applications, an uncertainty quantification mechanism can also be introduced. Through multiple random inactivation forward inferences, quantile regression output heads or equivalent probability modeling methods, a prediction distribution can be generated for the same input, and the prediction mean, the upper bound of the prediction interval and the lower bound of the prediction interval can be output. Based on the prediction results and their uncertainty information, the risk assessment is performed to determine whether electricity price spikes, abnormal jumps, or negative electricity price events will occur within the future time window, and the corresponding risk level or warning indicator is output to enhance the application value of the method in electricity market transaction decision-making and operational risk warning.

[0042] The verification process is as follows: Acquire time-series electricity price data for the target electricity market and perform time alignment and cleaning preprocessing on the data. Select electricity price data (priced in AUD / MWh) for Region 1 AEMO electricity market from February 1 to June 30, 2023, with a sampling interval of 5 minutes, totaling 43,200 sample points; Regarding the missing data setting, random missing values ​​were introduced into the complete dataset. Based on the original electricity price data, a specific proportion of data points were randomly selected using Python's random number generator, and the values ​​at these locations were replaced with the missing value identifier NaN defined in the NumPy library. The experiment set four gradient missing percentages: 5%, 10%, 20%, and 30% to simulate different levels of missing data.

[0043] The imputation results are shown in Table 1. Through in-depth analysis of the evaluation indicators of each model under four missing rates (5%, 10%, 20%, and 30%), it can be found that the LLM model is significantly better than the comparison method in terms of imputation accuracy, stability, and adaptability.

[0044] Table 1 Comparison of metrics for random missing filler prediction models Regarding imputation accuracy, the LLM model achieved RMSE, MAE, MAPE, and R under four missing rate conditions. 2All four key metrics remained optimal. At a 5% missing data rate, its RMSE was 2.45, lower than KNN, MF, and GAN by 39.1%, 46.7%, and 59.4%, respectively. At a high missing data rate of 30%, LLM's RMSE was lower than KNN, MF, and GAN by 24.3%, 24.5%, and 44.7%, respectively. While the relative reduction was smaller compared to the 5% missing data rate, the absolute difference in RMSE widened further. Regarding the MAE metric, LLM achieved 2.02 at a 30% missing data rate, higher than KNN and MF at a 20% missing data rate, fully demonstrating the significant advantage of the LLM model in absolute error control. From a relative error perspective, LLM also performed exceptionally well in the MAPE metric, achieving 0.10 at a 5% missing data rate, lower than KNN, MF, and GAN by 23.1%, 28.6%, and 65.5%, respectively, and this advantage continued to increase with the missing data rate.

[0045] Furthermore, the LLM model exhibited the highest R-value across all missing values. 2 The value indicates that the imputation result can more accurately preserve the distribution characteristics of the original data. Especially under the condition of low missing rate (5%), the R value of LLM is high. 2 The R-value reached 0.9947, significantly outperforming the comparison model. As the missing data rate gradually increased to 10%, 20%, and 30%, the R-value of all models decreased. 2 All showed a downward trend, but the decline in LLM was the most gradual, indicating that it is not sensitive to missing data and has a strong ability to maintain its structure.

[0046] In terms of stability, the LLM model exhibits excellent robustness. As the missing data rate increases from 5% to 30%, the MAPE value of the LLM model rises from 0.10 to 0.29, an increase of 190%, while the corresponding increases for KNN, MF, and GAN reach 392%, 600%, and 1360%, respectively. This relatively gradual performance degradation trend indicates that the LLM model has lower sensitivity to missing data and stronger adaptability. Regarding the MAE metric, the advantage of LLM over KNN and MF at a 30% missing data rate is significantly greater than its advantage at a 5% missing data rate. Furthermore, the LLM model performs better in R... 2 It also shows certain advantages in terms of metrics, maintaining a determination coefficient of 0.9897 even with a 30% missing rate, which is significantly higher than other methods. The above analysis shows that the LLM model is particularly suitable for handling high missing rate scenarios commonly encountered in practical applications.

[0047] In summary, the comprehensive advantages demonstrated by the LLM model in the task of imputing randomly missing data are not only reflected in its significantly superior performance across various evaluation metrics, but also in its adaptability and stability to data with varying degrees of missing information. These characteristics make the LLM model an effective tool for solving data missing problems in practical engineering, and it has significant practical value, especially in application scenarios with high data quality requirements.

[0048] In the short-term electricity price forecasting case study, the model proposed in this embodiment was compared with three other models: LSTM, DLinear, and Autoformer. The model parameter settings are shown in Table 2.

[0049] Table 2 Model Parameter Settings Table 3 Comparison of Predictive Indicators for Region 1 Model Comparison of results indicators for each model is shown in Table 3 and Figure 4 As shown, according to Table 3 and Figure 4 In the comparison of model errors, the LLM proposed in this embodiment significantly outperforms other comparative models in all evaluation metrics, demonstrating excellent predictive performance. In terms of RMSE, the LLM model's value is 4.5559, which is approximately 43.3%, 49.0%, and 53.0% lower than Autoformer, DLinear, and LSTM, respectively, showing its significant advantage in overall error control. In MAE, the LLM value is 2.1834, which is approximately 49.2%, 51.7%, and 51.9% lower than the three models, respectively. In MAPE, the LLM value is only 4.84%, while Autoformer, DLinear, and LSTM are 7.35%, 9.42%, and 8.09%, respectively. The LLM reduces errors by approximately 34.2%, 48.6%, and 40.2% compared to the three models, indicating that it also performs best in relative error control. Furthermore, the LLM's coefficient of determination R0 is significantly higher than the MAE. 2 The LLM model achieved an accuracy of 0.9901, higher than Autoformer (0.9730), DLinear (0.9656), and LSTM (0.9590), indicating that it can better explain the variation in electricity price data and has a better fit. In summary, the LLM model significantly outperforms other benchmark models on all key error metrics, with improvements generally exceeding 30% to 50%, validating its higher accuracy and stability in short-term electricity price forecasting tasks.

[0050] To further demonstrate the superior performance and strong generalization ability of the LLM model in short-term electricity price forecasting, a transfer validation was performed in another region (Region 2).

[0051] Table 4 Comparison of Predictive Indicators for Region 2 Model Comparison of results indicators for each model is shown in Table 4 and Figure 5 As shown, the LLM model maintains its leading position across all four evaluation metrics. Specifically, in terms of RMSE, LLM significantly reduces the error by 38.8%, 45.5%, and 34.3% compared to Autoformer, DLinear, and LSTM, respectively. In terms of MAE, LLM also shows a considerable reduction compared to the other three, reaching 35.6%, 39.2%, and 26.6%, respectively. These results indicate that the absolute deviation between the predicted and actual values ​​of the LLM model is much lower than that of the comparative models under different data distributions. From the perspective of relative error, the MAPE of LLM is much lower than that of the comparative models. Compared to Autoformer, DLinear, and LSTM, its relative error is reduced by 23.2%, 35.4%, and 32.4%, respectively. This further highlights the stable advantage of the LLM model in percentage error control. Furthermore, the coefficient of determination R of the LLM model... 2 The R² value reached 0.9856, indicating that it can explain 98.56% of the electricity price data variation, demonstrating an extremely high goodness of fit. In comparison, Autoformer, DLinear, and LSTM have lower R² values. 2 These figures decreased by approximately 2.03%, 5.43%, and 4.55%, respectively. This clearly demonstrates that the LLM model has a greater advantage in capturing the inherent patterns in the data.

[0052] In summary, on the new dataset, the performance improvement of the LLM model compared to the baseline model is generally between 20% and 45%. This result strongly replicates and reinforces the conclusions of previous experiments, fully demonstrating that the model not only has high accuracy, but also excellent generalization ability and robustness.

[0053] Imputation prediction under random missing values: To compare the performance of predictions under missing data conditions, prediction models were used to predict 1440 future time points with imputation of each model when data was missing.

[0054] The results are compared in Table 5. Overall, the LLM-LLM model proposed in this embodiment exhibits the best predictive performance across all missing rates. Taking RMSE as an example, at a missing rate of 5%, the LLM-LLM value is 4.56, significantly lower than LLM-Autoformer and LLM-LSTM, demonstrating its stronger robustness in handling partially missing data.

[0055] Table 5 Comparison of Predictive Metrics for Random Missing Information Imputation In terms of goodness of fit, LLM-LLM has an R-value of [missing value]. 2 The R-value remained above 0.988 across all missing values, reaching a maximum of 0.9899, ​​significantly outperforming other comparative models. For example, at a missing value of 20%, LLM-LLM achieved a high R-value. 2 The value was 0.9893, while GAN-LLM was only 0.9859, indicating that LLM-LLM can still capture the inherent patterns of electricity price time series well even under complex missing modes.

[0056] Further analysis of the MAPE and MAE metrics reveals that LLM-LLM maintains low errors across various missing data scenarios. For example, at a missing data rate of 10%, its MAPE is 4.89% and MAE is 2.22, both superior to KNN-LLM (4.87%, 2.31) and MF-LLM (4.87%, 2.52). It is noteworthy that while GAN-LLM exhibits slightly lower MAPE at certain missing data rates, its RMSE and MAE are generally higher, indicating greater fluctuations in its prediction results and lower stability compared to LLM-LLM.

[0057] In summary, the LLM-LLM model maintains high prediction accuracy and stability under various missing rates, highlighting its effectiveness and superiority in short-term electricity price forecasting in scenarios with large-scale missing data.

[0058] Example 2: The difference between this example and Example 1 is that an adaptive mutation-aware enhancement mechanism is introduced after generating the bidirectional fusion representation and before performing the linear transformation. Specifically: A mutation event detection branch is constructed to perform mutation sensitivity analysis on the bidirectional fusion representation: the ratio of the slope of the change in the representation in the neighborhood of the current time step to the background fluctuation level is calculated to obtain the local mutation index; the larger the index, the more likely the current position is to be at the electricity price mutation boundary; When the mutation index exceeds the preset threshold, the time step is determined to be in the potential mutation range, triggering adaptive context reconstruction: the original fixed window bidirectional fusion strategy is changed, and multiple stable anchor points in the forward and backward directions are dynamically searched with the current time step as the center. The stable anchor points must meet the restriction that the mutation index is less than half of the preset threshold, thereby forming an adaptive context window that excludes interference information. Relationship modeling of representations within an adaptive context window is performed using a graph attention network: an adaptive adjacency matrix based on temporal distance and representation similarity is constructed, enabling anchors that are temporally distant but semantically similar to interact directly, generating a mutation-aware enhanced representation; this representation reduces estimation uncertainty within mutation intervals by aggregating information from stable anchors. A mutation gating mechanism is constructed: a lightweight neural network dynamically generates gating coefficients based on the mutation index, and performs weighted fusion of the mutation-aware enhanced representation and the original bidirectional fusion representation; when the mutation index is high, the enhanced representation dominates, and the estimation bias of the mutation interval is corrected by using the reconstructed stable context; when the mutation index is low, the original bidirectional fusion representation is maintained to avoid excessive intervention in the normal fluctuation interval. The final fused representation replaces the original bidirectional fused representation, and subsequent linear transformation and missing value imputation operations are performed.

[0059] In response to abrupt changes such as peak electricity prices and negative electricity prices commonly seen in the electricity market, this embodiment significantly improves the prediction accuracy of extreme electricity price events by dynamically identifying the boundaries of these changes and reconstructing a reliable context, while maintaining the stability of predictions within normal ranges.

[0060] Example 3: A device for reconstructing and predicting electricity price time series based on a large language model, comprising: One or more processors; Memory, used to store one or more computer programs; When one or more programs are executed by one or more processors, the one or more processors perform the method in Embodiment 1 or Embodiment 2.

[0061] Example 4: A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method in Example 1 or Example 2.

Claims

1. A method for reconstructing and predicting electricity price sequences based on a large language model, characterized in that... Includes the following steps: S1. Obtain and preprocess the electricity price time series of the target electricity market, and construct a missing value identifier matrix of the same length as the electricity price time series to identify the location of observations and missing values. Determine the missing value type for the missing value location and construct a missing value pattern encoding matrix. S2. Extract dynamic prior features from the preprocessed electricity price time series, including local fluctuation intensity, time interval between adjacent valid observation points, and length of continuous missing intervals; S3. The preprocessed electricity price time series is constructed into forward input branches and backward input branches according to the forward and reverse time order, respectively. For each time step, the normalized electricity value, missing state, missing type and dynamic prior features are extracted and combined with the forward input branch and the backward input branch to form joint input features. These are mapped to the hidden dimension space of the pre-trained large language model to obtain the missing time embedding. After superimposing the multi-scale time position encoding, it is input into the pre-trained large language model to extract the forward time representation and the backward time representation. S4. Confidence assessment is performed on the forward time representation and the backward time representation. The fusion weight is dynamically calculated based on the confidence assessment results. The forward time representation and the backward time representation are weighted and fused to generate a bidirectional fused representation. The missing positions are numerically estimated based on the bidirectional fused representation to obtain the complete electricity price time series after the missing values ​​are filled. S5. Divide the complete electricity price time series into multiple patches according to a preset time window, map each patch to a unified hidden dimension space, and then perform patch reprogramming operation to make the patch embedding aligned to the semantic space of the pre-trained large language model. S6. Construct a hint prefix and concatenate the hint prefix with the reprogrammed Patch to form a unified sequence representation; S7. The unified sequence representation is fed into the pre-trained large language model for forward inference. The output result is linearly projected and inversely normalized to obtain the electricity price prediction results for multiple future time points.

2. The method for reconstructing and predicting electricity price sequences based on a large language model as described in claim 1, characterized in that, In S1, the preprocessing specifically involves: sorting the electricity price time series according to timestamps to construct a unified time index set; merging duplicate timestamps according to preset rules; and retaining timestamp gaps and marking the corresponding positions as missing value positions when there are timestamp gaps. Perform outlier detection and processing, identify outliers containing spike values, outliers, or anomalous jump points, and remove, correct, or mark outliers; the missing positions generated after outlier processing are also marked as missing value positions; Min-Max normalization is performed to map non-missing samples in the electricity price time series to the [0,1] interval.

3. The method for reconstructing and predicting electricity price sequences based on a large language model as described in claim 1, characterized in that, In S1, a mask matrix of the same length as the electricity price time series is constructed as a missing value identifier matrix. When the electricity price corresponding to the time point is an observed valid value, the mask is set to 1; when the electricity price corresponding to the time point is a missing value, the mask is set to 0 and the missing value identifier is set to NaN. The missing locations are identified by type. Based on the source of the missing points, the temporal distribution pattern, the length of consecutive missing points, and the characteristics of neighboring observations, the missing value locations are divided into random missing, continuous interrupted missing, abnormally removed missing, and artificially simulated missing. Missing pattern coding information is generated through one-hot coding to form a multi-state missing pattern coding matrix.

4. The method for reconstructing and predicting electricity price sequences based on a large language model as described in claim 1, characterized in that, In S3, the process of obtaining the forward time representation and the backward time representation is as follows: Multi-scale temporal location coding includes at least continuous time step location coding, intra-day periodic location coding, and intra-weekly periodic location coding, which are used to jointly describe temporal sequence features and periodic structure features. The forward input branch and its corresponding normalized electrical value, missing state, missing type and dynamic prior features constitute the forward joint feature sequence, and the reverse input branch and its corresponding normalized electrical value, missing state, missing type and dynamic prior features constitute the reverse joint feature sequence. The forward and reverse joint feature sequences are transformed into dimensions and mapped to the same dimension space as the hidden layer of the pre-trained large language model to generate forward missing time embedding sequences and reverse missing time embedding sequences. Multi-scale temporal position encoding is superimposed on forward and reverse missing-aware temporal embedding sequences to generate forward and reverse input representations, respectively. The forward input representation is fed into the forward branch of the pre-trained large language model to learn the trend continuation, periodic evolution and local stationarity features of the electricity price time series, and the forward time representation is extracted. The backward input representation is fed into the backward branch of the pre-trained large language model to learn the missing interval recovery, mutation boundary recognition and future context compensation features, and the backward time representation is extracted.

5. The method for reconstructing and predicting electricity price sequences based on a large language model according to claim 4, characterized in that, In S3, a direction-aware gating module is introduced at the input ends of the forward branch and the reverse branch respectively. Based on the missing rate, the length of the continuous missing interval, the intensity of local fluctuations and the distribution density of effective observation points in the input window, the ratio of the passage of historical context information to future context information is dynamically adjusted. For pre-trained large language models, freeze their multi-head self-attention layers and feedforward network layers, while opening up the positional encoding parameters, layer normalization parameters, and low-rank adapter parameters for updating.

6. The method for reconstructing and predicting electricity price sequences based on a large language model according to claim 1, characterized in that, In S4, the process of obtaining the complete electricity price time series after missing value filling is as follows: S4.1 Construct confidence evaluation branches for forward and backward time representations respectively. Estimate the confidence of forward and backward time representations based on local context integrity, neighborhood observation density, representation stability, context consistency, and the relative position of the current time step in the continuous missing interval. Forward confidence at time step t and backward confidence They are defined as follows: ； ； In the formula, This represents the forward time representation of time step t; represents the backward time representation at time step t; Sim represents the similarity function between the forward and backward time representations. This represents the distance from time step t to the previous valid observation point; This represents the distance from time step t to the next valid observation point; This indicates the local observation density in the forward direction; This indicates the local observation density in the backward direction; This represents the relative position encoding of time step t within a consecutive missing interval; Indicates the length of the consecutive missing interval at time step t; This represents the intensity of local fluctuations within the neighborhood of time step t. , , , , , For preset weighting coefficients, This is the Sigmoid activation function, used to constrain the confidence level to the (0,1) interval; Defined as: ； In the formula, To prevent positive numbers with a denominator of zero; S4.2, based on and The forward fusion weights and backward fusion weights are calculated using normalization: ； ； In the formula, Indicates the forward fusion weights at time step t; This represents the backward fusion weights at time step t; S4.3, to and Perform weighted fusion to generate a bidirectional fusion representation: ； In the formula, Represents the bidirectional fusion characterization of time step t; S4.4, to Perform a linear transformation, to Map back to a dimension space consistent with the original electricity price time series: ； In the formula, To reconstruct the mapping matrix, a learnable linear transformation weight matrix is ​​used to... Linear mapping back to the 1-dimensional electricity price numerical space; For bias terms; The reconstructed electrical value for time step t; S4.5 For missing value locations, fill the corresponding gaps with the reconstructed electricity value after inverse normalization; for valid observation locations, retain their original electricity value; and concatenate the electricity values ​​of all time steps in the original time order to obtain the complete electricity price time series after missing value filling.

7. The method for reconstructing and predicting electricity price sequences based on a large language model according to claim 1, characterized in that, In S5, the total number of patches P is: ； In the formula, T is the number of time steps; The length of each patch; S represents the horizontal sliding step size; The segmented patches are processed through a linear embedding layer. Mapped to a unified hidden input dimension The above yields the embedded representation. Where p is the index of the Patch. It represents the set of real numbers.

8. The method for reconstructing and predicting electricity price sequences based on a large language model according to claim 7, characterized in that, In S5, the Patch reprogramming operation specifically involves using the word embedding matrix of a pre-trained large language model. Reprogramming as the basis of the target semantic space Where V is the vocabulary size and D is the hidden dimension of the pre-trained large language model; A set of text prototypes semantically related to electricity price time series is selected through a learnable fully connected layer, resulting in a selected text prototype matrix. ,in This is the number of text prototypes retained after filtering. ; By using a multi-head cross-attention mechanism, each patch is embedded into a text prototype matrix for alignment and mapping, resulting in a reprogrammed patch representation.

9. The method for reconstructing and predicting electricity price sequences based on a large language model according to claim 1, characterized in that, In S6, the prompt prefix specifically includes: Basic data information, including data source, field background, and sampling frequency; The mission instructions specify the length of the historical observation sequence and the future step size to be predicted. Electricity price time series statistical features are extracted to transform the original electricity price time series into a semantic description with clear statistical meaning, including the maximum, minimum, median and overall trend of the data.

10. The method for reconstructing and predicting electricity price sequences based on a large language model according to claim 5, characterized in that, In step S7, the unified sequence representation is fed into the frozen pre-trained large language model for forward inference. The prefix part corresponding to the prompt prefix is ​​discarded, the output representation related to the patch is retained, the output representation is flattened, and mapped to the target prediction step size dimension through the output projection layer to generate normalized electricity price prediction values ​​for multiple future time points. The normalized electricity price prediction value is then denormalized to obtain the future electricity price prediction result for the actual electricity price value range.

Images