Wind power prediction method and equipment based on pre-trained large language model

Abstract

This invention relates to a wind power prediction method and device based on a pre-trained large language model, belonging to the field of wind power generation. The method includes: dividing a wind power data sequence into multiple time-series patches; dividing each time-series patch into multiple equal-length subsequences; extracting data features from the equal-length subsequences; converting the data features into semantic prompts; adding text instructions to the semantic prompts to obtain semantic instructions; concatenating the time-series patches and their corresponding positional codes, time codes, and semantic instructions to obtain fused samples corresponding to the time-series patches; inputting multiple fused samples into a large language model for pre-training and adjusting the parameters of the large language model to obtain a wind power prediction model; inputting fused samples of new wind power data into the wind power prediction model and outputting predicted wind power values. This invention can achieve high-precision wind power prediction by using a pre-trained large language model with fused samples, even with limited historical operating data samples.

Description

Technical Field

[0001] This invention relates to the field of wind power generation technology, and in particular to a wind power prediction method and device based on a pre-trained large language model. Background Technology

[0002] As the global energy landscape shifts towards low-carbon and zero-carbon energy, wind power, as a key pillar of this transition, is experiencing rapid growth in newly installed capacity. Accurately predicting the wind power output of new wind farms is therefore crucial, as it helps reduce grid instability and risks caused by wind power fluctuations and effectively mitigates the negative impact of wind power uncertainty on the power grid.

[0003] Currently, most mainstream wind power forecasting methods rely on a large amount of historical power data for training and supervision. However, the historical operating data samples of newly built wind farms in the initial operation phase are relatively small, which makes traditional forecasting models prone to performance bottlenecks. The prediction accuracy of current wind power forecasting methods drops significantly, making high-precision wind power forecasting extremely difficult.

[0004] Large Language Models (LLMs) possess significant few-shot learning capabilities. However, they cannot be directly applied to wind power prediction for several reasons. Firstly, LLMs are designed to process discrete tokens, while historical wind farm data is continuous time-series data. The data structures of the two are significantly different, making it impossible to directly apply the data processing methods of LLMs to wind power prediction. Secondly, LLMs do not learn how to interpret time-series patterns or perform related reasoning during pre-training, thus struggling to understand crucial temporal information. Summary of the Invention

[0005] To address the aforementioned issues and predict wind power output when historical operational data samples from wind farms are limited, the applicant has developed this invention, providing a wind power output prediction method and device based on a pre-trained large language model.

[0006] In a first aspect, the present invention provides a wind power prediction method based on a pre-trained large language model, comprising:

[0007] The wind power data sequence is divided into multiple time-series patches. Based on the sequence position of the wind power data within each time-series patch, the positional encoding corresponding to that patch is determined. Based on the timestamp of the wind power data within each time-series patch, the time-series encoding corresponding to that patch is determined. The time-series patches are then divided into multiple equal-length subsequences. Data features are extracted from each equal-length subsequence. These data features are converted into semantic prompts that can be understood by a large language model. Text instructions are added to these semantic prompts to obtain semantic instructions. The time-series patches, their corresponding positional encodings, time-series encodings, and semantic instructions are concatenated to obtain fused samples corresponding to each time-series patch. Multiple fused samples are input into a large language model for pre-training, and the parameters of the large language model are adjusted to obtain a wind power prediction model. New fused samples of wind power data are input into the wind power prediction model, and the wind power prediction model outputs predicted wind power values.

[0008] Secondly, the present invention provides a wind power prediction device based on a pre-trained large language model, comprising:

[0009] The data fusion module is used to divide the wind power data sequence into multiple time-series patches; determine the position code corresponding to the time-series patch based on the sequence position of the wind power data in the time-series patch; determine the time code corresponding to the time-series patch based on the timestamp of the wind power data in the time-series patch; divide the time-series patch into multiple equal-length subsequences; extract the data features of each equal-length subsequence; convert the data features into semantic prompts that can be understood by a large language model; add text instructions to the semantic prompts to obtain semantic instructions; and concatenate the time-series patch and its corresponding position code, time code, and semantic instructions to obtain the fusion sample corresponding to the time-series patch.

[0010] The model optimization module is used to input multiple fused samples into a large language model for pre-training and to adjust the parameters of the large language model to obtain a wind power prediction model.

[0011] The power prediction module is used to input the fused sample of new wind power data into the wind power prediction model, and the wind power prediction model outputs the wind power prediction value.

[0012] Based on the same inventive concept, this invention also provides a method for constructing a wind power prediction model based on a pre-trained large language model, comprising the following steps:

[0013] The wind power data sequence is divided into multiple time-series patches; the position code corresponding to the time-series patch is determined based on the sequence position of the wind power data in the time-series patch; the time code corresponding to the time-series patch is determined based on the timestamp of the wind power data in the time-series patch; the time-series patch is divided into multiple equal-length subsequences; the data features of each equal-length subsequence are extracted; the data features are converted into semantic prompts that can be understood by a large language model; text instructions are added to the semantic prompts to obtain semantic instructions; the time-series patch and its corresponding position code, time code, and semantic instructions are concatenated to obtain the fusion sample corresponding to the time-series patch; multiple fusion samples are input into the large language model for pre-training, and the parameters of the large language model are adjusted to obtain the wind power prediction model.

[0014] Based on the same inventive concept, this invention also provides a wind power prediction model based on a pre-trained large language model, comprising:

[0015] The sequence discretization module is used to divide the wind power data sequence into multiple time-series patches; determine the position code corresponding to the time-series patch based on the sequence position of the wind power data in the time-series patch; determine the time code corresponding to the time-series patch based on the timestamp of the wind power data in the time-series patch; divide the time-series patch into multiple equal-length subsequences; extract the data features of each equal-length subsequence; convert the data features into semantic prompts that can be understood by a large language model; add text instructions to the semantic prompts to obtain semantic instructions; and concatenate the time-series patch and its corresponding position code, time code, and semantic instructions to obtain the fusion sample corresponding to the time-series patch.

[0016] The pre-training and parameter tuning module is used to input multiple fused samples into the large language model for pre-training and to adjust the parameters of the large language model to obtain the wind power prediction model.

[0017] Based on the same inventive concept, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the aforementioned wind power prediction method based on a pre-trained large language model, or implements the aforementioned wind power prediction model construction method based on a pre-trained large language model.

[0018] Based on the same inventive concept, the present invention also provides a computer storage medium storing computer-executable instructions, which, when executed, implement the aforementioned wind power prediction method based on a pre-trained large language model, or implement the aforementioned wind power prediction model construction method based on a pre-trained large language model.

[0019] The beneficial effects of the above-mentioned technical solution provided by the present invention include at least the following:

[0020] The aforementioned wind power prediction method based on a pre-trained large language model can provide discrete fusion samples suitable for input to the large language model. These fusion samples include wind power data, location codes that retain relative positional information over time, time codes reflecting dynamic features of the time dimension, and semantic instructions carrying high-level semantic information of the time series. The discrete fusion samples not only integrate local and global features of the time series but also simultaneously capture dynamic time information and semantic context, providing rich input expressions for subsequent prediction tasks and enabling the wind power prediction model to better learn time dependencies. This method can achieve high-accuracy wind power prediction even with limited historical operational data samples, by using fusion samples to pre-train the large language model.

[0021] The aforementioned method for constructing a wind power prediction model based on a pre-trained large language model can provide discrete fusion samples suitable for input to the large language model. These fusion samples include wind power data, location codes that retain relative positional information over time, time codes reflecting dynamic features of the time dimension, and semantic instructions carrying high-level semantic information of the time series. The discrete fusion samples not only integrate local and global features of the time series but also simultaneously capture dynamic time information and semantic context, providing rich input expressions for subsequent prediction tasks and enabling the wind power prediction model to better learn time dependencies. The wind power prediction model constructed using this method can achieve high-accuracy wind power prediction even with limited historical operational data samples.

[0022] Other features and advantages of the present invention will be set forth in the following description.

[0023] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0024] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0025] Figure 1 This is a flowchart of the wind power prediction method based on a pre-trained large language model in an embodiment of the present invention;

[0026] Figure 2 This is one of the schematic diagrams of the PowerMistral framework for wind power prediction model in an embodiment of the present invention;

[0027] Figure 3 This is the second schematic diagram of the PowerMistral framework for wind power prediction model in this embodiment of the invention;

[0028] Figure 4This is the third schematic diagram of the PowerMistral framework for wind power prediction model in this embodiment of the invention;

[0029] Figure 5 This is a schematic diagram of time encoding conversion in an embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of a multi-head attention framework in an embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of the LoRA fine-tuning process in an embodiment of the present invention;

[0032] Figure 8 For each model R in the 100% dataset of this embodiment of the invention 2 Comparison chart;

[0033] Figure 9 For each model R in the 5% dataset of this embodiment of the invention 2 Comparison chart;

[0034] Figure 10 This is a schematic diagram illustrating the impact of different prediction horizons on model performance in an embodiment of the present invention;

[0035] Figure 11 This is a schematic diagram illustrating the impact of different low-rank values ​​on model performance in an embodiment of the present invention. Detailed Implementation

[0036] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0037] To address the problems existing in the prior art, embodiments of the present invention provide a wind power prediction method and device based on a pre-trained large language model, a method for constructing a wind power prediction model based on a pre-trained large language model, a wind power prediction model based on a pre-trained large language model, a computer program product, and a computer storage medium.

[0038] This invention provides a wind power prediction method based on a pre-trained large language model, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0039] Step S1: Divide the wind power data sequence into multiple time-series patches; determine the position code corresponding to the time-series patch based on the sequence position of the wind power data in the time-series patch; determine the time code corresponding to the time-series patch based on the timestamp of the wind power data in the time-series patch; divide the time-series patch into multiple equal-length subsequences; extract the data features of each equal-length subsequence; convert the data features into semantic prompts that can be understood by the large language model; add text instructions to the semantic prompts to obtain semantic instructions; concatenate the time-series patch and its corresponding position code, time code, and semantic instructions to obtain the fusion sample corresponding to the time-series patch.

[0040] By pre-training on a text corpus containing a large number of tokens, Large Language Models (LLMs) exhibit significant few-shot learning capabilities, making them particularly suitable for scenarios where data is scarce, sensitive, or expensive. Fine-tuning LLMs allows them to be adapted to specific tasks with only a small dataset. Furthermore, compared to training a model from scratch, fine-tuning significantly reduces computational resource requirements because it only requires adjusting a subset of weights and training on a smaller dataset.

[0041] Large language models, through pre-training and fine-tuning on large-scale text corpora, are able to handle long texts and possess strong context awareness capabilities.

[0042] In some specific embodiments, the Mistral-7B large language model based on the Transformer architecture is used for small-sample wind power prediction. The Mistral-7B constructed using the method of this invention is called the wind power prediction model, or PowerMistral. The Mistral-7B used in this embodiment is an open-source, small-to-medium-sized LLM based on the Transformer architecture, and incorporates Group Query Attention (GQA) and Sliding Window Attention (SWA) techniques. This allows Mistral-7B to improve inference speed while effectively solving the computational efficiency problem when processing long sequences and significantly reducing memory requirements, making it suitable for real-time applications. Mistral-7B has 7 billion parameters and demonstrates excellent language understanding and inference capabilities through massive and diverse pre-trained data. The model supports calls from multiple programming languages, facilitating integration with various systems.

[0043] Leveraging the powerful capabilities of Mistral-7B, this embodiment constructs a framework for wind power prediction—PowerMistral. This framework fully utilizes the transfer learning capabilities of pre-trained LLM models, effectively addressing prediction challenges in small-sample scenarios. Through model fine-tuning, training from scratch is avoided, significantly reducing data requirements and computational overhead while ensuring the accuracy of prediction results. PowerMistral employs a two-stage fine-tuning strategy, as shown in the framework below. Figure 2 , 3 As shown in Figure 4. Figure 2 The diagram shows the basic framework of PowerMistral, where Q, K, and V represent the query matrix, key matrix, and value matrix in Multi-Head Attention (MHA) or Multi-Head Self Attention (MHSA), respectively. , , Let Q, K, and V represent the parameters to be learned, respectively. LoRA stands for low-rank fine-tuning, which is an efficient fine-tuning strategy. Figure 3 The diagram shown illustrates data fusion. The input patch represents the time sequence patch used for data fusion. The data sequence in the input patch is merely an example, and such a time sequence patch may not necessarily be used in practice. Figure 4 The diagram illustrates a two-stage fine-tuning strategy. The first stage, time-series feature alignment, aims to help the LLM adapt to the temporal characteristics of wind power data, ensuring the model can effectively handle time dependencies and temporal information. The second stage, prediction fine-tuning, focuses on optimizing the model's performance in wind power time-series prediction tasks, improving prediction accuracy. In the time-series feature alignment stage, InstanceNorm is used to capture the temporal patterns of the data, and the learning effect of temporal features is optimized by calculating the error of the temporal patches. In the prediction fine-tuning stage, RevIN (Reversible Instance Normalization) is introduced, using a reversible mechanism to restore the model output to the original data scale, further reducing wind power prediction errors and improving model accuracy.

[0044] In the preprocessing of wind power data sequences, normalization is performed to eliminate differences in data distribution.

[0045] After normalization, the PatchTST method is used to divide the wind power data sequence into several time-series patches. Two consecutive patch regions can overlap or not overlap, and the specific length is determined by the time-series patch length and the horizontal sliding step size (the non-overlapping area between two consecutive patches).

[0046] In some specific embodiments, the wind power data sequence is divided into multiple time-series patches, including the following steps: The wind power data sequence is divided in a non-overlapping manner to obtain N time-series patches, where N = [(L - P) / S] + 2, L represents the length of the wind power data sequence, P represents the length of the time-series patch, and S represents the horizontal sliding step size on the wind power data sequence. (L - P) / S is then rounded down. This segmentation method has three major advantages: 1) It aggregates the local fluctuation characteristics of the wind power data sequence into each patch, preserving key information and compensating for the temporal dependency defects of point-by-point encoding; 2) It significantly reduces the number of input tokens, lowering computational complexity; 3) Each patch can be mapped to semantic prompts that LLM can understand through language representation.

[0047] To enable the model to accurately capture the sequence position information in the input features, a positional-encoding (PE) mechanism is introduced for each time series patch. PE assigns a unique positional identifier to each feature using sine and cosine functions, allowing the model to explicitly perceive the relative and absolute positional information within the sequence. In some specific embodiments, determining the positional encoding corresponding to the time series patch based on the sequence position of the wind power data within the time series patch includes the following steps:

[0048] Determine the position embedding matrix ;

[0049] pass Determine the position code corresponding to the timing patch. Indicates timing patch i The location encoding, where pos represents the sequence location index of the wind power data in the time-series patch. i Indicates the time-series patch index. Indicates the length of the timing patch.

[0050] Most current predictive models for univariate or multivariate variables focus on using historical values ​​of variables to uncover dependencies across time and variables, neglecting the potential semantic information in timestamps. Timestamps are crucial indicators of the context of a time series, indirectly reflecting the range of fluctuations in observed values ​​at a given moment. For example, peak and trough values ​​in electricity load often occur at fixed times each day. Similarly, in meteorological factors, during the day, the ground is heated by strong solar radiation, resulting in frequent heat exchange with the atmosphere, increased turbulence, and relatively high wind speeds; while at night, solar radiation weakens, the atmosphere tends to be more stable, and wind speeds are typically lower. Based on the above analysis, timestamps represent a potential constraint on or determine the observed values ​​of variables; different combinations of timestamps correspond to certain observed values ​​of variables in a multidimensional vector space.

[0051] To fully explore the potential information contained in timestamps, this embodiment uses a linear layer to perform time-series encoding on historical timestamps (minutes, hours, days, weeks) and map them to the hidden space, where the encoded timestamps are input into the model.

[0052] In some specific embodiments, such as Figure 5 As shown, determining the time code corresponding to the time series patch based on the timestamp of the wind power data in the time series patch includes the following steps:

[0053] The timestamps of the wind power data in the time series patch are subjected to the first linear transformation to obtain the time data after dimensionality reduction.

[0054] The time data is normalized and regularized, and then convolutional processing is performed to obtain the time data features;

[0055] The time data features are subjected to a second linear transformation to obtain the time code corresponding to the time series patch.

[0056] Normalized time-series patches are expressed as vectors It indicates, and further divides into I Non-overlapping subsequences of equal length .in, Each timing patch Transformed into basic embedding code using Patch Embedder Furthermore, by analyzing each small segment of data... Extract local features and transform them into semantic information that LLM can understand, including data trends. (Rising, Falling, Stable, Stopping) and numerical descriptions (e.g., maximum value, minimum value, average value, variance). Patch Embedder refers to an embedding encoding method, or the program corresponding to an embedding encoding method.

[0057] In some specific embodiments, converting the data features into semantic prompts that a large language model can understand includes the following steps:

[0058] The attention mechanism in the Transformer library is invoked to apply the timing patches respectively. i Data trends With numerical description Transformed into semantic cue words that can be understood by a large language model, among which, , , Indicates timing patch i Normalized vector, Indicates to Extracting data trend features Indicates to Perform numerical descriptive feature extraction. Indicates to Extract the maximum value. Indicates to Extract the minimum value. Indicates to Extract the average value. Indicates to Variance extraction was performed.

[0059] Based on this, by adding text instructions (such as "text instructions on the trend of data for this time segment" and "text instructions on the numerical description of this time segment"), two complete semantic instructions are integrated. and .

[0060] In some specific embodiments, the timing patch and its corresponding position code, time code, and semantic instructions are concatenated to obtain the fused sample corresponding to the timing patch, including the following steps:

[0061] pass Obtain timing patch i The corresponding fusion samples, among which, Indicates timing patch i Fusion samples, This indicates a splicing operation. Indicates timing patch i Basic embedding encoding, Indicates timing patch i Location encoding, Indicates timing patch i Time encoding, Indicates timing patch i Semantic instructions for data trends Indicates timing patch i The semantic instructions described by numbers.

[0062] The aforementioned fused samples comprehensively characterize wind power data sequences from multiple perspectives, and their embedding includes the following four parts: 1) Basic features 1) Preserves the local patterns of wind power segments; 2) Semantic feature embedding and It carries high-level semantic information of the time series; 3) Location embedding 4) Time embedding This reflects the dynamic characteristics of the time dimension. Through this fusion method, not only can local and global features of time series be integrated, but dynamic temporal information and semantic context can also be captured simultaneously, providing rich input representations for subsequent prediction tasks.

[0063] Step S2: Input multiple fused samples into the large language model for pre-training, and adjust the parameters of the large language model to obtain the wind power prediction model.

[0064] The core architecture of Mistral-7B is based on a Transformer model with an attention mechanism. See [link to specific architecture] for details. Figure 2 The model primarily consists of a multi-head self-attention mechanism (MHSA) and a position-forward network (FFN). To improve computational efficiency, the attention mechanism incorporates sliding window attention (SWA) and grouped query attention (GQA) for optimization. Furthermore, the model ensures efficient information transfer through residual connections and utilizes layer normalization (LayerNorm) to enhance training stability and convergence.

[0065] like Figure 6 As shown, Multi-Head Attention (MHA) is a core component of the Transformer architecture, used to capture the correlations between different parts of the input data in sequence modeling. Its core method involves computing multiple attention heads in parallel, enabling the model to focus on different features of the input data in different subspaces, thereby improving the model's ability to capture complex patterns.

[0066] If given H If there are multiple attention heads, the calculation process of the multi-head attention mechanism is as follows:

[0067] , ,

[0068] In the formula, , , The input to the multi-head attention mechanism consists of the query matrix, key matrix, and value matrix. to For independent attention heads; It is the output projection matrix. , , These are the query matrix, key matrix, and value matrix corresponding to the h-th attention group, respectively. This represents multi-head attention data. This indicates a splicing operation. This indicates the calculation of the attention head function.

[0069] A single attention head uses Scaled Dot-Product Attention, calculated as follows:

[0070] ,

[0071] In the formula, , , , , , This is the weight matrix; The dimension of the query vector; This is the input matrix for the multi-head attention mechanism.

[0072] The position-forward network, as a parallel module of the multi-head self-attention mechanism, operates at each sequence position. Through two fully connected layers and a non-linear activation function, it performs non-linear transformation and normalization on the features, enhancing the model's representation learning ability and enabling it to better adapt to diverse input patterns. The expression for the position-forward network is as follows:

[0073] ,

[0074] In the formula, This is the input matrix of the position feedforward network. and This is the weight matrix; and This is the bias matrix.

[0075] The Add&Layer Normalization module in Transformer combines the advantages of residual connections and layer normalization, enabling the output of each sub-layer to retain key information from the original input while stabilizing the feature distribution through normalization. The expression is as follows:

[0076] ,

[0077]

[0078] In the formula, The input matrix for Add&Layer Norm; This is the output of the current sublayer; for The mean; for The variance; , Learnable scaling and offset parameters; Use a small number, such as 0.001, to ensure numerical stability. Representation layer normalization. This represents the output of Add&LayerNorm.

[0079] In some specific implementations, the LoRA (Low-Rank Adaptation) method is used to adapt the pre-trained LLM, which is particularly suitable for small-sample scenarios. LoRA is an efficient fine-tuning strategy that freezes the pre-trained weights. The attention layer weights are then decomposed into a low-rank matrix to generate a parameter update matrix. .like Figure 7 As shown, From low-rank matrix The product structure introduces only a small number of trainable parameters, thereby significantly reducing memory and computational overhead, while preserving model knowledge and avoiding the destruction of its capabilities by parameter tuning.

[0080]

[0081]

[0082] In the formula, This is the new weight matrix; These are the initial weights for the pre-trained model; Incremental weights for specific tasks; A and B These are the matrix factorization factors.

[0083] In some specific embodiments, multiple fused samples are input into a large language model for pre-training, and the parameters of the large language model are adjusted, including a first stage and a second stage.

[0084] Before dividing the wind power data sequence into multiple time-series patches, the wind power data sequence is normalized.

[0085] In the first stage, the offset of each temporal patch is linearly projected using the hidden features of the Transformer, and the prediction formula is as follows: ,in, This represents the time series patch offset for the next time step in the time series predicted by the wind power prediction model. This represents the actual value at the corresponding time step. Loss represents the high-dimensional hidden features obtained by the Transformer. and The error between them. It should be noted that LoRA is only applied to the first stage of pre-training LLM, while other modules in the first stage do not involve LoRA, but instead adopt a full parameter training strategy.

[0086] In the second stage, the prediction patches are stitched together into a prediction vector through the Flatten operation, and the prediction vector is rearranged through the Rearrange operation to adapt to the structure of the time series. Then, the rearranged prediction vector is restored to the wind power data series prediction value by inverse normalization.

[0087] In the second stage, the model focuses on optimizing wind power forecasts for the entire time series. Specifically, the output hidden features of the Transformer are first concatenated into a single overall prediction vector through a Flatten operation (as shown in equation...). Subsequently, the Rearrange operation is applied to rearrange the flattened global prediction vector to fit the structure of the specific time series. Next, inverse normalization (RevIN) is used to restore the prediction results to the original data distribution. The expression for the above process is:

[0088] ,

[0089] ,

[0090] ,

[0091] The training objective is to minimize the prediction curve. Compared to the true curve Error between them:

[0092] In the formula, Output time series future S A complete real-world wind power sequence with a time step. This represents the single-point prediction result at step s; This represents the single-point true result at step s.

[0093] Step S3: Input the fused sample of new wind power data into the wind power prediction model, and the wind power prediction model outputs the wind power prediction value.

[0094] The method described in this embodiment provides discrete fusion samples suitable for input to a large language model. These fusion samples include wind power data, location codes that retain relative positional information over time, time codes reflecting dynamic features of the time dimension, and semantic instructions carrying high-level semantic information of the time series. The discrete fusion samples not only integrate local and global features of the time series but also simultaneously capture dynamic time information and semantic context, providing rich input expressions for subsequent prediction tasks and enabling the wind power prediction model to better learn time dependencies. This allows for high-precision wind power prediction by pre-training a large language model using fusion samples, even with limited historical operational data samples.

[0095] In one specific embodiment, an AutoDL deployment server was used to conduct wind power time series prediction research based on the Mistral 7B pre-trained language model. Mistral 7B has a 32-layer Transformer encoder and supports a context window of 8192 tokens. The experimental hardware configuration consisted of an Intel Core i9-10850K CPU (3.60GHz), 64GB of RAM, and an NVIDIA GeForce RTX 3090 (24GB VRAM). A two-stage fine-tuning strategy was employed: In the first stage, most of the Mistral 7B weights were frozen, and only the task-related LoRA adapter matrix (low-rank value set to 64) was updated, significantly reducing the parameter update range by inserting attention weight updates. Simultaneously, 4-bit quantization (QLoRA) and FP16 mixed-precision training were combined to improve training efficiency. In the second stage, an 8-bit AdamW optimizer was used, with an initial learning rate of 2e-5, gradually decreasing to 1e-6, and the learning rate was dynamically adjusted using a CosineAnnealingLR strategy.

[0096] To verify the effectiveness of the proposed prediction model, operational data collected from six actual wind farms in Tianjin, China, were selected for case studies and labeled WT1, WT2, WT3, WT4, WT5, and WT6. This data includes the actual power generation of each wind farm and wind speed meteorological information at different altitudes from January 1, 2020 to December 31, 2020. For model training and testing, the first 70% of the data was used for training, and the last 30% was used for testing. All data were obtained from the wind farm's SCADA system, with a sampling interval of 10 minutes. To align with the time granularity commonly found in practical engineering applications, the 10-minute interval data was converted to 1-hour data by averaging every 6 data points.

[0097] We selected models such as CNN, LSTM, GRU, PatchTST, FEDformer, Autoformer, Informer, and GPT4TS for comparison.

[0098] CNN: A model that extracts local features from time series data through convolutional and pooling layers, particularly adept at capturing patterns with significant trends and periodicity. Its local receptive field mechanism enables the model to efficiently handle short-term dependencies, while global pooling operations further enhance the model's robustness and generalization ability.

[0099] LSTM: A technique that utilizes unique gating mechanisms (such as input gate, forget gate, and output gate) to efficiently capture long-term dependency features in time series and successfully overcomes the gradient vanishing problem in traditional recurrent neural networks.

[0100] GRU: A lightweight variant of LSTM that efficiently captures long-term dependency features in time series data by leveraging its simplified gating mechanism (update gate and reset gate).

[0101] PatchTST: A Transformer network optimized for time series data. It divides the time series into multiple sub-sequence blocks as input and improves prediction performance through self-supervised learning. This design not only efficiently captures local features but also reduces the computational complexity of processing long-term series data.

[0102] FEDformer: A frequency-domain enhanced Transformer model that replaces the traditional self-attention mechanism with Fourier transform and wavelet transform modules to decompose time series signals into trend and periodic signals. The frequency-domain enhancement design enables the model to effectively capture the long-term structural information of time series.

[0103] Autoformer: A Transformer model that optimizes time series forecasting capabilities through trend and seasonal decomposition mechanisms. It also introduces an autocorrelation mechanism to replace part of the self-attention module, thereby efficiently capturing the dependencies between subsequences and further improving the model's forecasting performance.

[0104] Informer: A Transformer network designed for long-running sequences. It employs a sparse self-attention mechanism, which reduces computational complexity while retaining the ability to model global features. It can handle ultra-long sequence data and achieves a balance between performance and efficiency.

[0105] GPT4TS: A framework for applying large pre-trained language models to time series analysis. By freezing most of the model's layers (such as self-attention layers and FFN layers) and only fine-tuning the position embeddings and normalization layers, it retains the generalization knowledge of the original model and adapts it to time series tasks.

[0106] To ensure the robustness of the experimental results and reduce the interference of random factors, each model was run 5 times, and the average of the prediction results was taken as the final output. Subsequently, the prediction performance was comprehensively evaluated using mean absolute error (MAE), root mean square error (RMSE), and R².

[0107] The model was compared with a benchmark model on different training data ratios (100%, 10%, 5%) and different datasets. The input historical sequence length was 168, and the predicted future horizon was 24. Tables 1 to 3 summarize the MAE, RMSE, and R² metrics of each model on different wind farm datasets under different training data ratios.

[0108]

[0109]

[0110]

[0111] 1) 100% Data Scenario in Table 1: Under conditions of sufficient data, all models performed well, but PowerMistral still showed a significant advantage in prediction accuracy and fitting ability. For example, in the WT1 dataset, PowerMistral's MAE and RMSE were 1.88 MW and 3.11 MW, respectively, significantly better than the average values ​​of GPT4TS (2.63 MW and 3.32 MW) and Transformer models (such as PatchTST, FEDformer, and Informer) (3.1 MW and 3.9 MW). Although traditional models (CNN, LSTM, GRU) showed some improvement, their MAE was still as high as 6.46 MW, and their RMSE was close to 8 MW. PowerMistral's R² exceeded 0.95 in all datasets, while GPT4TS was 0.94 and traditional models were only 0.87, indicating that PowerMistral has stronger prediction accuracy and pattern capture ability in big data scenarios.

[0112] 2) 10% Data Scenario in Table 2: When the data volume is reduced to 10%, PowerMistral's robustness becomes even more pronounced. For example, in the WT3 dataset, PowerMistral's MAE is 5.41 MW, which is about 12% and 33% lower than GPT4TS (6.12 MW) and the Transformer model (8.13 MW), respectively, and nearly 60% lower than the traditional model (MAE exceeding 13 MW). In terms of RMSE, PowerMistral is 6.48 MW, which is 10% and 30% lower than GPT4TS (7.17 MW) and the Transformer model (9 MW), respectively, while the traditional model is over 12 MW. PowerMistral's R² value in this scenario is still higher than 0.88, which is better than GPT4TS (0.85), the Transformer model (0.81), and the traditional model (0.65), fully demonstrating its fitting ability and generalization performance under small sample conditions.

[0113] 3) 5% Data Scenarios in Table 3: When the data volume is further reduced to 5%, the R² value of traditional models (CNN, LSTM, GRU) drops to negative, indicating severe underfitting. While the R² of the Transformer model is positive, it is only around 0.53, resulting in a significant drop in prediction performance. In contrast, PowerMistral's R² remains above 0.84, demonstrating excellent robustness. Taking the WT3 dataset as an example, PowerMistral's MAE and RMSE are 7.58 MW and 9.48 MW respectively, significantly outperforming GPT4TS (7.82 MW and 9.82 MW) and the Transformer model (MAE exceeding 12 MW, RMSE exceeding 14 MW), while traditional models reach as high as 23 MW and 29 MW. PowerMistral reduces the error by approximately 33% compared to the Transformer model and by approximately 67% compared to traditional models, demonstrating a significant advantage in extremely small sample scenarios.

[0114] In summary, PowerMistral, with its superior performance under both full-data and small-sample conditions, is the optimal model for wind farm power prediction among the compared models. Compared to GPT4TS and mainstream Transformer models, PowerMistral achieves comprehensive superiority in error control, fitting ability, and small-sample handling, demonstrating its unique advantages and potential in the field of time series forecasting.

[0115] Comprehensive ablation experiments were conducted, and a series of model variants were designed, including those removing the temporal embedding (E1), trend prompt embedding (E2), and numerical prompt embedding (E3), as well as models that skipped the time alignment phase (E4) and prediction fine-tuning phase (E5), respectively. Multi-dataset tests at 100% and 5% data scales demonstrated the impact of each component and training phase on model performance. The R² performance of the models is shown below. Figure 8 and Figure 9 As shown.

[0116] In the ablation experiment, the R² of the wind power prediction model PowerMistral proposed in this invention was 0.9537-0.9645 and 0.8240-0.8429 in the 100% and 5% data scenarios, respectively. In the 100% data scenario ( Figure 8The removal of each module has a relatively small impact on performance. For example, on the WT5 dataset, the R² of the complete model is 0.9541, which drops to 0.9109 (a decrease of 4.53%) after removing the time embedding (E1); after removing the trend prompt (E2) and numerical prompt (E3), the R² drops to 0.9134 (a decrease of 4.27%) and 0.9185 (a decrease of 3.73%), respectively. After removing the time alignment stage (E4), the R² drops to 0.9248 (a decrease of 3.07%), while removing the prediction fine-tuning stage (E5) has the greatest impact on performance, with the R² dropping to 0.8832 (a decrease of 7.43%). The analysis shows that even with sufficient data, each module still plays an important role in improving model performance, especially the prediction fine-tuning stage (E5), which is crucial for adaptability to complex tasks.

[0117] In the 5% data scenario ( Figure 9 The impact of each module on performance is significantly enhanced. For example, on the WT2 dataset, the R² of the PowerMistral wind power prediction model is 0.8240. After removing the time embedding (E1), it drops to 0.7131 (a decrease of 13.46%). After removing the trend prompt (E2) and numerical prompt (E3), the R² drops to 0.7189 (a decrease of 12.75%) and 0.7488 (a decrease of 9.10%), respectively. Skipping the time alignment stage (E4) reduces the R² to 0.6997 (a decrease of 15.08%), while removing the prediction fine-tuning stage (E5) causes the performance to drop to 0.6573 (a decrease of 20.22%). On the WT4 dataset, removing the time embedding (E1) reduces the R² from 0.8423 to 0.7298, similar to the impact of the time alignment stage (E4). This indicates that the time embedding is crucial for the model to understand temporal characteristics and periodic patterns.

[0118] Further analysis reveals that each module plays a unique and irreplaceable role in performance improvement. In small-sample scenarios, temporal embedding (E1) enhances the model's ability to capture periodic patterns by explicitly encoding temporal characteristics; cueing modules (E2, E3) improve the model's ability to perceive trends and numerical statistical characteristics; the time alignment stage (E4) maintains the consistency of time series data, providing stable support for model training and prediction; and the prediction fine-tuning stage (E5), as a core module, plays a crucial role in adapting to small-sample distributions and improving generalization ability.

[0119] The ablation experiments described above demonstrate the importance of each module under different data scales, proving the rationality and complementarity of the module design. The absence of time embedding, the cueing module, the time alignment stage, or the prediction fine-tuning stage all significantly reduce model performance, further validating the model's efficiency and robustness in time series prediction tasks.

[0120] Sensitivity analysis was performed on the PowerMistral wind power prediction model to assess its performance under different prediction horizons (24, 96, 168). The results are as follows: Figure 10 As shown. By Figure 10 As can be seen, the model performs excellently in short time step predictions (168 / 24), with R² values ​​remaining above 0.95 for all datasets, demonstrating the model's accurate fitting ability to short-term time series features. With increasing prediction time steps (168 / 96 and 168 / 168), although the R² value decreases slightly, the model still maintains high prediction accuracy on most datasets. Especially on the WT1 and WT4 datasets, the model's R² is almost unaffected by the increase in prediction time steps, demonstrating its robustness and superior performance in handling stable datasets.

[0121] Compare the impact of different low-rank values ​​on model prediction performance (e.g.) Figure 11 (As shown). The low-rank value is a key hyperparameter in the LoRA method, determining the rank of the adapter matrix inserted into the pre-trained model's weight matrix during fine-tuning. Figure 11 It can be seen that when the low-rank value is 16, the model's R² is slightly lower, decreasing by approximately 0.5%-1.0% compared to a low-rank value of 64. At a low-rank value of 64, the R² reaches its peak for all datasets, demonstrating the best performance balance. However, when the low-rank value increases to 256, the prediction performance of some datasets (such as WT4, WT5, and WT6) decreases slightly, by approximately 0.2%-0.5% compared to a low-rank value of 64. This may be because excessively high low-rank values ​​lead to information redundancy, thus affecting the model's generalization ability. Overall, although the impact of low-rank value changes on model performance is limited, choosing a moderate low-rank value (such as 64) can achieve a better balance between computational resources and performance requirements, making it a more robust configuration.

[0122] This paper proposes a Few-Shot wind power prediction framework—PowerMistral—based on a pre-trained large language model. Developed using the Mistral-7B small-to-medium-sized pre-trained large language model, this framework efficiently completes wind power prediction tasks without requiring training from scratch. The core method includes a two-stage fine-tuning strategy: the first stage focuses on the temporal characteristics of wind power data, extracting key time-series features through an adaptation mechanism; the second stage fine-tunes the model for the specific prediction task, further optimizing prediction performance. Simultaneously, PowerMistral encodes the temporal features of the wind power sequence and combines them with tailored numerical and trend feature text, effectively guiding the model to improve its predictive capabilities. The main conclusions are as follows:

[0123] 1) PowerMistral demonstrates its unique advantages and potential in the field of time series forecasting. Whether in full-data or small-sample scenarios, PowerMistral outperforms the benchmark model in MAE, RMSE, and R2 across six wind power datasets. Particularly in small-sample scenarios with only 5% of the data, PowerMistral reduces error by approximately 33% compared to the Transformer model and by approximately 67% compared to traditional models.

[0124] 2) Ablation experiments validated the design rationality of each module in the PowerMistral framework. The temporal embedding module effectively captures periodic features, while the trend and numerical prompts enhance the ability to perceive trends and values. The prediction fine-tuning stage improves generalization performance in small sample scenarios. Experimental results show that the synergistic effect of each module is the key to performance improvement.

[0125] 3) PowerMistral exhibits extremely high robustness across different prediction sequence step sizes, with R² consistently remaining above 0.88 and only showing a slight decrease. Furthermore, selecting a moderately low rank value can achieve a better balance between computational resources and performance requirements.

[0126] This embodiment proposes a Few-Shot wind power prediction model based on a pre-trained large language model—PowerMistral. This model is developed based on the Mistral-7B small-to-medium-sized pre-trained large language model, enabling efficient wind power prediction tasks without requiring training from scratch. The core method includes a two-stage fine-tuning strategy: the first stage focuses on the temporal characteristics of wind power data, extracting key time-series features through an adaptation mechanism; the second stage fine-tunes the model for the specific prediction task, further optimizing prediction performance. Simultaneously, PowerMistral encodes the temporal features of the wind power sequence and combines them with tailored numerical and trend feature text, effectively guiding the model to improve its prediction capabilities. The main technical advantages of PowerMistral include:

[0127] 1) PowerMistral demonstrates its unique advantages and potential in the field of time series forecasting. Whether in full-data or small-sample scenarios, PowerMistral outperforms the benchmark model in MAE, RMSE, and R² across six wind power datasets. Particularly in small-sample scenarios with only 5% of the data, PowerMistral reduces error by approximately 33% compared to the Transformer model and by approximately 67% compared to traditional models.

[0128] 2) Ablation experiments validated the design rationality of each module in the PowerMistral framework. The temporal embedding module effectively captures periodic features, while the trend and numerical prompts enhance the ability to perceive trends and values. The prediction fine-tuning stage improves generalization performance in small sample scenarios. Experimental results show that the synergistic effect of each module is the key to performance improvement.

[0129] 3) PowerMistral exhibits extremely high robustness across different prediction sequence step sizes, with R² consistently remaining above 0.88 and only showing a slight decrease. Furthermore, selecting a moderately low rank value can achieve a better balance between computational resources and performance requirements.

[0130] Those skilled in the art can change the above order without departing from the scope of protection of this invention.

[0131] Another embodiment of the present invention provides a wind power prediction device based on a pre-trained large language model, comprising:

[0132] The data fusion module is used to divide the wind power data sequence into multiple time-series patches; determine the position code corresponding to the time-series patch based on the sequence position of the wind power data in the time-series patch; determine the time code corresponding to the time-series patch based on the timestamp of the wind power data in the time-series patch; divide the time-series patch into multiple equal-length subsequences; extract the data features of each equal-length subsequence; convert the data features into semantic prompts that can be understood by a large language model; add text instructions to the semantic prompts to obtain semantic instructions; and concatenate the time-series patch and its corresponding position code, time code, and semantic instructions to obtain the fusion sample corresponding to the time-series patch.

[0133] The model optimization module is used to input multiple fused samples into a large language model for pre-training and to adjust the parameters of the large language model to obtain a wind power prediction model.

[0134] The power prediction module is used to input the fused sample of new wind power data into the wind power prediction model, and the wind power prediction model outputs the wind power prediction value.

[0135] Regarding the device in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0136] The aforementioned wind power prediction equipment based on a pre-trained large language model can provide discrete fusion samples suitable for input to the large language model. These fusion samples include wind power data, location codes that retain relative positional information over time, time codes reflecting dynamic features of the time dimension, and semantic instructions carrying high-level semantic information of the time series. The discrete fusion samples not only integrate local and global features of the time series but also simultaneously capture dynamic time information and semantic context, providing rich input expressions for subsequent prediction tasks and enabling the wind power prediction model to better learn time dependencies. Even with limited historical operational data samples, the fusion sample-based pre-trained large language model can achieve high-accuracy wind power prediction.

[0137] Based on the same inventive concept, this invention also provides a method for constructing a wind power prediction model based on a pre-trained large language model, comprising the following steps:

[0138] The wind power data sequence is divided into multiple time-series patches; the position code corresponding to the time-series patch is determined based on the sequence position of the wind power data in the time-series patch; the time code corresponding to the time-series patch is determined based on the timestamp of the wind power data in the time-series patch; the time-series patch is divided into multiple equal-length subsequences; the data features of each equal-length subsequence are extracted; the data features are converted into semantic prompts that can be understood by a large language model; text instructions are added to the semantic prompts to obtain semantic instructions; the time-series patch and its corresponding position code, time code, and semantic instructions are concatenated to obtain the fusion sample corresponding to the time-series patch; multiple fusion samples are input into the large language model for pre-training, and the parameters of the large language model are adjusted to obtain the wind power prediction model.

[0139] The aforementioned method for constructing a wind power prediction model based on a pre-trained large language model can provide discrete fusion samples suitable for input to the large language model. These fusion samples include wind power data, location codes that retain relative positional information over time, time codes reflecting dynamic features of the time dimension, and semantic instructions carrying high-level semantic information of the time series. The discrete fusion samples not only integrate local and global features of the time series but also simultaneously capture dynamic time information and semantic context, providing rich input expressions for subsequent prediction tasks and enabling the wind power prediction model to better learn time dependencies. The wind power prediction model constructed using this method can achieve high-accuracy wind power prediction even with limited historical operational data samples.

[0140] Based on the same inventive concept, this invention also provides a wind power prediction model based on a pre-trained large language model, comprising:

[0141] The sequence discretization module is used to divide the wind power data sequence into multiple time-series patches; determine the position code corresponding to the time-series patch based on the sequence position of the wind power data in the time-series patch; determine the time code corresponding to the time-series patch based on the timestamp of the wind power data in the time-series patch; divide the time-series patch into multiple equal-length subsequences; extract the data features of each equal-length subsequence; convert the data features into semantic prompts that can be understood by a large language model; add text instructions to the semantic prompts to obtain semantic instructions; and concatenate the time-series patch and its corresponding position code, time code, and semantic instructions to obtain the fusion sample corresponding to the time-series patch.

[0142] The pre-training and parameter tuning module is used to input multiple fused samples into the large language model for pre-training and to adjust the parameters of the large language model to obtain the wind power prediction model.

[0143] The wind power prediction model based on the pre-trained large language model described above can provide discrete fusion samples suitable for input to the large language model. These fusion samples include wind power data, location codes that retain relative positional information over time, time codes reflecting dynamic features of the time dimension, and semantic instructions carrying high-level semantic information of the time series. The discrete fusion samples not only integrate local and global features of the time series but also simultaneously capture dynamic time information and semantic context, providing rich input expressions for subsequent prediction tasks and enabling the wind power prediction model to better learn time dependencies. The wind power prediction model constructed using this method can achieve high-accuracy wind power prediction even with limited historical operating data samples.

[0144] Based on the same inventive concept, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the aforementioned wind power prediction method based on a pre-trained large language model, or implements the aforementioned wind power prediction model construction method based on a pre-trained large language model.

[0145] Based on the same inventive concept, the present invention also provides a computer storage medium storing computer-executable instructions, which, when executed, implement the aforementioned wind power prediction method based on a pre-trained large language model, or implement the aforementioned wind power prediction model construction method based on a pre-trained large language model.

[0146] Any modifications, additions, and equivalent substitutions made within the scope of the principles of this invention shall still fall within the patent coverage of this invention.

Claims

1. A wind power prediction method based on a pre-trained large language model, characterized in that, Includes the following steps: The wind power data sequence is divided into multiple time-series patches; Based on the sequence position of wind power data in the time-series patch, the position encoding corresponding to the time-series patch is determined, specifically including: determining the position embedding matrix. ; pass Determine the position code corresponding to the timing patch. Indicates timing patch i The location encoding, where pos represents the sequence location index of the wind power data in the time-series patch. i Indicates the time-series patch index. Indicates the length of the timing patch; Based on the timestamps of the wind power data in the time series patch, the time code corresponding to the time series patch is determined, specifically including: performing a first linear transformation on the timestamps of the wind power data in the time series patch to obtain the time data after dimensionality reduction; The time data is normalized and regularized, and then convolutional processing is performed to obtain the time data features; The time data features are subjected to a second linear transformation to obtain the time code corresponding to the time series patch. The time series patch is divided into multiple equal-length subsequences, specifically including: the normalized time series patch as a vector. It indicates, and further divides into I Non-overlapping subsequences of equal length ,in, ; Extracting data features from each equal-length subsequence and converting these features into semantic prompts that a large language model can understand, specifically includes: for each small data segment... Local features are extracted and transformed into semantic information that LLM can understand, including data trends. and numerical description Data trends The semantic information range includes rise, fall, plateau, and shutdown, and the numerical description The semantic information range includes the maximum, minimum, average, and variance; the attention mechanism in the Transformer library is invoked to apply the time-series patches respectively. i Data trends With numerical description Transformed into semantic cue words that can be understood by a large language model, among which, , , Indicates timing patch i Normalized vector, Indicates to Extracting data trend features Indicates to Perform numerical descriptive feature extraction. Indicates to Extract the maximum value. Indicates to Extract the minimum value. Indicates to Extract the average value. Indicates to Perform variance extraction; Add text instructions to the semantic prompt words to obtain semantic instructions; The timing patch and its corresponding position code, time code and semantic instructions are concatenated to obtain the fusion sample corresponding to the timing patch; Multiple fused samples are input into a large language model for pre-training, and the parameters of the large language model are adjusted to obtain a wind power prediction model. The fused sample of new wind power data is input into the wind power prediction model, and the wind power prediction model outputs the predicted wind power value.

2. The method as described in claim 1, characterized in that, The wind power data sequence is divided into multiple time-series patches, including the following steps: The wind power data sequence is divided into N time series patches using a non-overlapping method, where N = [(L - P) / S] + 2, L represents the length of the wind power data sequence, P represents the length of the time series patch, and S represents the horizontal sliding step size on the wind power data sequence.

3. The method as described in claim 1, characterized in that, The timing patch and its corresponding positional encoding, time encoding, and semantic instructions are concatenated to obtain the fused sample corresponding to the timing patch, including the following steps: pass Obtain timing patch i The corresponding fusion samples, among which, Indicates timing patch i Fusion samples, This indicates a splicing operation. Indicates timing patch i Basic embedding encoding, Indicates timing patch i Location encoding, Indicates timing patch i Time encoding, Indicates timing patch i Semantic instructions for data trends Indicates timing patch i The semantic instructions described by numbers.

4. The method as described in claim 1, characterized in that, Multiple fused samples are input into a large language model for pre-training, and the parameters of the large language model are tuned, including the first and second stages. Before dividing the wind power data sequence into multiple time-series patches, the wind power data sequence is normalized. In the first stage, the offset of each temporal patch is linearly projected using the hidden features of the Transformer, and the prediction formula is as follows: ,in, This represents the time series patch offset for the next time step in the time series predicted by the wind power prediction model. This represents the actual value at the corresponding time step. Loss represents the high-dimensional hidden features obtained by the Transformer. and The error between; In the second stage, the prediction patches are stitched together into a prediction vector through the Flatten operation, and the prediction vector is rearranged through the Rearrange operation to adapt to the structure of the time series. Then, the rearranged prediction vector is restored to the wind power data series prediction value by inverse normalization.

5. A wind power prediction device based on a pre-trained large language model, characterized in that, include: The data fusion module is used to divide the wind power data sequence into multiple time-series patches; Determine the position embedding matrix ; pass Determine the position code corresponding to the timing patch. Indicates timing patch i The location encoding, where pos represents the sequence location index of the wind power data in the time-series patch. i Indicates the time-series patch index. The time series patch length is represented; the timestamps of the wind power data in the time series patch are subjected to the first linear transformation to obtain the dimensionality-reduced time data; the time data is normalized and regularized and then convolutional to obtain the time data features; The time data features are subjected to a second linear transformation to obtain the time code corresponding to the time series patch. The time series patch is divided into multiple equal-length subsequences, specifically including: the normalized time series patch as a vector. It indicates, and further divides into I Non-overlapping subsequences of equal length ,in, Extracting data features from each equal-length subsequence and converting these features into semantic prompts that a large language model can understand, specifically includes: for each small data segment... Local features are extracted and transformed into semantic information that LLM can understand, including data trends. and numerical description Data trends The semantic information range includes rise, fall, plateau, and shutdown, and the numerical description The semantic information range includes the maximum, minimum, average, and variance; the attention mechanism in the Transformer library is invoked to apply the time-series patches respectively. i Data trends With numerical description Transformed into semantic cue words that can be understood by a large language model, among which, , , Indicates timing patch i Normalized vector, Indicates to Extracting data trend features Indicates to Perform numerical descriptive feature extraction. Indicates to Extract the maximum value. Indicates to Extract the minimum value. Indicates to Extract the average value. Indicates to Variance extraction is performed; text instructions are added to the semantic prompts to obtain semantic instructions; the temporal patch and its corresponding positional code, time code and semantic instructions are concatenated to obtain the fusion sample corresponding to the temporal patch; The model optimization module is used to input multiple fused samples into a large language model for pre-training and to adjust the parameters of the large language model to obtain a wind power prediction model. The power prediction module is used to input the fused sample of new wind power data into the wind power prediction model, and the wind power prediction model outputs the wind power prediction value.

6. A method for constructing a wind power prediction model based on a pre-trained large language model, characterized in that, Includes the following steps: The wind power data sequence is divided into multiple time-series patches; Determine the position embedding matrix ; pass Determine the position code corresponding to the timing patch. Indicates timing patch i The location encoding, where pos represents the sequence location index of the wind power data in the time-series patch. i Indicates the time-series patch index. Indicates the length of the timing patch; The timestamps of the wind power data in the time series patch are subjected to the first linear transformation to obtain the time data after dimensionality reduction. The time data is normalized and regularized, and then convolutional processing is performed to obtain the time data features; The time data features are subjected to a second linear transformation to obtain the time code corresponding to the time series patch. The time series patch is divided into multiple equal-length subsequences, specifically including: the normalized time series patch as a vector. It indicates, and further divides into I Non-overlapping subsequences of equal length ,in, ; Extracting data features from each equal-length subsequence and converting these features into semantic prompts that a large language model can understand, specifically includes: for each small data segment... Local features are extracted and transformed into semantic information that LLM can understand, including data trends. and numerical description Data trends The semantic information range includes rise, fall, plateau, and shutdown, and the numerical description The semantic information range includes the maximum, minimum, average, and variance; the attention mechanism in the Transformer library is invoked to apply the time-series patches respectively. i Data trends With numerical description Transformed into semantic cue words that can be understood by a large language model, among which, , , Indicates timing patch i Normalized vector, Indicates to Extracting data trend features Indicates to Perform numerical descriptive feature extraction. Indicates to Extract the maximum value. Indicates to Extract the minimum value. Indicates to Extract the average value. Indicates to Perform variance extraction; Add text instructions to the semantic prompt words to obtain semantic instructions; The timing patch and its corresponding position code, time code and semantic instructions are concatenated to obtain the fusion sample corresponding to the timing patch; Multiple fused samples are input into a large language model for pre-training, and the parameters of the large language model are adjusted to obtain a wind power prediction model.

7. A wind power prediction model based on a pre-trained large language model, characterized in that, include: The sequence discretization module is used to divide the wind power data sequence into multiple time-series patches; Determine the position embedding matrix ; pass Determine the position code corresponding to the timing patch. Indicates timing patch i The location encoding, where pos represents the sequence location index of the wind power data in the time-series patch. i Indicates the time-series patch index. The time series patch length is represented; the timestamps of the wind power data in the time series patch are subjected to the first linear transformation to obtain the dimensionality-reduced time data; the time data is normalized and regularized and then convolutional to obtain the time data features; The time data features are subjected to a second linear transformation to obtain the time code corresponding to the time series patch. The time series patch is divided into multiple equal-length subsequences, specifically including: the normalized time series patch as a vector. It indicates, and further divides into I Non-overlapping subsequences of equal length ,in, Extracting data features from each equal-length subsequence and converting these features into semantic prompts that a large language model can understand, specifically includes: for each small data segment... Local features are extracted and transformed into semantic information that LLM can understand, including data trends. and numerical description Data trends The semantic information range includes rise, fall, plateau, and shutdown, and the numerical description The semantic information range includes the maximum, minimum, average, and variance; the attention mechanism in the Transformer library is invoked to apply the time-series patches respectively. i Data trends With numerical description Transformed into semantic cue words that can be understood by a large language model, among which, , , Indicates timing patch i Normalized vector, Indicates to Extracting data trend features Indicates to Perform numerical descriptive feature extraction. Indicates to Extract the maximum value. Indicates to Extract the minimum value. Indicates to Extract the average value. Indicates to Variance extraction is performed; text instructions are added to the semantic prompts to obtain semantic instructions; the temporal patch and its corresponding positional code, time code and semantic instructions are concatenated to obtain the fusion sample corresponding to the temporal patch; The pre-training and parameter tuning module is used to input multiple fused samples into the large language model for pre-training and to adjust the parameters of the large language model to obtain the wind power prediction model.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the wind power prediction method based on a pre-trained large language model as described in any one of claims 1 to 4, or implements the wind power prediction model construction method based on a pre-trained large language model as described in claim 6.

9. A computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which, when executed, implement the wind power prediction method based on a pre-trained large language model as described in any one of claims 1 to 4, or implement the wind power prediction model construction method based on a pre-trained large language model as described in claim 6.

Images