Method for performing wind power generation prediction by fine tuning pre-training large model

Abstract

The invention discloses a method for performing wind power generation prediction through a fine-tuning pre-training large model, and the method comprises the steps: obtaining a historical multi-dimensional time series data set, and carrying out the spatial-temporal feature fusion through a sliding window, and generating a wind speed trend sequence; based on the wind speed gradients of the adjacent time windows, generating step length adjustment parameters by using a normalization function; performing dynamic parameter adjustment on the pre-trained time sequence prediction model based on the step length adjustment parameters, and performing domain adaptation on top network parameters of the time sequence prediction model through an adaptive optimization algorithm in combination with a hierarchical transfer learning strategy to obtain an optimized time sequence prediction model; and processing the real-time multi-dimensional time sequence data set by using the optimized model to generate a wind power generation prediction result. By dynamically adjusting model parameters and a hierarchical transfer learning strategy, the precision of wind power generation prediction and the model adaptation capability are effectively improved, and the method is suitable for time sequence prediction scenes in the field of wind power generation.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence application and new energy utilization, and in particular relates to a method for fine-tuning a pre-trained large model to perform wind power generation prediction. Background Art

[0002] With the development of intelligent technologies in the wind power generation sector, data-driven time series forecasting has become a key approach to improving power generation efficiency and grid stability. By modeling the spatiotemporal correlation between historical meteorological data and power generation, these methods can effectively adapt to the nonlinear dynamic characteristics of wind power systems. Transfer learning methods based on pre-trained models have become a research hotspot due to their ability to reuse common time series feature representations and reduce reliance on domain data. Traditionally, wind power generation forecasting relies primarily on statistical models (such as ARIMA and SARIMA), shallow machine learning methods (such as support vector machines (SVMs), and early deep learning models (such as LSTMs and TCNs). These methods extract multidimensional time series features such as wind speed, direction, and temperature using a fixed window, and optimize model parameters using a gradient descent algorithm with a static learning rate. With the application of large pre-trained models in time series forecasting, some approaches have attempted to improve model generalization through transfer learning strategies that freeze the underlying network and fine-tune the top-level parameters, but these approaches still rely on fixed-step parameter updates. Current wind power prediction methods have significant defects when dynamically adapting to complex meteorological environments: on the one hand, traditional fixed-step optimization strategies are difficult to match the non-stationary change characteristics of wind speed sequences. When the wind speed gradient fluctuates violently, it is easy to cause an imbalance between the model convergence speed and prediction accuracy; on the other hand, the existing transfer learning framework lacks multi-stage progressive optimization of the domain adaptation process of the top-level network parameters, and does not effectively constrain the parameter distribution consistency of the pre-trained model and the domain model, making it difficult for the model to capture refined wind speed trend changes when processing real-time multi-dimensional spatiotemporal feature fusion, ultimately leading to insufficient long-term prediction accuracy and weak model robustness. Summary of the Invention

[0003] Based on this, it is necessary to provide a method for fine-tuning a pre-trained large model for wind power generation prediction that can solve the above problems.

[0004] In a first aspect, the present application provides a method for fine-tuning a pre-trained large model for wind power generation prediction, comprising:

[0005] Obtain a historical multidimensional time series dataset and generate a wind speed trend series by fusing spatiotemporal features through a sliding window.

[0006] Based on the wind speed gradients of adjacent time windows in the wind speed trend sequence, a normalization function is used to generate the step size adjustment parameter;

[0007] Dynamically adjust the parameters of the pre-trained time series prediction model based on the step size adjustment parameter, and combine it with the layered transfer learning strategy to perform domain adaptation on the top-level network parameters of the time series prediction model through the adaptive optimization algorithm to obtain the optimized time series prediction model;

[0008] Obtain real-time multi-dimensional time series data sets and use the optimized time series forecasting model to generate wind power generation forecast results.

[0009] In one embodiment, a step size adjustment parameter is generated using a normalization function based on wind speed gradients in adjacent time windows in a wind speed trend sequence, specifically including:

[0010] Calculate the wind speed mean vector of adjacent time windows and obtain the vector sequence {V t} and {V t+1}, where t represents the time window index;

[0011] Based on the vector sequence {V t} and {V t+1}, and construct the wind speed gradient matrix ΔV=||V by differential operation while retaining the gradient direction feature t+1 -V t ||2;

[0012] Based on the wind speed gradient matrix ΔV, a dynamic normalization function F= Where V max It represents the maximum wind speed change within the sliding window period, and α represents the smoothing coefficient;

[0013] Linearly combine the dynamic normalization function F with the preset basic step size to generate the step size adjustment parameter Step size =β·(1+F), where β represents the preset basic step size.

[0014] In one embodiment, dynamically adjusting parameters of a pre-trained time series prediction model based on a step size adjustment parameter includes:

[0015] Based on the step size adjustment parameters, the corresponding dynamic learning rate parameter set is generated through the parameter mapping function, and the parameter mapping function satisfies: A t =λ1·tanh(λ2·Step size )+λ3, where A t Indicates the dynamic learning rate at time t, Step size represents the step size adjustment parameter, λ1, λ2, and λ3 represent the adjustable coefficients initialized by transfer learning;

[0016] The convolution kernel weights of the convolutional neural network layer are updated with gradients according to the dynamic learning rate parameter set. The update formula used is: Among them, Wt represents the weight matrix of the t-th iteration, represents the gradient of the loss function, ρ represents the weight attenuation coefficient, and ⊙ represents the Hadamard product;

[0017] The momentum of the hidden state transfer matrix of the recurrent neural network layer is updated by the sliding average algorithm, and the update amount calculation satisfies: Where μ represents the momentum factor, ΔH t Represents the matrix update amount at time t.

[0018] In one embodiment, a hierarchical transfer learning strategy is combined with an adaptive optimization algorithm to perform domain adaptation on the top-level network parameters of the time series prediction model to obtain an optimized time series prediction model, including:

[0019] The network structure of the time series prediction model is divided into the bottom feature extraction layer and the top prediction adaptation layer according to the hierarchical depth;

[0020] Freeze the network parameters of the bottom feature extraction layer, and dynamically optimize the fully connected parameters of the top prediction adaptation layer through the domain adaptation loss function to obtain the domain adaptation meta-parameter set;

[0021] An adaptive optimizer based on meta-learning strategy is used to perform multi-stage progressive fine-tuning on the domain adaptation meta-parameter set to obtain the optimized time series prediction model. The multi-stage progressive fine-tuning includes a multi-step optimization process with at least two parameter adjustment stages. In the first stage, the parameters are initialized by fitting the distribution of historical wind speed data. In the second stage, online gradient correction is performed through real-time data stream. The KL divergence constraint term is used to maintain the parameter distribution consistency between the pre-trained model and the domain adaptation model during the transfer learning process.

[0022] In one embodiment, the network structure of the time series prediction model is divided into a bottom feature extraction layer and a top prediction adaptation layer according to the hierarchical depth, including:

[0023] The convolutional layers and recurrent neural network encoders in the time series prediction model are divided into the bottom feature extraction layers. The last k fully connected layers in the pre-trained time series model are reconstructed into a domain adaptation prediction head. Learnable domain embedding vectors are introduced to form the top prediction adaptation layer. The level of the fully connected layer group is dynamically determined by the following formula: Among them, D in Denotes the input feature dimension, D out Denotes the output prediction dimension, D hidden Represents the baseline dimension of the hidden layer of the pre-trained model.

[0024] In one embodiment, the adaptive optimizer adopts an improved Adam algorithm based on dynamic weighted averaging, and its parameter update formula satisfies:

[0025]

[0026] in, Represents the dynamic basic learning rate, Ω represents the learning rate scaling factor, β represents the preset basic step size, Step size represents the step size adjustment parameter, z1 represents the first-order moment decay rate, γ represents the parameter regularization coefficient, g t represents the gradient at time t, and They represent the bias-corrected first-order and second-order moment estimates, and ε represents a numerical stability constant.

[0027] In one embodiment, the domain adaptation loss function adopts the following multi-objective joint optimization formula:

[0028] L total =I1L pred +I2L KL +I3L smooth

[0029] Among them, L pred represents the mean square error loss of wind speed prediction, L KL Represents the KL divergence constraint term of the parameter distribution of the pre-trained model and the adapted model, L smooth represents the temporal smoothing constraint term based on the second-order derivative of adjacent prediction results, and I1, I2, and I3 represent learnable adaptive weighting coefficients.

[0030] In a second aspect, the present application further provides a device for fine-tuning a pre-trained large model for wind power generation prediction, characterized in that the device comprises:

[0031] The data preprocessing module is used to obtain historical multidimensional time series data sets and generate wind speed trend series by fusing spatiotemporal features through sliding windows;

[0032] A step size parameter generation module is used to generate a step size adjustment parameter using a normalization function based on the wind speed gradients of adjacent time windows in the wind speed trend sequence;

[0033] The model optimization module is used to dynamically adjust the parameters of the pre-trained time series prediction model based on the step size adjustment parameters. In combination with the layered transfer learning strategy, the top-level network parameters of the time series prediction model are domain-adapted through the adaptive optimization algorithm to obtain the optimized time series prediction model.

[0034] The real-time prediction module is used to obtain real-time multi-dimensional time series data sets and generate wind power generation prediction results using the optimized time series prediction model.

[0035] In a third aspect, the present application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the above method of fine-tuning a pre-trained large model for wind power generation prediction.

[0036] The present invention provides a method for fine-tuning a pre-trained large model for wind power generation prediction, which effectively captures the spatiotemporal correlation characteristics of multidimensional data in wind power generation scenarios by acquiring a historical multidimensional time series data set and using a sliding window to fuse spatiotemporal features to generate a wind speed trend sequence; a dynamic normalization function is constructed based on the wind speed gradient of adjacent time windows to generate a step adjustment parameter, which can adaptively adjust the step size in the optimization process according to the non-stationary characteristics of wind speed changes, solving the problem of imbalance between model convergence speed and prediction accuracy caused by the traditional fixed step size strategy when the wind speed fluctuates violently; dynamic parameter adjustment is implemented on the pre-trained time series prediction model through the step size adjustment parameter, and the step size adjustment is combined with the hierarchical transfer learning strategy to freeze the wind speed trend sequence. The bottom feature extraction layer is represented by reusing the universal time series features of the pre-trained model, and an adaptive optimization algorithm is used for domain adaptation for the top prediction adaptation layer. At the same time, the parameter distribution consistency of the pre-trained model and the domain adaptation model is maintained through the KL divergence constraint term, reducing the dependence on a large amount of domain data, and strengthening the model's ability to fit the specific spatiotemporal characteristics of the wind power generation field. Combined with multi-stage progressive fine-tuning and time series smoothing constraints, the model's ability to capture the refined wind speed trend changes in real-time multi-dimensional spatiotemporal characteristics is improved, thereby improving the long-term accuracy and dynamic environment robustness of wind power generation predictions, and solving the problems of insufficient ability to dynamically adapt to complex meteorological environments and lack of effective optimization of the domain adaptation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 A flowchart of a method for fine-tuning a pre-trained large model to predict wind power generation according to the present invention;

[0039] Figure 2 This is a schematic diagram of the structure of a device for fine-tuning a pre-trained large model to predict wind power generation according to the present invention;

[0040] Figure 3 A schematic structural diagram of a device for fine-tuning a pre-trained large model for wind power generation prediction in one embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0042] The method provided in this application for fine-tuning a pre-trained large model for wind power generation prediction is applicable to a hardware environment including terminal equipment, a server, and a data storage system. The terminal collects wind power generation data and transmits it to the server. The server uses the pre-trained model to process and predict the data, and the results are fed back to the terminal. The data storage system is used to store relevant data and model parameters to support efficient prediction and management of wind power generation.

[0043] In one embodiment, Figure 1 As shown, the method includes the following steps:

[0044] S101, obtain a historical multidimensional time series data set, and use a sliding window to fuse spatiotemporal features to generate a wind speed trend sequence.

[0045] Among them, historical multidimensional time series data sets can be collected by wind farms and surrounding meteorological monitoring equipment. The data dimensions include but are not limited to meteorological parameters such as wind speed, wind direction, temperature, air pressure, humidity, as well as power generation-related data such as wind turbine power output and blade angle. Each data point is accompanied by a timestamp (such as time t) and spatial coordinates (such as sensor location and wind turbine number). Using sliding window technology, a series of overlapping time windows corresponding to time intervals can be generated based on a time window of a preset length and a preset step size along the time axis. The spatial dimension data within each time window is aggregated to extract spatial features, such as calculating the mean, variance, maximum, minimum, and other statistical quantities of the wind speed at each spatial node in each window, or generating comprehensive wind speed features through weighted averaging (taking into account the weight of the sensor location) to generate a wind speed trend sequence.

[0046] S102 , generating a step size adjustment parameter using a normalization function based on wind speed gradients in adjacent time windows in the wind speed trend sequence.

[0047] Among them, by comparing the wind speed differences in adjacent time windows, the rate of wind speed increase or decrease, namely the wind speed gradient, is determined. The absolute value of the wind speed gradient reflects the severity of the change. The larger the gradient, the faster the wind speed changes, which may correspond to scenarios such as sudden strong winds or sudden drops in wind force. The wind speed gradient can be converted into the step size adjustment parameter required for model training using special mathematical transformations. For example, through a normalization function such as the S-shaped function, the continuous value of the wind speed gradient is mapped to a preset step size parameter interval to generate a step size adjustment parameter. This allows the step size parameter to change smoothly within a reasonable range, neither too large to cause model oscillation nor too small to cause slow training.

[0048] S103, dynamically adjust the parameters of the pre-trained time series prediction model based on the step size adjustment parameter, and combine the layered transfer learning strategy to perform domain adaptation on the top-level network parameters of the time series prediction model through the adaptive optimization algorithm to obtain the optimized time series prediction model.

[0049] Among them, the core parameters of the pre-trained time series prediction model are dynamically optimized based on the step size adjustment parameter, which may include: adaptive learning rate adjustment: dynamically adjust the learning rate during the model training process according to the severity of the real-time wind speed change (reflected by the step size parameter). For example, when the wind speed gradient is large (that is, the step size parameter indicates a drastic change), increase the learning rate to speed up the model's response to new trends; when the wind speed tends to be stable, reduce the learning rate to avoid excessive parameter fluctuations, ensuring that the training process is both efficient and stable; differentiated module updates: different parameter update strategies are used for different types of network layers in the model (such as convolutional neural network layers and recurrent neural network layers). For example, when performing gradient correction on the weights of the convolution layer that extracts spatiotemporal features, the update amplitude is dynamically adjusted in combination with the step size parameter, so that the model can more accurately capture the local spatiotemporal correlation characteristics of the wind speed series. During optimization, the network structure of the pre-trained time series prediction model can be divided into levels, and the top-level network parameters can be progressively adapted through an adaptive optimization algorithm. For example, multi-stage training can be used to optimize the top level, reduce computing resource consumption, and quickly realize the migration of the pre-trained model to the wind power generation scenario, thus solving the problems of low efficiency of training from scratch and insufficient direct fine-tuning in traditional methods.

[0050] S104, obtaining a real-time multi-dimensional time series data set, and generating a wind power generation prediction result using the optimized time series prediction model.

[0051] Among them, the real-time multi-dimensional time series data set is input into the optimized time series prediction model after preprocessing. The features of the real-time data set are successively extracted through the bottom feature extraction layer of the model to extract the general time series features and the top prediction adaptation layer (the parameters adapted to the domain are mapped to the power generation prediction target). The wind power generation forecast value for a specific time interval in the future (such as the next 1 hour) is output layer by layer, and the prediction result combining the coordinates of the power generation equipment, wind force changes and power generation changes is generated according to the predicted value.

[0052] The above-mentioned method of fine-tuning a pre-trained large model for wind power generation prediction obtains a historical multi-dimensional time series data set and uses a sliding window to fuse spatiotemporal features to generate a wind speed trend sequence, effectively capturing the spatiotemporal correlation characteristics of multi-dimensional data in the wind power generation scenario, and providing more representative input features for subsequent model optimization; constructing a dynamic normalization function based on the wind speed gradient of adjacent time windows to generate a step size adjustment parameter, which can adaptively adjust the step size in the optimization process according to the non-stationary characteristics of wind speed changes, and solve the problem of imbalance between model convergence speed and prediction accuracy caused by the traditional fixed step size strategy when the wind speed fluctuates violently; implementing dynamic normalization of the pre-trained time series prediction model based on the step size adjustment parameter Parameter adjustment, combined with the hierarchical transfer learning strategy, freezes the underlying feature extraction layer to reuse the general time series feature representation of the pre-trained model, reducing dependence on a large amount of domain data. At the same time, the top-level prediction adaptation layer is adapted to the domain through the adaptive optimization algorithm, thereby enhancing the model's ability to fit the specific spatiotemporal characteristics of the wind power generation field, effectively improving the model's ability to capture the refined wind speed trend changes in real-time multi-dimensional spatiotemporal characteristics under complex meteorological environments, generating high-precision wind power generation prediction results, and improving the long-term accuracy and dynamic environment robustness of wind power generation predictions. It solves the problems of insufficient model adaptability and limited prediction accuracy in existing technologies, and is suitable for time series prediction scenarios in the wind power generation field.

[0053] In one embodiment, a step size adjustment parameter is generated using a normalization function based on wind speed gradients in adjacent time windows in a wind speed trend sequence, specifically including:

[0054] S201, calculate the wind speed mean vector of adjacent time windows and obtain the vector sequence {V t} and {V t+1}, where t represents the time window index;

[0055] S202, based on the vector sequence {V t} and {V t+1}, and construct the wind speed gradient matrix ΔV=||V by differential operation while retaining the gradient direction feature t+1 -V t ||2;

[0056] S203, constructing a dynamic normalization function F based on historical gradient extreme values based on the wind speed gradient matrix ΔV Where V max It represents the maximum wind speed change within the sliding window period, and α represents the smoothing coefficient;

[0057] S204: linearly combine the dynamic normalization function F with the preset basic step length to generate a step length adjustment parameter Step size =β·(1+F), where β represents the preset basic step size.

[0058] Specifically, for adjacent time windows t and t+1, the mean vector V of the internal multidimensional wind speed data is calculated to form a vector sequence {V t} and {V t+1 For example, if a time window contains wind speed data (wind speed, wind direction, temperature) at three time points, then the vector V is the mean vector of the wind speed, wind direction, and temperature dimensions at these three time points. Perform a differential operation on the mean vectors of adjacent windows to obtain a gradient vector. Each dimension represents the change in the corresponding wind speed feature (such as the wind speed change rate, the wind direction change angle), while retaining the direction information of the gradient vector (i.e., the positive / negative sign), indicating whether the wind speed is increasing or decreasing. Arrange the gradient vectors of all time windows in chronological order to form a wind speed gradient matrix ΔV = || V t+1 -V t ||2, by calculating the maximum wind speed change V during the sliding window period max Get the historical extreme value, and introduce the smoothing coefficient to control the sensitivity of the normalization function to obtain the dynamic normalization function Combined with the preset basic step size linearly, the step size adjustment parameter Step is generated size =β·(1+F). Through a dynamic normalization function, the step size parameter automatically adapts to the severity of wind speed fluctuations, responding quickly to sudden strong wind changes while maintaining stability in calm wind conditions. A normalization design based on historical extreme values and the introduction of a smoothing coefficient suppress the interference of abnormal wind speed fluctuations on the step size, improving the robustness of model training. Independent of a specific model architecture, it can be combined with a variety of time series forecasting models (such as LSTM and Transformer), demonstrating its broad applicability.

[0059] In one embodiment, dynamically adjusting parameters of a pre-trained time series prediction model based on a step size adjustment parameter includes:

[0060] S301, based on the step size adjustment parameter, generates a corresponding dynamic learning rate parameter set through a parameter mapping function, and the parameter mapping function satisfies: A t =λ1·tanh(λ2·Step size )+λ3, where A t Indicates the dynamic learning rate at time t, Step size represents the step size adjustment parameter, λ1, λ2, and λ3 represent the adjustable coefficients initialized by transfer learning;

[0061] S302, performing gradient update on the convolution kernel weights of the convolutional neural network layer according to the dynamic learning rate parameter set, using the update formula: Among them, W t represents the weight matrix of the t-th iteration, represents the gradient of the loss function, ρ represents the weight attenuation coefficient, and ⊙ represents the Hadamard product;

[0062] S303, performing momentum update on the hidden state transfer matrix of the recurrent neural network layer using a sliding average algorithm, and the update amount calculation satisfies: Where μ represents the momentum factor, ΔH t Represents the matrix update amount at time t.

[0063] Specifically, the step adjustment parameter (reflecting the severity of wind speed changes) is converted into a dynamic learning rate for model training, so that the learning rate is adaptively adjusted with wind speed fluctuations. t =λ1·tanh(λ2·Step size )+λ3, controls the step size of parameter update. λ1, λ2, and λ3 are initialized as adjustable coefficients through transfer learning to calibrate the mapping range of the learning rate. The nonlinear characteristics of the tanh function are used to map the step size parameter to a reasonable range to avoid violent fluctuations in the learning rate due to sudden changes in wind speed while retaining sensitivity to the gradual trend. Gradient update is implemented for the convolution kernel weights of the convolutional neural network layer using the formula Through the loss function gradient Guide the update direction, ρ||W t ||2 prevents overfitting, and the Hadamard product ⊙ implements element-by-element scaling of the gradient by the dynamic learning rate. The momentum mechanism is introduced into the hidden state transfer matrix of the recurrent neural network layer that processes time-dependent dependencies to enhance the modeling ability of long-term sequence dependencies. The update amount is calculated as By accumulating historical update directions through the momentum factor μ, convergence is accelerated and oscillations are reduced, enabling the recurrent neural network layer to more stably transmit hidden states when processing long sequences, preventing gradient vanishing or exploding. The gradient update amplitude is adjusted based on wind speed changes to adapt to the non-stationary characteristics of time series data (such as periodicity and sudden changes in wind speed). This allows the model to dynamically balance training speed and accuracy in complex meteorological environments, effectively capturing refined trends in wind speed and improving the long-term stability and robustness of wind power forecasting.

[0064] In one embodiment, a hierarchical transfer learning strategy is combined with an adaptive optimization algorithm to perform domain adaptation on the top-level network parameters of the time series prediction model to obtain an optimized time series prediction model, including:

[0065] S401, dividing the network structure of the time series prediction model into a bottom feature extraction layer and a top prediction adaptation layer according to the layer depth;

[0066] S402, freezing the network parameters of the bottom feature extraction layer, and dynamically optimizing the fully connected parameters of the top prediction adaptation layer using a domain adaptation loss function to obtain a domain adaptation meta-parameter set;

[0067] S403, using an adaptive optimizer based on a meta-learning strategy to perform multi-stage progressive fine-tuning on the domain adaptation meta-parameter set to obtain an optimized time series prediction model, wherein the multi-stage progressive fine-tuning includes a multi-step optimization process with at least two parameter adjustment stages. In the first stage, historical wind speed data distribution fitting is used for parameter initialization. In the second stage, online gradient correction is performed through real-time data stream, and the parameter distribution consistency between the pre-trained model and the domain adaptation model in the transfer learning process is maintained through the KL divergence constraint term.

[0068] Exemplarily, the underlying feature extraction layer is responsible for extracting common features of time series data (such as wind speed fluctuation patterns and periodic changes), which has a certain universality in different meteorological scenarios, so the parameters are frozen to retain the pre-trained knowledge. The top-level prediction adaptation layer consists of a fully connected layer, which is responsible for mapping the underlying features to wind power prediction results. The parameters of this layer can be dynamically optimized through domain adaptation to adapt to the characteristics of the target scenario (such as the terrain influence of a specific wind farm), and can be dynamically optimized through a domain adaptation loss function (such as MSE) to obtain a domain adaptation meta-parameter set. Through a multi-stage training strategy, the meta-parameter set is gradually optimized while maintaining the model's memory of pre-trained knowledge. In the first stage, the historical wind speed data of the target domain (such as hourly data for the past year) can be used to initialize the model to establish preliminary adaptation to the target domain while maintaining consistency with the pre-trained knowledge. In the second stage, online gradient updates are performed using real-time wind speed data (such as minute-level data at the current moment), enabling the model to capture real-time changing trends in wind speed and adapt to the dynamic evolution of meteorological conditions. The KL divergence constraint ensures that the distribution of model parameters does not deviate too much from the pre-training knowledge during domain adaptation, thereby balancing adaptability and stability.

[0069] In one embodiment, the network structure of the time series prediction model is divided into a bottom feature extraction layer and a top prediction adaptation layer according to the hierarchical depth, including:

[0070] In step S501, the convolutional layers and recurrent neural network encoder in the time series prediction model are divided into the bottom feature extraction layer, and the last k fully connected layers in the pre-trained time series model are reconstructed into a domain adaptation prediction head. A learnable domain embedding vector is introduced to form a top prediction adaptation layer. In step S502, the level of the fully connected layer group is dynamically determined by the following formula: Among them, D in Denotes the input feature dimension, D out Denotes the output prediction dimension, D hidden Represents the baseline dimension of the hidden layer of the pre-trained model.

[0071] Specifically, the convolutional layer is used to capture local patterns in time series data (such as short-term fluctuations and periodic changes in wind speed). The recurrent neural network encoder is responsible for processing sequence dependencies and extracting long-term time series features (such as wind speed change trends and seasonal patterns). The general time series feature representation learned in the pre-training stage is divided into the bottom feature extraction layer. The last k fully connected layers of the pre-trained model are reorganized into a domain adaptation prediction head, and the output is directly mapped to the prediction result (such as directly outputting power generation). A learnable domain embedding vector related to the input data dimension can be introduced. Through training, the specific features of the target domain (such as the impact of terrain on wind speed) are learned and integrated with the features provided by the bottom layer to form the top prediction adaptation layer. Formula for dynamically determining the number of fully connected layer groups By calculating the required number of fully connected layer groups k, the network can smoothly transform the input feature dimension D in Mapped to the corresponding output prediction dimension D out This hierarchical design enables the model to achieve prediction accuracy comparable to that of a model trained from scratch using only a small amount of target field data, while reducing the time required for model training.

[0072] In one embodiment, S601, the adaptive optimizer uses an improved Adam algorithm based on dynamic weighted averaging, and its parameter update formula satisfies:

[0073]

[0074] in, Represents the dynamic basic learning rate, Ω represents the learning rate scaling factor, β represents the preset basic step size, Step size represents the step size adjustment parameter, z1 represents the first-order moment decay rate, γ represents the parameter regularization coefficient, g t represents the gradient at time t, and They represent the bias-corrected first-order and second-order moment estimates, and ε represents a numerical stability constant.

[0075] For example, a dynamic base learning rate The step size parameter Step is calculated by the inverse hyperbolic tangent function size Mapped to the learning rate, it automatically increases the learning rate when the wind speed changes drastically (accelerates convergence) and reduces it when it is stable (improves stability), and uses the Ω scaling factor to adjust the overall amplitude of the learning rate; momentum mixing term Instead of the traditional exponential average, the bias-corrected first-order moment estimate (historical momentum) is balanced by linear interpolation. and the gradient g at time t t z1 controls the weight of historical information and can be reduced at high wind speed gradients to quickly respond to new trends. Regularization term Used to prevent model overfitting and enhance generalization ability. Second-order moment estimation term This algorithm inherits the characteristics of the Adam algorithm (an optimization algorithm that combines momentum with adaptive learning rates), adaptively adjusting the learning rates of various parameters to alleviate gradient sparsity. Leveraging an improved Adam algorithm with dynamic learning rates, a hybrid momentum mechanism, and built-in regularization, it improves adaptability to non-stationary time series data in wind speed forecasting tasks, enabling more efficient and stable convergence to meet the real-time and sudden changes required by wind power generation scenarios.

[0076] In one embodiment, at S701, the domain adaptation loss function adopts the following multi-objective joint optimization formula:

[0077] L total =I1L pred +I2L KL +I3L smooth

[0078] Among them, L pred represents the mean square error loss of wind speed prediction, L KL Represents the KL divergence constraint term of the parameter distribution of the pre-trained model and the adapted model, L smooth represents the temporal smoothing constraint term based on the second-order derivative of adjacent prediction results, and I1, I2, and I3 represent learnable adaptive weighting coefficients.

[0079] Specifically, the mean square error loss of wind speed prediction can be defined as the mean square error (MSE) between the predicted wind speed (or power generation) and the true value, which can be refined as follows: where y t is the predicted value, is the true value, and T represents the time step. It is used to ensure that the model prediction results are close to the real data. KL divergence constraint term L KL =D KL (P pre (θ)||P adapted (θ)), where P pre (θ) represents the parameter distribution of the pre-trained model, that is, the parameter distribution obtained by training on general time series data (such as weather, power load, etc.), which serves as a reference distribution for knowledge preservation, fine-tuning starting point and KL divergence constraint, P adapted(θ) represents the distribution of the adapted model parameters, that is, the distribution of the model parameters after fine-tuning (domain adaptation). As domain adaptation and task specialization, the KL divergence constraint is used to balance the retention of general knowledge and learning domain characteristics during the adaptation process; the difference between the distribution of model parameters after fine-tuning and the distribution of pre-trained model parameters is measured to prevent the top-level parameters from deviating excessively from the pre-trained model during fine-tuning, retain the general timing patterns learned in pre-training, and constrain the model complexity to avoid overfitting when there is less data in the target domain. KL divergence is calculated only for the parameters of the top adaptation layer, rather than the entire model, to reduce the amount of calculation. Time series smoothing constraint term Among them, y t+1 、y t and y t-1 The predicted values from the model for three consecutive time steps are forced to be smooth by penalizing the drastic changes in adjacent predicted values to avoid unreasonable oscillations. The time step index t is used to cover all adjacent triplets and the predicted value y is adjusted by backpropagation. t While maintaining the prediction accuracy, it satisfies the smoothness and penalizes the second-order derivative (curvature) of the prediction result to reflect the local curvature change of the prediction result. pred Dominant, weighted coefficient I1 can increase the acceleration of convergence and reduce the prediction error. KL and L smooth The weight coefficients I2 and I3 are increased to prevent overfitting, without manual adjustment of loss weights, and multiple objectives are dynamically balanced according to the training status; it adapts to different data distributions (such as stable wind conditions and sudden wind conditions) and improves the generalization ability of the model; while ensuring accuracy, it reduces the cost of hyperparameter tuning.

[0080] The present invention provides a method for fine-tuning a pre-trained large model for wind power generation prediction, which effectively improves the accuracy and model adaptability of wind power generation prediction by dynamically adjusting the parameters of the pre-trained large model and the layered transfer learning strategy. Specifically, it uses the spatiotemporal feature fusion technology of the sliding window to capture the spatiotemporal correlation characteristics of multi-dimensional wind speed data, combines the dynamic normalized step size parameters generated by the wind speed gradient of adjacent time windows, and adaptively adjusts the model learning rate and parameter update strategy to solve the problem of imbalance between convergence speed and accuracy of the traditional fixed step size when the wind speed suddenly changes; freezes the underlying feature extraction layer through layered transfer learning to retain the pre-trained The improved Adam algorithm integrates the momentum mechanism and the regularization term, combines the time series smoothing constraint in the multi-objective joint loss function, and balances the prediction accuracy and smoothness under the dynamic learning rate regulation, effectively suppresses overfitting and improves the robustness of the model to complex non-stationary wind speed changes, thus achieving high-precision long-term wind power generation prediction and stable output in real-time environment.

[0081] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0082] Based on the same inventive concept, the embodiments of the present application also provide a device for implementing the aforementioned fine-tuning of a pre-trained large model for wind power generation prediction. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the device for fine-tuning a pre-trained large model for wind power generation prediction provided below can be found in the limitations of the method for fine-tuning a pre-trained large model for wind power generation prediction provided above, and will not be repeated here.

[0083] In an exemplary embodiment, Figure 2 As shown, a device for fine-tuning a pre-trained large model for wind power generation prediction is provided, comprising:

[0084] The data preprocessing module 11 is used to obtain a historical multi-dimensional time series data set and generate a wind speed trend sequence by fusing spatiotemporal features through a sliding window;

[0085] a step size parameter generating module 12, configured to generate a step size adjustment parameter using a normalization function based on wind speed gradients in adjacent time windows in a wind speed trend sequence;

[0086] The model optimization module 13 is used to dynamically adjust the parameters of the pre-trained time series prediction model based on the step size adjustment parameter, and in combination with the layered transfer learning strategy, to perform domain adaptation on the top-level network parameters of the time series prediction model through the adaptive optimization algorithm to obtain the optimized time series prediction model;

[0087] The real-time prediction module 14 is used to obtain a real-time multi-dimensional time series data set and generate a wind power generation prediction result using an optimized time series prediction model.

[0088] In one embodiment, the step parameter generating module 12 is further configured to:

[0089] Calculate the wind speed mean vector of adjacent time windows and obtain the vector sequence {V t} and {V t+1}, where t represents the time window index;

[0090] Based on the vector sequence {V t} and {V t+1}, and construct the wind speed gradient matrix ΔV=||V by differential operation while retaining the gradient direction feature t+1 -V t ||2;

[0091] Constructing a dynamic normalization function based on historical gradient extreme values based on the wind speed gradient matrix ΔV Where V max It represents the maximum wind speed change within the sliding window period, and α represents the smoothing coefficient;

[0092] Linearly combine the dynamic normalization function F with the preset basic step size to generate the step size adjustment parameter Step size =β·(1+F), where β represents the preset basic step size.

[0093] In one embodiment, Figure 3 As shown, the model optimization module 13 also includes a dynamic parameter adjustment unit for:

[0094] Based on the step size adjustment parameters, the corresponding dynamic learning rate parameter set is generated through the parameter mapping function, and the parameter mapping function satisfies: A t =λ1·tanh(λ2·Step size)+λ3, where A t Indicates the dynamic learning rate at time t, Step size represents the step size adjustment parameter, λ1, λ2, and λ3 represent the adjustable coefficients initialized by transfer learning;

[0095] The convolution kernel weights of the convolutional neural network layer are updated with gradients according to the dynamic learning rate parameter set. The update formula used is: Among them, W t represents the weight matrix of the t-th iteration, represents the gradient of the loss function, ρ represents the weight attenuation coefficient, and ⊙ represents the Hadamard product;

[0096] The momentum of the hidden state transfer matrix of the recurrent neural network layer is updated by the sliding average algorithm, and the update amount calculation satisfies: Where μ represents the momentum factor, ΔH t Represents the matrix update amount at time t.

[0097] In one embodiment, Figure 3 As shown, the model optimization module 13 also includes a hierarchical transfer learning unit for:

[0098] The network structure of the time series prediction model is divided into the bottom feature extraction layer and the top prediction adaptation layer according to the hierarchical depth;

[0099] Freeze the network parameters of the bottom feature extraction layer, and dynamically optimize the fully connected parameters of the top prediction adaptation layer through the domain adaptation loss function to obtain the domain adaptation meta-parameter set;

[0100] An adaptive optimizer based on meta-learning strategy is used to perform multi-stage progressive fine-tuning on the domain adaptation meta-parameter set to obtain the optimized time series prediction model. The multi-stage progressive fine-tuning includes a multi-step optimization process with at least two parameter adjustment stages. In the first stage, the parameters are initialized by fitting the distribution of historical wind speed data. In the second stage, online gradient correction is performed through real-time data stream. The KL divergence constraint term is used to maintain the parameter distribution consistency between the pre-trained model and the domain adaptation model during the transfer learning process.

[0101] In one embodiment, the hierarchical transfer learning unit is further configured to:

[0102] The convolutional layers and recurrent neural network encoders in the time series prediction model are divided into the bottom feature extraction layers. The last k fully connected layers in the pre-trained time series model are reconstructed into a domain adaptation prediction head. Learnable domain embedding vectors are introduced to form the top prediction adaptation layer. The level of the fully connected layer group is dynamically determined by the following formula: Among them, D in Denotes the input feature dimension, D outDenotes the output prediction dimension, D hidden Represents the baseline dimension of the hidden layer of the pre-trained model.

[0103] In one embodiment, the hierarchical transfer learning unit is further configured to: construct an adaptive optimizer using an improved Adam algorithm based on dynamic weight averaging, wherein the parameter update formula satisfies:

[0104]

[0105] in, Represents the dynamic basic learning rate, Ω represents the learning rate scaling factor, β represents the preset basic step size, Step size represents the step size adjustment parameter, z1 represents the first-order moment decay rate, γ represents the parameter regularization coefficient, g t represents the gradient at time t, and They represent the bias-corrected first-order and second-order moment estimates, and ε represents a numerical stability constant.

[0106] In one embodiment, the hierarchical transfer learning unit is further configured to employ the following multi-objective joint optimization formula:

[0107] L total =I1L pred +I2L KL +I3L smooth

[0108] Among them, L pred represents the mean square error loss of wind speed prediction, L KL Represents the KL divergence constraint term of the parameter distribution of the pre-trained model and the adapted model, L smooth represents the temporal smoothing constraint term based on the second-order derivative of adjacent prediction results, and I1, I2, and I3 represent learnable adaptive weighting coefficients.

[0109] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of fine-tuning a pre-trained large model to perform wind power generation prediction as described above.

[0110] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0111] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. A method for fine-tuning a pre-trained large model for wind power generation prediction, characterized in that: The method comprises: Obtain a historical multidimensional time series dataset and generate a wind speed trend series by fusing spatiotemporal features through a sliding window. generating a step size adjustment parameter using a normalization function based on wind speed gradients in adjacent time windows in the wind speed trend sequence; Dynamically adjusting the parameters of the pre-trained time series prediction model based on the step size adjustment parameter, and combining the layered transfer learning strategy to perform domain adaptation on the top-level network parameters of the time series prediction model through an adaptive optimization algorithm to obtain an optimized time series prediction model; A real-time multi-dimensional time series data set is obtained, and a wind power generation prediction result is generated using the optimized time series prediction model.

2. The method according to claim 1, characterized in that The step size adjustment parameter is generated by using a normalization function based on the wind speed gradients of adjacent time windows in the wind speed trend sequence, specifically including: Calculate the wind speed mean vector of the adjacent time windows to obtain the vector sequence {V t } and {V t+1 }, where t represents the time window index; Based on the vector sequence {V t } and {V t+1 }, and construct the wind speed gradient matrix ΔV=||V by differential operation while retaining the gradient direction feature t+1 -V t ||2; Based on the wind speed gradient matrix ΔV, a dynamic normalization function based on historical gradient extreme values is constructed. Where V max It represents the maximum wind speed change within the sliding window period, and α represents the smoothing coefficient; The dynamic normalization function F is linearly combined with the preset basic step size to generate the step size adjustment parameter Step size =β·(1+F), where β represents the preset basic step size.

3. The method according to claim 1, characterized in that The dynamically adjusting the parameters of the pre-trained time series prediction model based on the step size adjustment parameter includes: Based on the step size adjustment parameter, a corresponding dynamic learning rate parameter set is generated through a parameter mapping function, and the parameter mapping function satisfies: A t =λ1·tanh(λ2·Step size )+λ3, where A t Indicates the dynamic learning rate at time t, Step size represents the step size adjustment parameter, λ1, λ2, and λ3 represent the adjustable coefficients initialized by transfer learning; The convolution kernel weights of the convolutional neural network layer are gradient updated according to the dynamic learning rate parameter set, and the update formula used is: Among them, W t represents the weight matrix of the t-th iteration, represents the gradient of the loss function, ρ represents the weight attenuation coefficient, and ⊙ represents the Hadamard product; The momentum of the hidden state transfer matrix of the recurrent neural network layer is updated by the sliding average algorithm, and the update amount calculation satisfies: Where μ represents the momentum factor, ΔH t Represents the matrix update amount at time t.

4. The method according to claim 1, wherein The layered transfer learning strategy is combined with an adaptive optimization algorithm to perform domain adaptation on the top-level network parameters of the time series prediction model to obtain an optimized time series prediction model, including: Dividing the network structure of the time series prediction model into a bottom feature extraction layer and a top prediction adaptation layer according to the hierarchical depth; Freezing the network parameters of the bottom feature extraction layer, and dynamically optimizing the fully connected parameters of the top prediction adaptation layer through a domain adaptation loss function to obtain a domain adaptation meta-parameter set; An adaptive optimizer based on a meta-learning strategy is used to perform multi-stage progressive fine-tuning on the domain adaptation meta-parameter set to obtain an optimized time series prediction model. The multi-stage progressive fine-tuning includes a multi-step optimization process with at least two parameter adjustment stages. In the first stage, historical wind speed data distribution fitting is used for parameter initialization. In the second stage, online gradient correction is performed through real-time data stream. The parameter distribution consistency between the pre-trained model and the domain adaptation model is maintained through the KL divergence constraint term during the transfer learning process.

5. The method according to claim 4, characterized in that The network structure of the time series prediction model is divided into a bottom feature extraction layer and a top prediction adaptation layer according to the hierarchical depth, including: The convolutional layers and recurrent neural network encoder in the time series prediction model are divided into the bottom feature extraction layer, and the last k fully connected layers in the pre-trained time series model are reconstructed into a domain adaptation prediction head, and a learnable domain embedding vector is introduced to form the top prediction adaptation layer; wherein the level of the fully connected layer group is dynamically determined by the following formula: Among them, D in Denotes the input feature dimension, D out Denotes the output prediction dimension, D hidden Represents the baseline dimension of the hidden layer of the pre-trained model.

6. The method according to claim 4, characterized in that The adaptive optimizer adopts an improved Adam algorithm based on dynamic weighted averaging, and its parameter update formula satisfies: in, Represents the dynamic basic learning rate, Ω represents the learning rate scaling factor, β represents the preset basic step size, Step size represents the step size adjustment parameter, z1 represents the first-order moment decay rate, γ represents the parameter regularization coefficient, g t represents the gradient at time t, and They represent the bias-corrected first-order and second-order moment estimates, and ε represents a numerical stability constant.

7. The method according to claim 4, characterized in that The domain adaptation loss function adopts the following multi-objective joint optimization formula: L total =I1L pred +I2L KL +I3L smooth Among them, L pred represents the mean square error loss of wind speed prediction, L KL Represents the KL divergence constraint term of the parameter distribution of the pre-trained model and the adapted model, L smooth represents the temporal smoothing constraint term based on the second-order derivative of adjacent prediction results, and I1, I2, and I3 represent learnable adaptive weighting coefficients.

8. A device for fine-tuning a pre-trained large model for wind power generation prediction, characterized in that: The device comprises: The data preprocessing module is used to obtain historical multidimensional time series data sets and generate wind speed trend series by fusing spatiotemporal features through sliding windows; a step size parameter generating module, configured to generate a step size adjustment parameter using a normalization function based on wind speed gradients in adjacent time windows in the wind speed trend sequence; A model optimization module is used to dynamically adjust the parameters of the pre-trained time series prediction model based on the step size adjustment parameter, and in combination with the layered transfer learning strategy, to perform domain adaptation on the top-level network parameters of the time series prediction model through an adaptive optimization algorithm to obtain an optimized time series prediction model; The real-time prediction module is used to obtain a real-time multi-dimensional time series data set and generate a wind power generation prediction result using the optimized time series prediction model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images