A method for fine-tuning a pre-trained large model for wind power generation prediction

Abstract

The application discloses a method for fine-tuning a pre-trained large model for wind power generation prediction, which comprises: obtaining a historical multi-dimensional time series dataset, generating a wind speed trend sequence through space-time feature fusion by a sliding window; generating a step adjustment parameter by a normalization function based on the wind speed gradient of adjacent time windows; dynamically adjusting the pre-trained time series prediction model based on the step adjustment parameter, combining a hierarchical transfer learning strategy, and adjusting the top network parameters of the time series prediction model through an adaptive optimization algorithm to obtain an optimized time series prediction model; and processing real-time multi-dimensional time series data sets by using the optimized model to generate wind power prediction results. Through dynamic adjustment of model parameters and the hierarchical transfer learning strategy, the accuracy and model adaptation capability of wind power prediction are effectively improved, and the method is suitable for time series prediction scenarios in the field of wind power generation.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence application and new energy utilization, and particularly relates to a method for fine-tuning a pre-trained large model to perform wind power prediction. BACKGROUND

[0002] With the development of intelligent technology in the field of wind power generation, time series prediction technology based on data driving has gradually become a core means to improve power generation efficiency and grid stability. By modeling the spatio-temporal correlation characteristics of historical meteorological data and power generation, the nonlinear dynamic characteristics of the wind power system can be effectively adapted. The transfer learning method based on pre-trained models has become a current research hotspot due to its reusable general time series feature representation and reduced dependence on field data. In traditional technology, wind power prediction mainly relies on statistical models (such as ARIMA, SARIMA), shallow machine learning methods (such as support vector machine SVM), and early deep learning models (such as LSTM, TCN). By extracting multi-dimensional time series features such as wind speed, wind direction, and temperature through a fixed window, and using a gradient descent algorithm with a static learning rate to optimize model parameters. With the application of pre-trained large models in the field of time series prediction, some schemes attempt to improve the model generalization ability through the transfer learning strategy of freezing the bottom network and fine-tuning the top parameters, but still continue the fixed step parameter updating mechanism. The current wind power prediction method has significant defects in dynamically adapting to complex weather environments: on the one hand, the traditional fixed step optimization strategy is difficult to match the non-stationary change characteristics of the wind speed sequence, and when the wind speed gradient fluctuates sharply, it is easy to cause the imbalance between the model convergence speed and the prediction accuracy; on the other hand, the existing transfer learning framework lacks multi-stage progressive optimization in the domain adaptation process of the top network parameters, and does not effectively constrain the parameter distribution consistency between the pre-trained model and the domain model, making it difficult for the model to capture fine wind speed trend changes when processing real-time multi-dimensional spatio-temporal feature fusion, ultimately leading to insufficient long-term prediction accuracy and weak model robustness. SUMMARY

[0003] Therefore, it is necessary to provide a method for fine-tuning a pre-trained large model to perform wind power prediction.

[0004] In a first aspect, the application provides a method for fine-tuning a pre-trained large model to perform wind power prediction, comprising:

[0005] obtaining a historical multi-dimensional time series dataset, and generating a wind speed trend sequence by spatio-temporal feature fusion through a sliding window;

[0006] generating a step adjustment parameter using a normalization function based on the wind speed gradient of adjacent time windows in the wind speed trend sequence;

[0007] The pre-trained time series prediction model is dynamically parameter-adjusted based on a step adjustment parameter, and a hierarchical transfer learning strategy is combined to adjust the top network parameters of the time series prediction model through a self-adaptive optimization algorithm to obtain an optimized time series prediction model.

[0008] A wind power prediction result is generated by using the optimized time series prediction model based on the obtained real-time multi-dimensional time series data set.

[0009] In one of the embodiments, the step adjustment parameter is generated by using a normalization function based on the wind speed gradient of adjacent time windows in the wind speed trend sequence, and specifically includes:

[0010] The wind speed mean vector of the adjacent time windows is calculated to obtain vector sequences {V t} and {V t+1}, wherein t represents the time window index;

[0011] Based on the vector sequences {V t} and {V t+1}, the wind speed gradient matrix ΔV = ||V t+1 -V t ||2 is constructed through a difference operation under the reserved gradient direction feature.

[0012] The dynamic normalization function F = ΔV / ||ΔV|| is constructed based on the wind speed gradient matrix ΔV based on the historical gradient extreme value. Wherein V max represents the maximum wind speed change amount statistically in the sliding window period, and α represents the smoothing coefficient.

[0013] The dynamic normalization function F is linearly combined with a preset basic step to generate a step adjustment parameter Step size = β·(1+F), wherein β represents the preset basic step.

[0014] In one of the embodiments, the pre-trained time series prediction model is dynamically parameter-adjusted based on the step adjustment parameter, including:

[0015] Based on the step adjustment parameter, a corresponding dynamic learning rate parameter set is generated by using a parameter mapping function, and the parameter mapping function satisfies: A t = λ1·tanh(λ2·Step size )+λ3, wherein A t represents the dynamic learning rate at t moment, Step size represents the step adjustment parameter, and λ1, λ2, and λ3 respectively represent adjustable coefficients initialized by transfer learning.

[0016] According to the dynamic learning rate parameter set, the convolution kernel weight of the convolutional neural network layer is gradient-updated, and the update formula used is: Wherein, W tdenotes the weight matrix of the t-th iteration, denotes the loss function gradient, p denotes the weight decay coefficient, and denotes the Hadamard product;

[0017] The momentum update is performed on the hidden state transition matrix of the recurrent neural network layer by the sliding average algorithm, and the update amount calculation satisfies: wherein, μ denotes the momentum factor, ΔH t denotes the matrix update amount at time t, H t denotes the hidden state transition matrix at time t.

[0018] In one embodiment, the top layer network parameters of the time series prediction model are adjusted by an adaptive optimization algorithm in combination with a hierarchical transfer learning strategy to obtain an optimized time series prediction model, including:

[0019] The network structure of the time series prediction model is divided into a bottom feature extraction layer and a top prediction adaptation layer according to the hierarchical depth;

[0020] The network parameters of the bottom feature extraction layer are frozen, and the fully connected parameters of the top prediction adaptation layer are dynamically optimized by a preset loss function to obtain a set of adaptation layer meta-parameters;

[0021] An adaptive optimizer based on a meta-learning strategy is used to perform multi-stage progressive fine-tuning on the set of adaptation layer meta-parameters to obtain an optimized time series prediction model, wherein the multi-stage progressive fine-tuning includes a multi-step optimization process of at least two parameter adjustment stages, the first stage uses historical wind speed data distribution fitting for parameter initialization, the second stage performs online gradient correction through real-time data flow, and the parameter distribution consistency between the time series prediction model and the optimized time series prediction model during the transfer learning process is maintained through a KL divergence constraint term.

[0022] In one embodiment, the network structure of the time series prediction model is divided into a bottom feature extraction layer and a top prediction adaptation layer according to the hierarchical depth, including:

[0023] The convolutional layer and the recurrent neural network encoder in the time series prediction model are divided into the bottom feature extraction layer, and for the last k fully connected layers in the time series prediction model, a learnable domain embedding vector is introduced to reconstruct the top prediction adaptation layer; wherein the hierarchical level of the fully connected layer group is dynamically determined by the following formula: wherein, D in denotes the input feature dimension, D out denotes the output prediction dimension, and D hidden denotes the time series prediction model hidden layer reference dimension.

[0024] In one embodiment, the adaptive optimizer uses an improved Adam algorithm based on dynamic weight averaging, and the parameter update formula satisfies:

[0025]

[0026] wherein, denotes a dynamic base learning rate, Ω denotes a learning rate scaling factor, β denotes a preset base step size, Step size denotes a step size adjustment parameter, z1 denotes a first moment decay rate, γ denotes a parameter regularization coefficient, g t denotes a gradient at time t, and denote the first and second moment estimates after bias correction, respectively, ε denotes a numerical stability constant, θ t denotes a set of adaptive layer element parameters at time t, θ t+1 denotes a set of adaptive layer element parameters after one update. t denotes a set of adaptive layer element parameters after one update.

[0027] In one embodiment, the preset loss function adopts the following multi-objective joint optimization formula:

[0028] L total = I1L pred + I2L KL + I3L smooth

[0029] wherein, L pred denotes a mean square error loss of wind speed prediction, L KL denotes a KL divergence constraint term between the parameter distribution of the time series prediction model and the optimized time series prediction model, L smooth denotes a time series smoothing constraint term based on the second derivative of adjacent prediction results, I1, I2 and I3 denote learnable adaptive weighting coefficients.

[0030] In a second aspect, the application further provides a device for fine-tuning a pre-trained large model for wind power prediction, characterized in that the device comprises:

[0031] a data preprocessing module configured to obtain a historical multi-dimensional time series data set and generate a wind speed trend sequence by spatiotemporal feature fusion through a sliding window;

[0032] a step size parameter generation module configured to generate a step size adjustment parameter by using a normalization function based on the wind speed gradient of adjacent time windows in the wind speed trend sequence;

[0033] a model optimization module configured to dynamically adjust a pre-trained time series prediction model based on the step size adjustment parameter, and adjust the weight matrix parameters of the top layer network of the time series prediction model by an adaptive optimization algorithm in combination with a hierarchical transfer learning strategy, to obtain an optimized time series prediction model;

[0034] A real-time prediction module is configured to obtain a real-time multi-dimensional time series dataset and generate a wind power prediction result by using an optimized time series prediction model.

[0035] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method for fine-tuning a pre-trained large model to predict wind power as above when executing the computer program.

[0036] The method for fine-tuning a pre-trained large model to predict wind power provided by the present application effectively captures the spatio-temporal correlation characteristics of multi-dimensional data in a wind power scene by obtaining a historical multi-dimensional time series dataset and generating a wind speed trend sequence by spatio-temporal feature fusion using a sliding window; the step length adjustment parameter is generated based on the wind speed gradient of adjacent time windows, which can adaptively adjust the step length in the optimization process according to the non-stationary characteristics of wind speed changes, thereby solving the imbalance between the convergence speed and the prediction accuracy of the model caused by the traditional fixed step length strategy when the wind speed fluctuates sharply; the pre-trained time series prediction model is dynamically adjusted by the step length adjustment parameter, the hierarchical transfer learning strategy is used to freeze the bottom feature extraction layer to reuse the general time series feature representation of the pre-trained model, and the adaptive optimization algorithm is used for the top prediction adaptation layer to adjust the top network parameters of the time series prediction model, while the parameter distribution consistency between the time series prediction model and the optimized time series prediction model in the transfer learning process is maintained through the KL divergence constraint term, thereby reducing the dependence on a large amount of field data, strengthening the fitting ability of the model to the specific spatio-temporal characteristics in the wind power field, and improving the long-term accuracy and dynamic environmental robustness of wind power prediction by using the multi-stage progressive fine-tuning and time series smoothing constraint to improve the ability of the model to capture the fine wind speed trend changes in real-time multi-dimensional spatio-temporal characteristics. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed in the embodiment or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 The flowchart of the method for fine-tuning a pre-trained large model to predict wind power of the present application;

[0039] Figure 2 The structural schematic diagram of the device for fine-tuning a pre-trained large model to predict wind power of the present application;

[0040] Figure 3A structure schematic diagram of a fine-tuning pre-training large model for wind power generation prediction device in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0042] The method for fine-tuning pre-training large model for wind power generation prediction provided by the present application is applicable to a hardware environment comprising terminal equipment, a server and a data storage system. The terminal collects wind power generation data and transmits it to the server. The server processes and predicts the data using a pre-training model. The result is fed back to the terminal. The data storage system is used to store relevant data and model parameters, supporting efficient prediction and management of wind power generation.

[0043] In one embodiment, as shown in Figure 1 the method comprises the following steps:

[0044] S101, a historical multi-dimensional time series data set is obtained, and spatio-temporal feature fusion is performed through a sliding window to generate a wind speed trend sequence.

[0045] The historical multi-dimensional time series data set can be collected by a wind farm and surrounding meteorological monitoring equipment. The data dimensions include, but are not limited to, meteorological parameters such as wind speed, wind direction, temperature, air pressure and humidity, as well as power generation related data such as wind turbine power output and blade angle, and each data point has a time stamp (such as time t) and spatial coordinates (such as sensor position, wind turbine number). Using the sliding window technology, a series of overlapping time windows and corresponding time intervals can be generated by sliding along the time axis with a preset step length according to a preset length of the time window. The spatial dimension data in each time window is aggregated to extract spatial features, such as calculating the mean, variance, maximum and minimum of the wind speed of each spatial node in each window, or generating a comprehensive wind speed feature through weighted average (considering the weight of the sensor position) to generate a wind speed trend sequence.

[0046] S102, based on the wind speed gradient of adjacent time windows in the wind speed trend sequence, a step adjustment parameter is generated using a normalization function.

[0047] The wind speed gradient is determined by comparing the wind speed difference of adjacent time windows. The absolute value of the wind speed gradient reflects the degree of change, and the larger the gradient, the faster the wind speed changes, which may correspond to strong wind or sudden wind reduction. The wind speed gradient can be converted into a step adjustment parameter required for model training through mathematical transformation. For example, the continuous value of the wind speed gradient is mapped to a preset step parameter interval through a normalization function such as a sigmoid function, generating a step adjustment parameter. This can make the step parameter change smoothly within a reasonable range, neither too large to cause model oscillation nor too small to cause slow training.

[0048] In S103, the pre-trained time series prediction model is dynamically parameter adjusted based on the step adjustment parameter, and the top layer network parameters of the time series prediction model are adjusted through an adaptive optimization algorithm combined with a hierarchical transfer learning strategy, to obtain an optimized time series prediction model.

[0049] The dynamic optimization of the core parameters of the pre-trained time series prediction model based on the step adjustment parameter can include: adaptive learning rate adjustment, which dynamically adjusts the learning rate in the model training process according to the degree of real-time wind speed change (reflected by the step parameter). For example, when the wind speed gradient is large (i.e., the step parameter indicates a large change), the learning rate is increased to speed up the model's response to new trends; when the wind speed tends to be stable, the learning rate is reduced to avoid excessive parameter fluctuations and ensure efficient and stable training process; and differential update of different modules, which uses different parameter update strategies for different types of network layers in the model (such as convolutional neural network layers and recurrent neural network layers). For example, when updating the weights of the convolutional layer that extracts spatial and temporal features, the update amplitude is dynamically adjusted based on the step parameter to enable the model to more accurately capture the local spatial and temporal correlation characteristics of the wind speed sequence. During optimization, the network structure of the pre-trained time series prediction model can be divided into levels, and the top layer network parameters can be progressively adjusted through an adaptive optimization algorithm, such as multi-stage training of the top layer to reduce computational resource consumption, quickly realize the transfer of the pre-trained model to the wind power generation scenario, and solve the problems of low efficiency of training from scratch and insufficient direct fine-tuning adaptation in traditional methods.

[0050] In S104, real-time multi-dimensional time series data sets are obtained, and the optimized time series prediction model is used to generate wind power generation prediction results.

[0051] The real-time multi-dimensional time series data set is input into the optimized time series prediction model after preprocessing. The features of the real-time data set are sequentially extracted through the bottom feature extraction layer and the top prediction adaptation layer (parameters adapted to the field, mapping features to power generation prediction targets) of the model. The future wind power generation prediction value of a specific time interval (such as 1 hour) is output layer by layer, and the prediction result combined with the coordinates of the power generation equipment, the wind speed change and the power generation change is generated based on the prediction value.

[0052] The method for fine-tuning a pre-trained large model for wind power generation prediction, by obtaining a historical multi-dimensional time series data set and using a sliding window to perform spatio-temporal feature fusion to generate a wind speed trend sequence, effectively captures the spatio-temporal correlation characteristics of multi-dimensional data in the wind power generation scene, providing more representative input features for subsequent model optimization; a dynamic normalization function is constructed based on the wind speed gradient of adjacent time windows to generate a step adjustment parameter, which can adaptively adjust the step size in the optimization process according to the non-stationary characteristics of wind speed changes, solving the problem of imbalance between model convergence speed and prediction accuracy caused by traditional fixed step size strategy when wind speed fluctuates sharply; based on the step adjustment parameter, the pre-trained time series prediction model is implemented dynamically parameter adjustment, combined with the hierarchical transfer learning strategy to freeze the bottom feature extraction layer to reuse the general time series feature representation of the pre-trained model, reducing the dependence on a large amount of field data, while adjusting the top prediction adaptation layer through an adaptive optimization algorithm, strengthening the model's fitting ability for specific spatio-temporal features in the wind power generation field, effectively improving the model's ability to capture fine wind speed trend changes in real-time multi-dimensional spatio-temporal features in complex weather environments, generating high-precision wind power generation prediction results, improving the long-term accuracy and dynamic environmental robustness of wind power generation prediction, solving the problem of insufficient model adaptation and limited prediction accuracy in the prior art, and being applicable to time series prediction scenarios in the wind power generation field.

[0053] In one embodiment, based on the wind speed gradient of adjacent time windows in the wind speed trend sequence, a step adjustment parameter is generated using a normalization function, specifically including:

[0054] S201, calculating the wind speed mean vector of adjacent time windows, obtaining vector sequences {V t} and {V t+1}, where t represents the time window index;

[0055] S202, based on the vector sequences {V t} and {V t+1}, the gradient direction feature is retained and a wind speed gradient matrix ΔV = ||V t+1 -V t ||2 is constructed by difference operation;

[0056] S203, based on the wind speed gradient matrix ΔV, a dynamic normalization function F based on historical gradient extreme value is constructed where V max represents the maximum wind speed change during the sliding window period, and α represents the smoothing coefficient;

[0057] S204, linearly combining the dynamic normalization function F and the preset basic step size to generate the step adjustment parameter Step size= β · (1 + F), wherein β represents a preset basic step length.

[0058] Specifically, for adjacent time windows t and t+1, the mean vector V of the multi-dimensional wind speed data inside is calculated to form a vector sequence {V t} and {V t+1}, for example, if a time window contains wind speed data (wind speed, wind direction, temperature) of 3 time points, the vector V is the mean vector of the wind speed, wind direction and temperature dimensions of the 3 time points. The difference operation is performed on the mean vectors of adjacent windows to obtain the gradient vector, each dimension representing the change amount (such as wind speed change rate, wind direction change angle) of the corresponding wind speed feature, while retaining the direction information (i.e. positive / negative sign) of the gradient vector, indicating whether the wind speed is rising or falling. The gradient vectors of all time windows are arranged in chronological order to form a wind speed gradient matrix ΔV = ||V t+1 -V t ||2, the maximum wind speed change V max is obtained by calculating in the sliding window period, and a smoothing coefficient is introduced to control the sensitivity of the normalization function, and a dynamic normalization function F is obtained is linearly combined with the preset basic step length to generate a step length adjustment parameter Step size = β · (1 + F). Through the dynamic normalization function, the step length parameter can automatically adapt to the severity of wind speed change, quickly respond in strong wind mutation, and remain stable in stable wind conditions. The normalization design based on the historical extreme value and the introduction of the smoothing coefficient suppress the interference of abnormal wind speed fluctuations on the step length, and improve the robustness of model training. It does not depend on a specific model architecture and can be used with various time series prediction models (such as LSTM, Transformer), and has wide applicability.

[0059] In one of the embodiments, dynamic parameter adjustment is performed on the pre-trained time series prediction model based on the step length adjustment parameter, including:

[0060] S301, based on the step length adjustment parameter, a corresponding dynamic learning rate parameter set is generated through a parameter mapping function, which satisfies: A t = λ1·tanh(λ2·Step size )+λ3, wherein A t represents the dynamic learning rate at time t, Step size represents the step length adjustment parameter, and λ1, λ2, λ3 respectively represent adjustable coefficients initialized through transfer learning;

[0061] S302, gradient update is performed on the convolution kernel weight of the convolutional neural network layer according to the dynamic learning rate parameter set, and the update formula used is: wherein W t represents the weight matrix of the tth iteration, denotes the loss function gradient, p denotes the weight decay coefficient, and denotes the Hadamard product;

[0062] In S303, the hidden state transition matrix of the recurrent neural network layer is updated by a moving average algorithm, and the update amount satisfies: wherein, m represents a momentum factor, and AH t denotes the matrix update amount at time t, H t denotes the hidden state transition matrix at time t.

[0063] Specifically, the step adjustment parameter (reflecting the degree of change in wind speed) is converted into a dynamic learning rate of model training, so that the learning rate is adaptively adjusted according to wind speed fluctuations. Through a parameter mapping function A t = l1·tanh(l2·Step size )+ l3, the step of parameter update is controlled, and l1, l2 and l3 are adjustable coefficients initialized by transfer learning to calibrate the mapping range of the learning rate. The non-linear characteristics of the tanh function are used to map the step parameter to a reasonable interval, avoiding the sharp oscillation of the learning rate due to sudden changes in wind speed, while retaining sensitivity to gradual trends. The gradient update is performed on the convolution kernel weight of the convolutional neural network layer, and the formula The loss function gradient points to the update direction, p||W t ||2 prevents overfitting, and the Hadamard product is used to realize the element-by-element scaling of the dynamic learning rate to the gradient. The momentum mechanism is introduced to the hidden state transition matrix of the recurrent neural network layer that handles time series dependence, to enhance the modeling capability of long-term sequence dependence. The update amount is calculated as The momentum factor m accumulates the historical update direction to accelerate convergence and reduce oscillation, so that the recurrent neural network layer can more stably transfer the hidden state when processing long sequences, avoiding gradient vanishing or explosion. Combined with wind speed change adjustment of gradient update amplitude, the non-stationary characteristics of time series data (such as periodicity and suddenness of wind speed) are adapted. The model can dynamically balance the training speed and accuracy in complex weather conditions, effectively capture the fine trend changes of wind speed, and improve the long-term stability and robustness of wind power prediction.

[0064] In one embodiment, a layered transfer learning strategy is combined to adjust the top layer network parameters of the time series prediction model through an adaptive optimization algorithm to obtain an optimized time series prediction model, including:

[0065] In S401, the network structure of the time series prediction model is divided into a bottom feature extraction layer and a top prediction adaptation layer according to the hierarchical depth.

[0066] S402, freeze the network parameters of the bottom feature extraction layer, and dynamically optimize the full connection parameters of the top prediction adaptation layer through a preset loss function to obtain an adaptation layer meta-parameter set;

[0067] S403, using an adaptive optimizer based on a meta-learning strategy to perform multi-stage progressive fine-tuning on the adaptation layer meta-parameter set to obtain an optimized time series prediction model, wherein the multi-stage progressive fine-tuning includes a multi-step optimization process of at least two parameter adjustment stages, the first stage uses historical wind speed data distribution fitting to initialize parameters, and the second stage performs online gradient correction through real-time data flow, and a KL divergence constraint term is used to maintain the parameter distribution consistency between the time series prediction model in the transfer learning process and the optimized time series prediction model.

[0068] Illustratively, the bottom feature extraction layer is responsible for extracting general features of time series data (such as wind speed fluctuation patterns, periodic changes), which has certain universality in different meteorological scenarios, so the parameters are frozen to retain the pre-training knowledge. The top prediction adaptation layer is composed of fully connected layers, which is responsible for mapping the bottom features to the wind power prediction results. The parameters of this layer can be dynamically optimized through domain adaptation, to adapt to the characteristics of the target scene (such as the topographic influence of a specific wind farm). The preset loss function (such as an improved loss function based on MSE) can be used for dynamic optimization to obtain an adaptation layer meta-parameter set. Through a multi-stage training strategy, the meta-parameter set is gradually optimized while maintaining the model's memory of pre-training knowledge. In the first stage, the model can be initialized using historical wind speed data (such as hourly data for the past year) in the target domain to establish an initial adaptation to the target domain while maintaining consistency with the pre-training knowledge. In the second stage, online gradient update is performed through real-time wind speed data (such as minute-level data at the current time) to enable the model to capture real-time trends of wind speed, adapt to dynamic evolution of meteorological conditions, and ensure that the model parameter distribution does not deviate too much from the pre-training knowledge during the learning process through a KL divergence constraint term, balancing adaptability and stability.

[0069] In one embodiment, the network structure of the time series prediction model is divided into a bottom feature extraction layer and a top prediction adaptation layer according to the hierarchical depth, including:

[0070] S501, dividing the convolutional layers and the recurrent neural network encoder in the time series prediction model into the bottom feature extraction layer, and reconstructing the last k fully connected layers in the time series prediction model into the top prediction adaptation layer by introducing a learnable domain embedding vector; wherein the hierarchical level of the fully connected layer group is dynamically determined by the following formula: wherein, D in represents the input feature dimension, D out represents the output prediction dimension, and D hidden represents the hidden layer reference dimension of the time series prediction model.

[0071] Specifically, the convolutional layer is used to capture the local patterns in the time series data (such as short-term fluctuations, periodic changes in wind speed), and the recurrent neural network encoder is responsible for handling sequence dependencies and extracting long-term time series features (such as wind speed trends, seasonal patterns). In the pre-training phase, a general time series feature representation has been learned, which is divided into a bottom feature extraction layer. For the last k fully connected layers in the time series prediction model, a learnable domain embedding vector related to the input data dimension can be introduced. Through training, the specific features of the target domain (such as the influence of terrain on wind speed) are learned, and are fused with the features provided by the bottom layer to form a top prediction adaptation layer. Formula for dynamically determining the number of fully connected layer groups By calculating the required number of fully connected layer groups k, it is ensured that the network can smoothly map the input feature dimension D in to the corresponding output prediction dimension D out The hierarchical design can achieve prediction accuracy comparable to that of a model trained from scratch while reducing the time required for model training using only a small amount of target field data.

[0072] In one embodiment, the adaptive optimizer uses an improved Adam algorithm based on dynamic weight averaging, and the parameter update formula satisfies:

[0073]

[0074] where, denotes the dynamic base learning rate, Ω denotes the learning rate scaling factor, β denotes the preset base step size, Step size denotes the step size adjustment parameter, z1 denotes the first-order moment decay rate, γ denotes the parameter regularization coefficient, g t denotes the gradient at time t, and denote the first-order and second-order moment estimates after bias correction, respectively, ε denotes a numerical stability constant, θ t denotes the adaptation layer parameter set at time t, θ t+1 denotes the adaptation layer parameter set after one update. t

[0075] Exemplarily, in the process of updating θ t to θ t+1 , the dynamic base learning rate The step size parameter Step size is mapped to the learning rate through the inverse hyperbolic tangent function, which automatically increases the learning rate when the wind speed changes rapidly (accelerates convergence), and reduces it when it is stable (improves stability), and the Ω scaling factor is used to adjust the overall amplitude of the learning rate; the momentum mixing term ​Instead of traditional exponential average, the first moment estimate after bias correction is balanced by linear interpolation (historical momentum) The gradient g at time t t z1 controls the weight of historical information, which can be reduced at high wind speed gradient to quickly respond to new trends. Regularization term Used to prevent model overfitting and enhance generalization ability. Second moment estimate term Used to inherit the characteristics of the Adam algorithm (an optimization algorithm that combines momentum method and adaptive learning rate), to adaptively adjust the learning rate of each parameter and alleviate the gradient sparsity problem. By using the improved Adam algorithm, through dynamic learning rate, hybrid momentum mechanism and built-in regularization, the adaptability to non-stationary time series data in wind speed prediction tasks is improved, which can converge more efficiently and stably, and adapt to the real-time and mutability requirements of wind power generation scenarios.

[0076] In one of the embodiments, S701, the preset loss function adopts the following multi-objective joint optimization formula:

[0077] L total = I1L pred + I2L KL + I3L smooth

[0078] Where L pred represents the mean square error loss of wind speed prediction, L KL represents the KL divergence constraint term between the parameter distribution of the time series prediction model and the optimized time series prediction model, L smooth represents the time series smoothing constraint term based on the second derivative of adjacent prediction results, and I1, I2 and I3 represent the learnable adaptive weighting coefficients.

[0079] Specifically, the mean square error loss of wind speed prediction can be defined as the mean square error (MSE) between the predicted wind speed (or power generation) and the true value, which can be refined as Where y t is the predicted value, is the true value, and T represents the time step. Used to ensure that the model prediction result approaches the true data. The KL divergence constraint term L KL = D KL (P pre (θ)||P adapted (θ)), where P pre (θ) represents the parameter distribution of the time series prediction model, i.e. the parameter distribution obtained by training on general time series data (such as meteorological, power load, etc.), which serves as a reference distribution for knowledge preservation, provides a starting point for fine-tuning and KL divergence constraint, P adapted(θ) represents the parameter distribution of the optimized time series prediction model, that is, the model parameter distribution after fine-tuning, as task specialization, through the KL divergence constraint, balancing the preservation of general knowledge and the learning of domain characteristics in the configuration process; measure the difference between the model parameter distribution after fine-tuning and the pre-training model parameter distribution, prevent the top layer parameters from deviating too much from the pre-training model during the fine-tuning process, retain the general time series patterns learned by pre-training, and when the target domain data is small, constrain the model complexity to avoid overfitting. Only the parameters of the top adaptation layer are used to calculate the KL divergence, not the entire model, to reduce the amount of calculation. Time series smoothing constraint term wherein y t+1 , y t and y t-1 come from the prediction values of the model for the three consecutive time steps, by penalizing the dramatic change of adjacent prediction values, forcing the prediction curve to be smooth, avoiding unreasonable oscillation, and the time step index t is used to cover all adjacent triplets, the prediction value y t is adjusted through back propagation, while maintaining prediction accuracy and smoothness, penalizing the second derivative (curvature) of the prediction result, reflecting the local curvature change of the prediction result. At the beginning of training, L pred is dominant, and the weighting coefficient I1 can increase to accelerate convergence and reduce prediction error. In the middle of training, the weight coefficients I2 and I3 of L KL and L smooth rise to prevent overfitting, without the need for manual adjustment of loss weights, dynamically balancing multiple objectives according to the training state; adapt to different data distributions (such as smooth wind conditions, sudden wind conditions), improve model generalization ability; while ensuring accuracy, reduce hyperparameter tuning cost.

[0080] The method for fine-tuning a pre-trained large model for wind power prediction of the present application effectively improves the accuracy and model adaptability of wind power prediction by dynamically adjusting the parameters of the pre-trained large model and the hierarchical transfer learning strategy. Specifically, the spatiotemporal feature fusion technology of the sliding window is used to capture the spatiotemporal correlation characteristics of multi-dimensional wind speed data, and the dynamic normalization step parameter generated by the wind speed gradient of the adjacent time window is used to adaptively adjust the learning rate and parameter update strategy of the model, solving the problem of imbalance between convergence speed and accuracy when the wind speed suddenly changes in the traditional fixed step length; the bottom feature extraction layer is frozen by hierarchical transfer learning to retain the general time series features of the pre-trained model, and the top network is fine-tuned in multiple stages by an adaptive optimizer based on dynamic weight averaging, with KL divergence constraint to maintain parameter distribution consistency, reducing the dependence on domain data and enhancing the adaptability to wind power scenarios; the improved Adam algorithm combines the momentum mechanism and the regularization term, and the time series smoothing constraint in the multi-objective joint loss function, balances the prediction accuracy and smoothness under the dynamic learning rate regulation, effectively suppresses overfitting and improves the robustness of the model to complex non-stationary wind speed changes, realizes high-precision long-term wind power prediction and stable output in real-time environment.

[0081] It should be understood that, although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above-described embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0082] Based on the same inventive concept, the embodiments of the present application also provide a device for implementing the above-mentioned fine-tuning of a pre-trained large model for wind power generation prediction. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more device embodiments for fine-tuning a pre-trained large model for wind power generation prediction provided below can be referred to the limitations of the method for fine-tuning a pre-trained large model for wind power generation prediction described above, which will not be repeated here.

[0083] In one exemplary embodiment, as shown in Figure 2 a device for fine-tuning a pre-trained large model for wind power generation prediction is provided, comprising:

[0084] a data preprocessing module configured to obtain a historical multi-dimensional time series data set and generate a wind speed trend sequence by spatiotemporal feature fusion through a sliding window;

[0085] a step parameter generation module configured to generate a step adjustment parameter using a normalization function based on the wind speed gradient of adjacent time windows in the wind speed trend sequence;

[0086] a model optimization module configured to dynamically adjust the pre-trained time series prediction model based on the step adjustment parameter, and adjust the weight matrix parameter of the top network of the time series prediction model through an adaptive optimization algorithm combined with a hierarchical transfer learning strategy, to obtain an optimized time series prediction model;

[0087] a real-time prediction module configured to obtain a real-time multi-dimensional time series data set and generate a wind power generation prediction result using the optimized time series prediction model.

[0088] In one embodiment, the step parameter generation module 12 is further configured to:

[0089] calculate the wind speed mean vector of the adjacent time windows to obtain a vector sequence {Vt} and {V t+1}, wherein t represents a time window index;

[0090] based on the vector sequence {V t} and {V t+1}, a wind speed gradient matrix ΔV = ||V t+1 -V t ||2 is constructed through a difference operation under a gradient direction feature;

[0091] a dynamic normalization function based on historical gradient extreme values is constructed based on the wind speed gradient matrix ΔV wherein V max represents a maximum wind speed change amount statistically in a sliding window period, and a represents a smoothing coefficient;

[0092] the dynamic normalization function F is linearly combined with a preset basic step size to generate a step size adjustment parameter Step size = β·(1+F), wherein β represents the preset basic step size.

[0093] In one embodiment, as shown in Figure 3 the model optimization module 13 further includes a dynamic parameter adjustment unit, configured to:

[0094] based on the step size adjustment parameter, a corresponding dynamic learning rate parameter set is generated through a parameter mapping function, and the parameter mapping function satisfies: A t = λ1·tanh(λ2·Step size )+λ3, wherein A t represents a dynamic learning rate at time t, Step size represents the step size adjustment parameter, λ1, λ2, and λ3 respectively represent adjustable coefficients initialized through transfer learning;

[0095] According to the dynamic learning rate parameter set, the convolution kernel weight of the convolutional neural network layer is updated, and the update formula used is: wherein W t represents a weight matrix of the tth iteration, represents a loss function gradient, and ρ represents a weight decay coefficient, and represents a Hadamard product;

[0096] The momentum update of the hidden state transition matrix of the recurrent neural network layer is performed through a sliding average algorithm, and the update amount calculation satisfies: wherein μ represents a momentum factor, ΔH t represents a matrix update amount at time t, and H t represents a hidden state transition matrix at time t.

[0097] In one embodiment, as shown inFigure 3 As shown, the model optimization module 13 further includes a hierarchical transfer learning unit for:

[0098] dividing the network structure of the time series prediction model into a bottom feature extraction layer and a top prediction adaptation layer according to the hierarchical depth;

[0099] freezing the network parameters of the bottom feature extraction layer, and dynamically optimizing the full connection parameters of the top prediction adaptation layer through a preset loss function to obtain an adaptation layer meta-parameter set;

[0100] adopting an adaptive optimizer based on a meta-learning strategy to perform multi-stage progressive fine-tuning on the adaptation layer meta-parameter set to obtain an optimized time series prediction model, wherein the multi-stage progressive fine-tuning includes a multi-step optimization process of at least two parameter adjustment stages, the first stage adopts historical wind speed data distribution fitting for parameter initialization, the second stage performs online gradient correction through real-time data flow, and the parameter distribution consistency between the time series prediction model in the transfer learning process and the optimized time series prediction model is maintained through a KL divergence constraint term.

[0101] In one embodiment, the hierarchical transfer learning unit is further configured to:

[0102] divide the convolutional layer and the recurrent neural network encoder in the time series prediction model into the bottom feature extraction layer, and for the last k full connection layers in the time series prediction model, reconstruct the top prediction adaptation layer by introducing a learnable domain embedding vector; wherein the hierarchical level of the full connection layer group is dynamically determined by the following formula: wherein, D in represents the input feature dimension, D out represents the output prediction dimension, and D hidden represents the hidden layer reference dimension of the time series prediction model.

[0103] In one embodiment, the hierarchical transfer learning unit is further configured to: adopt an improved Adam algorithm based on dynamic weight averaging to construct an adaptive optimizer, and the parameter update formula satisfies:

[0104]

[0105] wherein, represents the dynamic base learning rate, Ω represents the learning rate scaling factor, β represents the preset base step, Step size represents the step adjustment parameter, z1 represents the first moment decay rate, γ represents the parameter regularization coefficient, g t represents the gradient at time t, and respectively represent the first and second moment estimates after bias correction, ε represents a numerical stability constant, and θ tθt represents the adapted layer element parameter set at time t t+1 θt+1 represents the adapted layer element parameter set after one update t θt+1 represents the adapted layer element parameter set after one update

[0106] In one embodiment, the hierarchical transfer learning unit is further configured to adopt the following multi-objective joint optimization formula:

[0107] L total = I1L pred + I2L KL + I3L smooth

[0108] wherein L pred represents the mean square error loss of wind speed prediction, L KL represents the KL divergence constraint term of the parameter distribution between the time series prediction model and the optimized time series prediction model, L smooth represents the time series smoothing constraint term based on the second derivative of the adjacent prediction result, and I1, I2 and I3 represent the learnable adaptive weighting coefficients.

[0109] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of fine-tuning the pre-trained large model for wind power prediction as described above when executing the computer program.

[0110] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts are described in the method embodiment. The device embodiments described above are only schematic, and the components described as separate components can or can not be physically separate, and the components displayed as a unit can or can not be a physical unit, i.e. they can be located in one place, or distributed on multiple network units. Some or all of the modules can be selected to achieve the purpose of the present disclosure according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0111] The above-described embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be noted that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A method for fine-tuning a pre-trained large model for wind power generation prediction, characterized in that, The method comprises: acquiring a historical multi-dimensional time series data set, and generating a wind speed trend sequence by spatiotemporal feature fusion through a sliding window; based on the wind speed gradient of adjacent time windows in the wind speed trend sequence, generating a step adjustment parameter by using a normalization function; based on the step adjustment parameter, dynamically adjusting the pre-trained time series prediction model, and combining a hierarchical transfer learning strategy to adjust the top layer network parameters of the time series prediction model through an adaptive optimization algorithm to obtain an optimized time series prediction model; acquiring a real-time multi-dimensional time series data set, and generating a wind power generation prediction result by using the optimized time series prediction model; wherein, the model optimization module is further used for: dividing the network structure of the time series prediction model into a bottom layer feature extraction layer and a top layer prediction adaptation layer according to the hierarchical depth; freezing the network parameters of the bottom layer feature extraction layer, and dynamically optimizing the fully connected parameters of the top layer prediction adaptation layer through a preset loss function to obtain an adaptation layer meta parameter set; using an adaptive optimizer based on a meta learning strategy to perform multi-stage progressive fine-tuning on the adaptation layer meta parameter set to obtain an optimized time series prediction model, wherein the multi-stage progressive fine-tuning includes a multi-step optimization process of at least two parameter adjustment stages, the first stage uses historical wind speed data distribution fitting for parameter initialization, the second stage performs online gradient correction through real-time data flow, and the parameter distribution consistency between the time series prediction model and the optimized time series prediction model in the transfer learning process is maintained through a KL divergence constraint term.

2. The method of claim 1, wherein, the step adjustment parameter is generated by using a normalization function based on the wind speed gradient of adjacent time windows in the wind speed trend sequence, comprising: calculating the wind speed mean vector of the adjacent time window, obtaining a vector sequence {V t} and {V t+1}, wherein t represents the time window index; Based on the vector sequence {V t} and {V t+1}, the wind speed gradient matrix ΔV = ||V t+1 -V t ||2 is constructed by difference operation under the gradient direction feature reservation. constructing a dynamic normalization function based on historical gradient extremes based on the wind speed gradient matrix ΔV where V max denotes the maximum wind speed variation amount counted within the sliding window period, and α denotes a smoothing coefficient; combining the dynamic normalization function F with a preset base step length linearly to generate a step length adjustment parameter Step size = β · (1 + F), wherein β represents a preset base step length.

3. The method of claim 1, wherein, the dynamic parameter adjustment of the pre-trained time series prediction model based on the step adjustment parameter comprises: Based on the step adjustment parameter, a corresponding dynamic learning rate parameter set is generated through a parameter mapping function, and the parameter mapping function satisfies: A t = λ1·tanh(λ2·Step size )+ λ3, wherein A t represents the dynamic learning rate at time t, Step size represents the step adjustment parameter, λ1, λ2, λ3 respectively represent adjustable coefficients initialized through transfer learning; Gradient updates are performed on the convolution kernel weights of the convolutional neural network layer according to the dynamic learning rate parameter set, and the update formula used is: wherein, W t denotes the weight matrix of the t-th iteration, denotes the loss function gradient, ρ denotes the weight decay coefficient, and ⊙ denotes the Hadamard product; The momentum update is performed on the hidden state transition matrix of the recurrent neural network layer by a sliding average algorithm, and the update amount calculation satisfies: Wherein, μ represents a momentum factor, ΔH t represents the matrix update amount at time t, H t represents the hidden state transition matrix at time t.

4. The method of claim 1, wherein, the network structure of the time series prediction model is divided into a bottom layer feature extraction layer and a top layer prediction adaptation layer according to the hierarchical depth, comprising: The convolutional layer and the recurrent neural network encoder in the time series prediction model are divided into a bottom feature extraction layer, and for the last k fully connected layers in the time series prediction model, a learnable domain embedding vector is introduced to reconstruct a top prediction adaptation layer; wherein the level of the fully connected layer is dynamically determined by the following formula: Wherein, D in represents the input feature dimension, D out represents the output prediction dimension, D hidden represents the time series prediction model hidden layer reference dimension.

5. The method of claim 1, wherein, the adaptive optimizer uses an improved Adam algorithm based on dynamic weight averaging, and the parameter update formula satisfies: wherein, denotes a dynamic base learning rate, Ω denotes a learning rate scaling factor, β denotes a preset base step size, Step size denotes a step size adjustment parameter, z1 denotes a first moment decay rate, γ denotes a parameter regularization coefficient, g t denotes a gradient at time t, denotes a second moment estimation of the gradient at time t, denotes a first moment estimation of the gradient at time t-1, ε denotes a numerical stability constant, θ t denotes an adapted layer element parameter set at time t, θ t+1 denotes an adapted layer element parameter set at time t, θ t an adapted layer element parameter set after one update.

6. The method of claim 1, wherein, the preset loss function uses the following multi-objective joint optimization formula: L total = I1L pred + I2L KL + I3L smooth wherein L pred represents the mean square error loss of wind speed prediction, L KL represents the KL divergence constraint term of parameter distribution between the time series prediction model and the optimized time series prediction model, L smooth represents the time series smoothing constraint term based on the second derivative of adjacent prediction results, and I1, I2 and I3 represent the learnable adaptive weighting coefficients.

7. A device for fine-tuning a pre-trained large model for wind power generation prediction, characterized in that, the device comprises: a data preprocessing module for acquiring a historical multi-dimensional time series data set, and generating a wind speed trend sequence by spatiotemporal feature fusion through a sliding window; a step parameter generation module for generating a step adjustment parameter by using a normalization function based on the wind speed gradient of adjacent time windows in the wind speed trend sequence; a model optimization module for dynamically adjusting a pre-trained time series prediction model based on the step adjustment parameter, and combining a hierarchical transfer learning strategy to adjust the weight matrix parameters of the top layer network of the time series prediction model through an adaptive optimization algorithm to obtain an optimized time series prediction model; a real-time prediction module for acquiring a real-time multi-dimensional time series data set, and generating a wind power generation prediction result by using the optimized time series prediction model; wherein, the model optimization module is further used for: The network structure of the time series prediction model is divided into a bottom feature extraction layer and a top prediction adaptation layer according to a hierarchical depth; The network parameters of the bottom feature extraction layer are frozen, and the full connection parameters of the top prediction adaptation layer are dynamically optimized through a preset loss function to obtain an adaptation layer meta-parameter set; An adaptive optimizer based on a meta-learning strategy is used to perform multi-stage progressive fine-tuning on the adaptation layer meta-parameter set to obtain an optimized time series prediction model, wherein the multi-stage progressive fine-tuning includes a multi-step optimization process of at least two parameter adjustment stages, parameter initialization is performed through historical wind speed data distribution fitting in the first stage, online gradient correction is performed through real-time data flow in the second stage, and the parameter distribution consistency between the time series prediction model and the optimized time series prediction model in the transfer learning process is maintained through a KL divergence constraint term.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

Images