Short-term wind power federated learning distributed prediction method for unbalanced data

Abstract

The invention discloses an unbalanced data-oriented short-term wind power federated learning distributed prediction method, and relates to the field of artificial intelligence application of a power system. Abnormal data in the regional multi-wind-power-plant data set is cleaned through a quartile method and a cubic spline interpolation method; a local model under a federated learning framework is constructed based on a VMD-TPA-LSTM algorithm, space-time coupling features in a data set are fully extracted, and prediction precision is improved; and finally, optimizing a local model parameter uploading process of the federated learning framework based on a differential privacy protection mechanism, optimizing an aggregation direction of all local model parameters based on a Scaffold mechanism, finally completing distributed training, and perfecting a global model of federated learning. According to the method, important space-time coupling characteristics of Non-IID and unbalanced and insufficient wind power data sets are fully extracted, the limitation of multi-source numerical weather forecast NWP data acquisition and fusion is solved, and the power prediction precision of multiple wind power plants in a region is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence applications in power systems, and in particular to a short-term wind power federated learning distributed prediction method for unbalanced data. Background Art

[0002] With the improvement of environmental awareness and the development of renewable energy technology, the proportion of distributed energy such as wind power and photovoltaics in my country's energy structure has increased. However, the inherent characteristics of wind power generation, such as climate factors such as wind speed, are the core factors that determine the power output of wind power generation. Its changes are not only time-varying, but also spatially non-uniform. This dual characteristic leads to significant instability in the power output of wind power generation. Therefore, in order to improve the stability and reliability of the power grid, it is necessary to use advanced prediction technologies and methods to achieve accurate prediction of the power output of multiple wind farms in the region, so as to ensure the safe, stable and efficient operation of the power grid.

[0003] In recent years, deep learning in machine learning has gradually attracted attention. Data-driven deep learning algorithms optimize prediction models by training large amounts of data, automatically extracting features from data and learning complex mapping relationships. They can not only handle nonlinear and non-stationary meteorological data features, but also adapt to prediction needs at different time and spatial scales. The literature (Fu Yang, Ren Zixu, Wei Shurong, et al. Ultra-short-term power prediction of offshore wind power based on improved LSTM-TCN model [J]. Proceedings of the CSEE, 2022, 42(12): 4292-4303.) proposed a prediction method that integrates LSTM (Long Short-Term Memory, LSTM for short) and temporal convolutional neural network, which fully considers the spatial distribution characteristics of wind farm groups. The example results show that this method performs well in dealing with sudden changes in offshore wind speed conditions, has high adaptability, and can improve the accuracy of offshore wind power output prediction models. The literature (Zhou Jiakang. Research on short-term power forecasting method of wind power cluster based on deep learning [D]. North China Electric Power University (Beijing), 2022.) proposed a centralized short-term power forecasting model for wind farm clusters based on convolutional long short-term memory network. Compared with the field modeling model, this model shows higher prediction accuracy. From the above literature analysis, it can be seen that the data-driven deep learning method improves the power prediction accuracy of regional centralized multi-wind farms by inputting a large data set. However, the large training set also makes the data-driven model have the following two limitations: (1) The spatiotemporal characteristics of wind power data are difficult to effectively mine, and it is necessary to consider the complex interactive relationship of multiple variables in the time and space dimensions at the same time, which limits the prediction ability of the model in complex scenarios; (2) Different data sets of regional multi-wind farms have Non-IID (Non-Independent Identically Distribution, referred to as Non-IID) distribution and imbalance and insufficiency. Traditional centralized training will cause large prediction errors. In addition, the meteorological information and power information in the multi-source data set are usually distributed in different geographical locations and operating units. Data collection, integration and transmission are difficult, resulting in insufficient training data for the data-driven model.

[0004] Based on the above problems, a method is needed to integrate unbalanced data and perform distributed prediction of wind power. Summary of the invention

[0005] In order to solve the problem that a large training set also causes limitations in the data-driven model, restricts the model's prediction ability in complex scenarios and the training data of the data-driven model is insufficient, the present invention provides a short-term wind power federated learning distributed prediction method for unbalanced data.

[0006] The present invention is implemented by the following technical solution: a short-term wind power federated learning distributed prediction method for unbalanced data, comprising the following steps:

[0007] Phase 1: Data cleaning, including abnormal data deletion and missing data filling, using a data cleaning method based on the quartile method and cubic spline interpolation method; for the abnormal values ​​in the original wind power data set, the quartile method is first used to delete the abnormal data in the regional multi-wind farm data set; then the cubic spline interpolation method is used to solve the three-moment equations according to the continuous data, and the curve function group is calculated to fill the deleted abnormal data; specifically:

[0008] S1-1: Abnormal data deletion: Delete the abnormal data in the regional multi-wind farm data set. Use the quartile method to sort the data set from small to large and divide it into four equal parts, each containing the same number of data points. The lower quartile Q1 represents the boundary of the first 25% of the data, the median Q2 is the middle value of the data set, and the upper quartile Q3 represents the boundary of the last 25% of the data. By calculating the interquartile range IQR, that is, the difference between Q3 and Q1, the normal range of the data is defined. Data points outside this range are considered abnormal values. For the original data sample X = [x1, x2, …, x n ], the specific calculation formulas for Q1, Q2 and Q3 are as follows:

[0009]

[0010]

[0011] Where, e = 0, 1, 2, ...; and They represent the rounding up of one quarter and three quarters of the original data respectively; the interquartile range IQR obtained based on the calculated values ​​of Q1 and Q3 is:

[0012] IQR=Q3-Q1 (4)

[0013] The normal data range for the original data sample is:

[0014] [F1,F2]=[Q1-1.5IQR,Q3+1.5IQR] (5)

[0015] S1-2: Missing data filling: Directly removing abnormal data in step 1) will seriously affect the continuity of the multivariate sequence in the wind power data set. Therefore, the deleted abnormal data is filled based on the cubic spline interpolation method. The specific calculation process is as follows:

[0016] For the function y = f(x), the independent variable interval is [a, b], which is equally divided by n + 1 sampling points, that is, a = x0 < x1 < ... x n = b, y = f(x i ),i=0,1,…,n; if any piecewise function z(x) satisfies the following conditions, then the function is a cubic spline interpolation function:

[0017] ① It has a continuous second-order derivative on the interval [a,b];

[0018] ②In each subinterval [x i ,x i+1 ] is at most a cubic polynomial;

[0019] ③The function satisfies z(x i )=y i ;

[0020] Define function y node x i The first and second order derivatives at are s i and S i ,Right now

[0021] f'(x i )=s i (6)

[0022] f"(x i )=S i (7)

[0023] For the function y=f(x), its second-order derivative f"(x) is in the interval [x i ,x i+1 ] is a linear function, and the mathematical expression is:

[0024]

[0025] h i =x i+1 -x i (9)

[0026] The mathematical expression of the cubic spline is the quadratic integral of f"(x):

[0027]

[0028] Phase II: Propose a local model for short-term wind power prediction based on VMD-TPA-LSTM: First, use variational mode decomposition VMD (Variational Mode Decomposition, VMD for short) to decompose the small amount of outliers and noise that still exist after the original data set is cleaned into component sequences of different frequencies to enrich the input information of the model; secondly, use the temporal pattern attention mechanism TPA (Temporal Pattern Attention Mechanism, TPA for short) to automatically extract and weight the important features of the VMD component sequence input matrix; finally, use the long short-term memory LSTM (Long Short-Term Memory, LSTM for short) network to capture the long-term dependencies in the time series, extract the spatiotemporal coupling features, and perform short-term wind power prediction; provide a local model foundation for distributed prediction under the subsequent federated learning framework; specifically:

[0029] S2-1: VMD decomposes the original data: VMD decomposes the complex multi-component signal into several modal components with different center frequencies; VMD can solve some problems existing in traditional signal decomposition methods, such as modal aliasing, end effect, etc. VMD decomposes the original time series u0(t) into multiple components u with limited bandwidth k (t), for every u k (t) Perform Hilbert transform to obtain the one-sided spectrum:

[0030]

[0031] Where, δ(t) is the unit pulse function; is the impulse response; each component and operator Multiply them together to modulate all components to the corresponding baseband:

[0032]

[0033] Calculate the minimum square norm L of formula (12) 2 , the constrained variational model is expressed as follows:

[0034]

[0035] In the formula, u k ={u1,u2,…,u k} represents the decomposed components, w k ={w1,w2,…,w k} represents the center frequency of each component. To satisfy the requirement that the sum of all reconstructed components is equal to the original input time series u0(t), the constraints are:

[0036]

[0037] By using the quadratic penalty term and the Lagrangian multiplier method, the augmented Lagrangian function is introduced to transform the constrained problem into an unconstrained problem, which not only ensures the strictness of the constraints, but also ensures the accuracy of the decomposition sequence reconstruction under Gaussian noise environment. The expression is:

[0038]

[0039] Where τ(t) is the Lagrangian multiplier; α is the quadratic penalty factor; the Alternate Direction Method of Multipliers (ADMM) is used to continuously update each component and the center frequency of each component, and finally the optimal solution to the original sequence decomposition variation problem is obtained; the component u k (t), center frequency w k , the update process of the Lagrange multiplier τ(t) is as follows:

[0040]

[0041] In the formula, are u0(t), u k (t), Fourier transform of τ(t); w is the frequency; n is the number of iterations; λ is the noise. When the signal contains strong noise, set λ = 0 to achieve better denoising effect; repeat the calculation of formulas (16), (17), and (18) until the accuracy μ is met and stop the iteration, and finally obtain k modal components; the iteration condition is:

[0042]

[0043] S2-2: TPA-LSTM extracts spatiotemporal coupling features for short-term wind power forecasting: TPA weights each time step of the historical input sequence, increases the weight of important elements, and reduces the weight of unimportant elements, so that the LSTM model pays more attention to data features related to the target prediction; the original wind power data set is X = [X1, X2, …X n ], n is the number of input features; using the VMD decomposition algorithm, the input layer is decomposed according to the preset decomposition number m to obtain the input matrix of the VMD decomposition layer; in the LSTM iteration process, the hidden layer state of the forward unit is h t , the unit status information is s t , the time attention weight vector l corresponding to each historical moment at the current moment t for:

[0044] l t = relu(W d h t +bd ) (20)

[0045] In the formula, is the time window length of the input sequence; W d is the trainable weight matrix; b d Bias vector for calculating temporal attention weights;

[0046] Use the Softmax function for normalization to get the time attention weight β t :

[0047]

[0048] In the formula, For the The attention weight value at the moment, Comprehensive timing information status c t for:

[0049]

[0050] Finally, the comprehensive time series state is input into the fully connected layer, and the prediction result of the output layer is obtained through VMD component reconstruction. Finally, VMD-TPA-LSTM is established as the local model under the federated learning framework.

[0051] Phase 3: Distributed prediction of short-term wind power based on horizontal federated learning: First, each wind farm in the region completes distributed training in the form of "local training-parameter upload", and uses the differential privacy protection mechanism to improve the security of private data of wind farms in the local area, avoiding the concerns of various data sources about data privacy security, thus breaking the limitation of data barriers on centralized prediction; secondly, the Scaffold mechanism is used to solve the problem of decreased accuracy caused by insufficient data or uneven data distribution of multiple data sources, and improve the prediction performance and robustness of the global model; finally, the regression model evaluation index is used to verify the effectiveness of distributed training of the proposed horizontal federated learning framework. Based on the federated learning framework, each wind farm can use private data sets for local training. After the local training is completed, each wind farm uploads the model parameters to the federated central server, which is responsible for aggregating these parameters to update the global wind power prediction model. The advantage of this method is that the data set of the wind farm does not need to leave the local device during the entire training process, thus ensuring the security and privacy of the data. At the same time, the data sets of multiple wind farms in the region are non-independent and identically distributed (Non-IID) and unbalanced and insufficient. Through federated distributed training, the global model can learn the data characteristics that a single wind farm model in this region cannot capture, thereby improving the accuracy and generalization ability of the prediction. Specifically:

[0052] S3-1: Differential privacy protection mechanism: Based on differential privacy Laplace noise, the local model parameter transmission mechanism is improved. When training the local model in the wind farm, the protection noise is injected into the gradient parameter, so that when all local model parameters are transmitted to the central server, the data privacy information cannot be inverted and restored. The specific principles are as follows:

[0053] The Laplace distribution is centered at 0 and scaled by b, and its probability density distribution is:

[0054]

[0055] For p(x|b), it can be written as Lap(b), which represents the random variable x~Lap(b); for the function f:D→R of the wind power data set D d , for any input a, the Laplace mechanism is:

[0056]

[0057] Where ε represents the privacy budget; the Laplace mechanism satisfies the definition of (ε,0)-differential privacy, assuming m∈D, n∈D', satisfying ||mn||1≤1; p m represents the probability density function H(m), pn Express it as a probability density function H(n) and compare the probability density of the two at any time point t:

[0058]

[0059] Formula (26) is expressed as the probability of t, and the k′ outputs are independent of each other. The equation is simplified as follows:

[0060]

[0061] Using the norm deformation, and according to ||f(m) i -f(n) i ||1≤1 gives:

[0062]

[0063] S3-2: The algorithm flow of the Scaffold central server aggregation strategy is as follows:

[0064] a: Input: total number of clients N; number of clients selected for global iteration K; number of global iterations T; global update learning rate η g ; Global control variable c; Client number k″=1,2,…,n; Data volume on each client n k″ ; Client local update iteration number E; local update learning rate η l ; Control variable c k ;

[0065] b: Output: global model parameters w T ;

[0066] b1: The algorithm flow of the central server is as follows:

[0067] 1: Initialize the global model weight parameter w0;

[0068] 2: for t=0,1,…,T-1do;

[0069] 3: S t ←Select K client sets;

[0070] 4: for k″∈S t do;

[0071] 5:

[0072] 6: end for;

[0073] 7:

[0074] 8: w t+1 ←wt +η g Δw t+1 ,

[0075] 9: end for;

[0076] 10: return w T ;

[0077] b2: Client algorithm process Client:

[0078] 1: The last round of global parameters was sent to update the client parameters;

[0079] 2: for k″∈S t do;

[0080] 3: for i=0,1,…,E-1do

[0081] 4: Calculate the gradient;

[0082] 5: end for;

[0083] 6: or

[0084] 7: Send to the central server;

[0085] 8:

[0086] 9: end for;

[0087] 10: return c k″ ;

[0088] S3-3: Model performance evaluation indicators: Root mean square error RMSE (RMSE) and mean absolute error MAE (MAE) are used as regression model error measurement indicators; the calculation formulas of RMSE and MAE are as follows:

[0089]

[0090] In the formula, y i represents the true value; represents the predicted value; n represents the number of samples.

[0091] A short-term wind power federated learning distributed prediction method for unbalanced data is designed in the present invention, which is summarized as follows: the abnormal data in the regional multi-wind farm data set is cleaned by the quartile method and cubic spline interpolation method; the local model under the federated learning framework is constructed based on the VMD-TPA-LSTM algorithm to fully extract the spatiotemporal coupling characteristics in the data set and improve the prediction accuracy; finally, the local model parameter uploading process of the federated learning framework is optimized based on the differential privacy protection mechanism, and the aggregation direction of all local model parameters is optimized based on the Scaffold mechanism, and finally the distributed training is completed to improve the global model of federated learning.

[0092] Compared with the prior art, the present invention has the following beneficial effects: the short-term wind power federated learning distributed prediction method for unbalanced data provided by the present invention improves the following two problems: (1) The regional multi-wind farm data set has the characteristics of Non-IID and insufficient imbalance, which makes the power prediction of the regional multi-wind farm complex and inaccurate, causing serious wind abandonment; (2) The existing data-driven wind power prediction methods are often based on a single data source and a centralized data processing mode, while the meteorological information and power information in the regional multi-wind farm data set are usually distributed in different geographical locations and operating units, resulting in the problem of "data islands" in the collection, integration and transmission of multi-source data information. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 It is a flow chart of the technical solution proposed by the present invention.

[0094] Figure 2 This is the result of data cleaning.

[0095] Figure 3 It is a radar chart of the training results of different wind power prediction models.

[0096] Figure 4 It is the RMSE histogram of distributed training of the federated learning global model.

[0097] Figure 5 It is the MAE histogram of distributed training of the federated learning global model.

[0098] Figure 6 The effect of distributed training using the Scaffold aggregation strategy for different numbers of wind farms. DETAILED DESCRIPTION

[0099] The present invention is further described below in conjunction with specific embodiments.

[0100] This embodiment is aimed at the scenario of short-term power forecasting for eight wind farms in western China. The numerical weather prediction (NWP) data sets of different wind farms are shown in Table 1.

[0101] Table 1

[0102] Wind Farm Time span Completeness of the dataset 1 1.1-12.31 All NWP data for some dates are missing 2 1.1-12.31 Historical power data is missing for some dates 3 1.1-12.31 Historical power data is missing for some dates 4 1.1-12.31 70-meter wind speed data is missing for some dates 5 1.1-12.31 Lack of 30m and 50m wind speed data for different dates 6 1.1-12.31 Historical power data is missing for some dates 7 1.1-12.31 Some power and NWP data are missing 8 1.1-12.31 Historical power data is missing for some dates

[0103] The short-term wind power federated learning distributed prediction method for unbalanced data provided by the present invention is used to perform power prediction. The process is as follows: Figure 1 As shown, the following steps are included:

[0104] Phase 1: Data cleaning, including abnormal data deletion and missing data filling, using a data cleaning method based on the quartile method and cubic spline interpolation method; for the abnormal values ​​in the original wind power data set, the quartile method is first used to delete the abnormal data in the regional multi-wind farm data set; then the cubic spline interpolation method is used to solve the three-moment equations according to the continuous data, and the curve function group is calculated to fill the deleted abnormal data; specifically:

[0105] S1-1: Abnormal data deletion: Delete the abnormal data in the regional multi-wind farm data set. Use the quartile method to sort the data set from small to large and divide it into four equal parts, each containing the same number of data points. The lower quartile Q1 represents the boundary of the first 25% of the data, the median Q2 is the middle value of the data set, and the upper quartile Q3 represents the boundary of the last 25% of the data. By calculating the interquartile range IQR, that is, the difference between Q3 and Q1, the normal range of the data is defined. Data points outside this range are considered abnormal values. For the original data sample X = [x1, x2, …, x n ], the specific calculation formulas for Q1, Q2 and Q3 are as follows:

[0106]

[0107] Where, e = 0, 1, 2, ...; and They represent the rounding up of one quarter and three quarters of the original data respectively; the interquartile range IQR obtained based on the calculated values ​​of Q1 and Q3 is:

[0108] IQR=Q3-Q1 (4)

[0109] The normal data range for the original data sample is:

[0110] [F1,F2]=[Q1-1.5IQR,Q3+1.5IQR] (5)

[0111] S1-2: Missing data filling: Directly removing abnormal data in step 1) will seriously affect the continuity of the multivariate sequence in the wind power data set. Therefore, the deleted abnormal data are filled based on the cubic spline interpolation method. The box plot of the filled data is shown in the figure below. Figure 2 The specific calculation process is as follows:

[0112] For the function y = f(x), the independent variable interval is [a, b], which is equally divided by n + 1 sampling points, that is, a = x0 < x1 < ... x n = b, y = f(x i ),i=0,1,…,n; if any piecewise function z(x) satisfies the following conditions, then the function is a cubic spline interpolation function:

[0113] ① It has a continuous second-order derivative on the interval [a,b];

[0114] ②In each subinterval [x i ,x i+1 ] is at most a cubic polynomial;

[0115] ③The function satisfies z(x i )=y i ;

[0116] Define function y node x i The first and second order derivatives at are s i and S i ,Right now

[0117] f'(x i )=s i (6)

[0118] f"(x i )=S i (7)

[0119] For the function y=f(x), its second-order derivative f"(x) is in the interval [x i ,x i+1 ] is a linear function, and the mathematical expression is:

[0120]

[0121] h i =x i+1 -x i (9)

[0122] The mathematical expression of the cubic spline is the quadratic integral of f"(x):

[0123]

[0124] Phase II: Propose a local model for short-term wind power prediction based on VMD-TPA-LSTM: First, use variational mode decomposition VMD to decompose the small amount of outliers and noise that still exist after the original data set is cleaned into component sequences of different frequencies to enrich the input information of the model; automatically extract and weight the important features of the VMD component sequence input matrix; finally, use the long short-term memory LSTM network to capture the long-term dependencies in the time series, extract the spatiotemporal coupling features, and perform short-term wind power prediction; provide a local model foundation for distributed prediction under the subsequent federated learning framework; specifically:

[0125] S2-1: VMD decomposes the original data: VMD decomposes the complex multi-component signal into several modal components with different center frequencies; VMD can solve some problems existing in traditional signal decomposition methods, such as modal aliasing, end effect, etc. VMD decomposes the original time series u0(t) into multiple components u with limited bandwidth k (t), for every u k (t) Perform Hilbert transform to obtain the one-sided spectrum:

[0126]

[0127] Where, δ(t) is the unit pulse function; is the impulse response; each component and operator Multiply them together to modulate all components to the corresponding baseband:

[0128]

[0129] Calculate the minimum square norm L of formula (12) 2 , the constrained variational model is expressed as follows:

[0130]

[0131] In the formula, u k ={u1,u2,…,u k} represents the decomposed components, w k ={w1,w2,…,w k} represents the center frequency of each component. To satisfy the requirement that the sum of all reconstructed components is equal to the original input time series u0(t), the constraints are:

[0132]

[0133] By using the quadratic penalty term and the Lagrangian multiplier method, the augmented Lagrangian function is introduced to transform the constrained problem into an unconstrained problem, which not only ensures the strictness of the constraints, but also ensures the accuracy of the decomposition sequence reconstruction under Gaussian noise environment. The expression is:

[0134]

[0135] In the formula, τ(t) is the Lagrangian multiplier; α is the quadratic penalty factor; the ADMM method is used to continuously update each component and the center frequency of each component, and finally the optimal solution of the original sequence decomposition variation problem is obtained; the component u k (t), center frequency w k , the update process of the Lagrange multiplier τ(t) is as follows:

[0136]

[0137] In the formula, are u0(t), u k (t), Fourier transform of τ(t); w is the frequency; n is the number of iterations; λ is the noise. When the signal contains strong noise, set λ = 0 to achieve better denoising effect; repeat the calculation of formulas (16), (17), and (18) until the accuracy μ is met and stop the iteration, and finally obtain k modal components; the iteration condition is:

[0138]

[0139] S2-2: TPA-LSTM extracts spatiotemporal coupling features for short-term wind power forecasting: TPA weights each time step of the historical input sequence, increases the weight of important elements, and reduces the weight of unimportant elements, so that the LSTM model pays more attention to data features related to the target prediction; the original wind power data set is X = [X1, X2, …X n ], n is the number of input features; using the VMD decomposition algorithm, the input layer is decomposed according to the preset decomposition number m to obtain the input matrix of the VMD decomposition layer; in the LSTM iteration process, the hidden layer state of the forward unit is h t , the unit status information is s t , the time attention weight vector l corresponding to each historical moment at the current moment t for:

[0140] l t = relu(W d h t +b d ) (20)

[0141] In the formula, is the time window length of the input sequence; W d is the trainable weight matrix; b d Bias vector for calculating temporal attention weights;

[0142] Use the Softmax function for normalization to get the time attention weight β t :

[0143]

[0144] In the formula, For the The attention weight value at the moment, Comprehensive timing information status c t for:

[0145]

[0146] Finally, the comprehensive time series state is input into the fully connected layer, and the prediction result of the output layer is obtained through VMD component reconstruction. Finally, VMD-TPA-LSTM is established as the local model under the federated learning framework.

[0147] The regression error indicators of various wind power prediction models are shown in Table 2. In order to more comprehensively evaluate the performance of different models in wind power prediction, we compare the regression error indicators of different models through radar charts, such as Figure 3 As shown. Through the analysis of the radar chart, it can be seen that under the same training data and similar model parameter conditions, the VMD-TPA-LSTM model has shown significant advantages in all indicators, and its RMSE and MAE indicators are both the smallest, indicating that the deviation between the prediction results of the model and the actual values ​​is the smallest, and the prediction performance is the best. This result not only proves the efficiency and accuracy of the VMD-TPA-LSTM model in wind power prediction, but also further verifies its feasibility as a local model for federated learning.

[0148] Table 2

[0149] Model RMSE MAE TPA-LSTM 0.252 0.227 LSTM 0.359 0.266 TPA-GRU 0.311 0.244 GRU 0.395 0.275 RNN 0.354 0.262 MLP 0.446 0.346 RF 0.439 0.325 VMD-TPA-LSTM 0.201 0.195 VMD-LSTM 0.312 0.245 VMD-TPA-GRU 0.295 0.230 VMD-GRU 0.325 0.246

[0150] Phase 3: Conduct short-term distributed wind power prediction based on horizontal federated learning: Based on the federated learning framework, each wind farm can use private data sets for local training. After the local training is completed, each wind farm only needs to upload the model parameters to the federal central server. The server is responsible for aggregating these parameters to update the global wind power prediction model. The advantage of this method is that the wind farm data set does not need to leave the local device during the entire training process, thereby ensuring the security and privacy of the data. At the same time, the data sets of multiple wind farms in the region are non-independent and identically distributed (Non-IID) and unbalanced and insufficient. Through federated distributed training, the global model can learn the data characteristics that cannot be captured by a single wind farm model in this area, thereby improving the accuracy and generalization of the prediction. Specifically:

[0151] S3-1: Differential privacy protection mechanism: Based on differential privacy Laplace noise, the local model parameter transmission mechanism is improved. When training the local model in the wind farm, the protection noise is injected into the gradient parameter, so that when all local model parameters are transmitted to the central server, the data privacy information cannot be inverted and restored. The specific principles are as follows:

[0152] The Laplace distribution is centered at 0 and scaled by b, and its probability density distribution is:

[0153]

[0154] For p(x|b), it can be written as Lap(b), which represents the random variable x~Lap(b); for the function f:D→R of the wind power data set D d , for any input a, the Laplace mechanism is:

[0155]

[0156] Where ε represents the privacy budget; the Laplace mechanism satisfies the definition of (ε,0)-differential privacy, assuming m∈D, n∈D', satisfying ||mn||1≤1; p m represents the probability density function H(m), p n Express it as a probability density function H(n) and compare the probability density of the two at any time point t:

[0157]

[0158] Formula (26) is expressed as the probability of t, and the k′ outputs are independent of each other. The equation is simplified as follows:

[0159]

[0160] Using the norm deformation, and according to ||f(m) i -f(n) i ||1≤1 gives:

[0161]

[0162] S3-2: The algorithm flow of the Scaffold central server aggregation strategy is as follows:

[0163] a: Input: total number of clients N; number of clients selected for global iteration K; number of global iterations T; global update learning rate η g ; Global control variable c; Client number k″=1,2,…,n; Data volume on each client n k″ ; Client local update iteration number E; local update learning rate η l; Control variable c k ;

[0164] b: Output: global model parameters w T ;

[0165] b1: The algorithm flow of the central server is as follows:

[0166] 1: Initialize the global model weight parameter w0;

[0167] 2: for t=0,1,…,T-1do;

[0168] 3: S t ←Select K client sets;

[0169] 4: for k″∈S t do;

[0170] 5:

[0171] 6: end for;

[0172] 7:

[0173] 8: w t+1 ←w t +η g Δw t+1 ,

[0174] 9: end for;

[0175] 10: return w T ;

[0176] b2: Client algorithm process Client:

[0177] 1: The last round of global parameters is sent to update the client parameters; 2: fork″∈S t do;

[0178] 3: for i=0,1,…,E-1do

[0179] 4: Calculate the gradient;

[0180] 5: end for;

[0181] 6: or

[0182] 7: Send to the central server;

[0183] 8:

[0184] 9: end for;

[0185] 10: return c k″ ;

[0186] S3-3: Model performance evaluation indicators: Root mean square error (RMSE) and mean absolute error (MAE) are used as regression model error measurement indicators; the calculation formulas of RMSE and MAE are as follows:

[0187]

[0188] In the formula, y i represents the true value; represents the predicted value; n represents the number of samples.

[0189] For scenarios with different numbers of wind farms and different aggregation strategies in distributed training of federated learning, we verify that: (1) regional multi-wind farm distributed collaborative training can improve the accuracy of individual training; (2) global model performance based on Scaffold aggregation strategy; (3) the impact of the number of clients on prediction results. The test results are shown in Table 3. Figure 4 and Figure 5 These are the RMSE and MAE bar graphs of the distributed training of the federated learning global model. This result proves that: 1) Compared with the individual training of wind farms, the distributed training of federated learning can effectively improve the prediction performance of the model; 2) The model offset can be improved through a reasonable and effective model aggregation strategy, which shows that the Scaffold aggregation strategy is better than FedAvg, and can fully consider the differences in the Non-IID partial distribution of different wind farm data sets, and accelerate the convergence of global parameters. Figure 6 The effect diagram of distributed training using Scaffold aggregation strategy for different numbers of wind farms; 3) The number of clients of federated learning is not always positively correlated with the prediction accuracy. The reasonable number of clients should be selected based on the spatiotemporal coupling characteristics of multiple wind farms in the region, and the geographical location and wind power output correlation of different wind farms should be comprehensively considered. Because in the data set of many wind farms in the region, the huge differences in geographical location and meteorological factors will affect the aggregation direction of local model parameters. Neighboring wind farms reduce the meteorological volatility in this area by making up for each other's missing data features, while wind farms that are too far apart will increase the uncontrollability of meteorological factors, resulting in a large difference in wind power output.

[0190] Table 3

[0191]

[0192]

[0193] The scope of protection claimed by the present invention is not limited to the above specific embodiments. For those skilled in the art, the present invention may have various deformations and changes. Any modifications, improvements and equivalent substitutions made within the concept and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A short-term wind power federated learning distributed prediction method for unbalanced data, characterized by: The steps include: Phase 1: Data cleaning, including abnormal data deletion and missing data filling, using a data cleaning method based on the quartile method and cubic spline interpolation method; for the abnormal values ​​in the original wind power data set, the quartile method is first used to delete the abnormal data in the regional multi-wind farm data set; then the cubic spline interpolation method is used to solve the three-moment equations according to the continuous data to calculate the curve function group, thereby filling the deleted abnormal data; Phase II: Propose a local model for short-term wind power prediction based on VMD-TPA-LSTM: First, use variational mode decomposition (VMD) to decompose the small amount of outliers and noise that still exist in the original data set after cleaning into component sequences of different frequencies to enrich the input information of the model; Secondly, the temporal pattern attention mechanism TPA is used to automatically extract important features and assign weights to the VMD component sequence input matrix; finally, the long short-term memory LSTM network is used to capture the long-term dependencies in the time series, extract the spatiotemporal coupling features, and perform short-term wind power forecasting, providing a local model foundation for distributed forecasting under the subsequent federated learning framework; Phase III: Distributed prediction of short-term wind power based on horizontal federated learning: First, each wind farm in the region completes distributed training in the form of "local training-parameter upload", and uses the differential privacy protection mechanism to improve the security of private data of wind farms retained locally, avoiding the concerns of various data sources about data privacy security, thus breaking the limitations of data barriers on centralized prediction; secondly, the Scaffold mechanism is used to solve the problem of decreased accuracy caused by insufficient data or uneven data distribution of multiple data sources, thereby improving the prediction performance and robustness of the global model; finally, the regression model evaluation index is used to verify the effectiveness of distributed training of the proposed horizontal federated learning framework.

2. The short-term wind power federated learning distributed prediction method for unbalanced data according to claim 1 is characterized by: Phase 1 is as follows: S1-1: Abnormal data deletion: Delete the abnormal data in the regional multi-wind farm data set. Use the quartile method to sort the data set from small to large and divide it into four equal parts, each containing the same number of data points. The lower quartile Q1 represents the boundary of the first 25% of the data, the median Q2 is the middle value of the data set, and the upper quartile Q3 represents the boundary of the last 25% of the data. By calculating the interquartile range IQR, that is, the difference between Q3 and Q1, the normal range of the data is defined. Data points outside this range are considered abnormal values. For the original data sample X = [x1, x2, …, x n ], the specific calculation formulas for Q1, Q2 and Q3 are as follows: Where, e = 0, 1, 2, ...; and They represent the rounding up of one quarter and three quarters of the original data respectively; the interquartile range IQR obtained based on the calculated values ​​of Q1 and Q3 is: IQR=Q3-Q1 (4) The normal data range for the original data sample is: [F1,F2]=[Q1-1.5IQR,Q3+1.5IQR] (5) S1-2: Missing data filling: Directly removing abnormal data in step 1) will seriously affect the continuity of the multivariate sequence in the wind power data set. Therefore, the deleted abnormal data is filled based on the cubic spline interpolation method. The specific calculation process is as follows: For the function y = f(x), the independent variable interval is [a, b], which is equally divided by n + 1 sampling points, that is, a = x0 < x1 < ... x n = b, y = f(x i ),i=0,1,…,n; if any piecewise function z(x) satisfies the following conditions, then the function is a cubic spline interpolation function: ① It has a continuous second-order derivative on the interval [a,b]; ②In each subinterval [x i ,x i+1 ] is at most a cubic polynomial; ③The function satisfies z(x i )=y i ; Define function y node x i The first and second order derivatives at are s i and S i ,Right now f'(x i )=s i (6) f"(x i )=S i (7) For the function y=f(x), its second-order derivative f"(x) is in the interval [x i ,x i+1 ] is a linear function, and the mathematical expression is: h i =x i+1 -x i (9) The mathematical expression of the cubic spline is the quadratic integral of f"(x):

3. The short-term wind power federated learning distributed prediction method for unbalanced data according to claim 2 is characterized by: Phase 2 is as follows: S2-1: VMD decomposes the original data: VMD decomposes the complex multi-component signal into several modal components with different center frequencies; VMD decomposes the original time series u0(t) into multiple components u with limited bandwidth k (t), for every u k (t) Perform Hilbert transform to obtain the one-sided spectrum: Where, δ(t) is the unit pulse function; is the impulse response; each component and operator Multiply them together to modulate all components to the corresponding baseband: Calculate the minimum square norm L of formula (12) 2 , the constrained variational model is expressed as follows: In the formula, u k ={u1,u2,…,u k } represents the decomposed components, w k ={w1,w2,…,w k } represents the center frequency of each component. To satisfy the requirement that the sum of all reconstructed components is equal to the original input time series u0(t), the constraints are: By using the quadratic penalty term and the Lagrangian multiplier method, the augmented Lagrangian function is introduced to transform the constrained problem into an unconstrained problem, which not only ensures the strictness of the constraints, but also ensures the accuracy of the decomposition sequence reconstruction under Gaussian noise environment. The expression is: In the formula, τ(t) is the Lagrangian multiplier; α is the quadratic penalty factor; the multiplication operator alternating direction method ADMM is used to continuously update each component and the center frequency of each component, and finally the optimal solution of the original sequence decomposition variation problem is obtained; the component u k (t), center frequency w k , the update process of the Lagrange multiplier τ(t) is as follows: In the formula, are u0(t), u k (t), Fourier transform of τ(t); w is the frequency; n is the number of iterations; λ is the noise. When the signal contains strong noise, set λ = 0 to achieve better denoising effect; repeat the calculation of formulas (16), (17), and (18) until the accuracy μ is met and stop the iteration, and finally obtain k modal components; the iteration condition is: S2-2: TPA-LSTM extracts spatiotemporal coupling features for short-term wind power forecasting: TPA weights each time step of the historical input sequence, increases the weight of important elements, and reduces the weight of unimportant elements, so that the LSTM model pays more attention to data features related to the target prediction; the original wind power data set is X = [X1, X2, …X n ], n is the number of input features; Using the VMD decomposition algorithm, the input layer is decomposed according to the preset decomposition number m to obtain the input matrix of the VMD decomposition layer; during the LSTM iteration process, the hidden layer state of the forward unit is h t , the unit status information is s t , the time attention weight vector l corresponding to each historical moment at the current moment t for: l t =relu(W d h t +b d ) (20) In the formula, is the time window length of the input sequence; W d is the trainable weight matrix; b d Bias vector for calculating temporal attention weights; Use the Softmax function for normalization to get the time attention weight β t : In the formula, For the The attention weight value at the moment, Comprehensive timing information status c t for: Finally, the comprehensive time series state is input into the fully connected layer, and the prediction result of the output layer is obtained through VMD component reconstruction. Finally, VMD-TPA-LSTM is established as the local model under the federated learning framework.

4. The short-term wind power federated learning distributed prediction method for unbalanced data according to claim 1 is characterized by: Phase 3 is as follows: Based on the federated learning framework, each wind farm uses a private data set to perform local training. After the local training is completed, each wind farm uploads the model parameters to the federated central server, which is responsible for aggregating these parameters to update the global wind power prediction model. S3-1: Differential privacy protection mechanism: Based on differential privacy Laplace noise, the local model parameter transmission mechanism is improved. When training the local model in the wind farm, the protection noise is injected into the gradient parameter, so that when all local model parameters are transmitted to the central server, the data privacy information cannot be inverted and restored. The specific principles are as follows: The Laplace distribution is centered at 0 and scaled by b, and its probability density distribution is: For p(x|b), it can be written as Lap(b), which represents the random variable x~Lap(b); for the function f:D→R of the wind power data set D d , for any input a, the Laplace mechanism is: Where ε represents the privacy budget; the Laplace mechanism satisfies the definition of (ε,0)-differential privacy, assuming m∈D, n∈D', satisfying ||mn||1≤1; p m represents the probability density function H(m), p n Express it as a probability density function H(n) and compare the probability density of the two at any time point t: Formula (26) is expressed as the probability of t, and the k′ outputs are independent of each other. The equation is simplified as follows: Using the norm deformation, and according to ||f(m) i -f(n) i ||1≤1 gives: S3-2: The algorithm flow of the Scaffold central server aggregation strategy is as follows: a: Input: total number of clients N; number of clients selected for global iteration K; number of global iterations T; global update learning rate η g ; Global control variable c; Client number k″=1,2,…,n; Data volume on each client n k″ ; Client local update iteration number E; local update learning rate η l ; Control variable c k ; b: Output: global model parameters w T ; b1: The algorithm flow of the central server is as follows: 1: Initialize the global model weight parameter w0; 2: for t=0,1,…,T-1do; 3: S t ←Select K client sets; 4:for k″∈S t do; 5： 6: end for; 7： 8:w t+1 ←w t +n g Δw t+1 , 9: end for; 10:return w T ; b2: Client algorithm process Client: 1: The last round of global parameters was sent to update the client parameters; 2:for k″∈S t do; 3: for i=0,1,…,E-1do 4: Calculate the gradient; 5: end for; 6: or 7: Send to the central server; 8： 9: end for; 10:return c k″ ; S3-3: Model performance evaluation indicators: Root mean square error RMSE and mean absolute error MAE are used as regression model error measurement indicators; the calculation formulas of RMSE and MAE are as follows: In the formula, y i represents the true value; represents the predicted value; n represents the number of samples.

Images