Multi-station wind power prediction method based on feature embedding-transform federated learning
The method addresses wind power prediction challenges by using feature embedding and transformer models within a federated learning framework to handle diverse data sets and ensure privacy, resulting in improved accuracy and efficient computation for multi-station wind power forecasting.
Patent Information
- Application Number
- CN202411807491.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-03-14
AI Technical Summary
Existing wind power prediction methods face challenges in accurately predicting wind energy output due to data variability across multiple stations, data privacy concerns, and the limitations of traditional algorithms like LSTM, which struggle with handling diverse data sets and gradient issues during training.
The proposed method employs feature embedding and transformer-based federated learning to preprocess and predict wind power data across multiple stations, utilizing dynamic feature embeddings and transformer models for parallel computation, while maintaining data privacy through parameter-only transmission in federated learning.
This approach enhances prediction accuracy by leveraging inter-feature relationships and ensuring data privacy, overcoming the limitations of traditional methods by providing a consistent training framework and efficient computation for wind power forecasting.
Smart Images
Figure CN119627898A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a prediction method, in particular to a multi-station wind power prediction method based on feature embedding-transformer federated learning, and belongs to the technical field of multi-station wind power prediction. Background Art
[0002] With the continuous deepening of the national power system reform, wind power has attracted widespread attention for its advantages such as cleanliness, safety and sufficient resources. With the access of large-scale distributed wind farms, it plays an important role in optimizing resource allocation and increasing the proportion of clean energy. At the same time, the access of wind farms also increases the difficulty of the overall optimization and dispatching of the power system and increases the risk of the overall operation of the power system. Therefore, it is particularly important to accurately predict wind power generation and provide reliable information to the power system. For wind power generation prediction, on the one hand, wind power generation data is affected by relevant meteorological factors and is volatile and uncertain as a whole; on the other hand, with the access of multiple stations, the data collection methods and data distribution of each station are different, which is restricted by data privacy, making it difficult to complete data centralized training. For traditional single-station wind power generation prediction, considering the impact of historical data on future power generation data, the LSTM deep neural network algorithm is often used to predict future wind power generation data. On the one hand, this method is limited by the algorithm itself and cannot fully consider the impact of historical data. On the other hand, considering the privacy of multi-station data and the differences in data from each station, it is impossible to complete the centralized training task of data.
[0003] In response to the above-mentioned shortcomings, this application introduces a feature embedding transformer-federated learning algorithm to complete the distributed training and prediction of wind power generation data from multiple stations. First of all, considering the inconsistency of data features between different stations and the fact that traditional feature construction ignores the mutual influence between features, the feature data is dynamically embedded before building the prediction model. In addition, the introduction of the transformer model enables the model to fully calculate all historical data, avoiding the gradient vanishing and explosion phenomenon of the model when too much historical data is added to the LSTM; finally, with the introduction of federated learning, each station only transmits model parameters instead of data, which largely solves the data privacy problem. Summary of the invention
[0004] The present invention is aimed at the technical problems existing in the prior art, and provides a multi-station wind power prediction method based on feature embedding-transformer federated learning. The scheme introduces a dynamic feature embedding algorithm. Before the feature is input into the model, the feature embedding calculation is performed on the current feature. On the one hand, the problem of inconsistent feature data of different stations is solved. On the other hand, the introduction of the dynamic word embedding algorithm can make full use of the correlation between different features. Transformer can directly capture the dependency between any two positions, is not affected by the sequence length, and can better handle the prediction problem of time series data. In addition, transformer can fully realize parallel computing compared to LSTM, thereby greatly improving the computational efficiency of the model; finally, by introducing a federated learning algorithm, the data privacy problem of each wind power data station is solved by transmitting only model parameters and not internal data, and the privacy of the data is guaranteed.
[0005] In order to achieve the above object, the technical solution of the present invention is as follows: a multi-station wind power prediction method based on feature embedding-transformer federated learning, characterized in that the method comprises the following steps:
[0006] Step 1: Data preprocessing,
[0007] Step 2: Feature Embedding,
[0008] Step 3: Build the transformer prediction model,
[0009] Step 4: Federated learning joint training.
[0010] Among them, step 1 is as follows:
[0011] In order to prevent data errors from causing problems in the overall operation of the system, the data must be preprocessed before model training and prediction, including processing of missing values, singular values, etc. In addition, this application is different from other applications in that all data are discretized here, that is, all data are first identified, and the feature embedding task is completed by the identified data.
[0012] Specific implementation method For historical power generation data, since the installed capacity of different stations is different, the maximum installed capacity is taken as the data upper limit, and the historical power generation data is retained to two decimal places. The data is mapped according to the step size to obtain x 0～a , thus completing the discretization of historical power generation data. In addition, the meteorological data, date type data and the maximum historical power generation data are added to discretize the data to obtain x a+1～a+b With x a+b+1～a+b+cThus, historical power generation data, meteorological data, and date type data are integrated into one type of data, providing preparation for subsequent data encoding, as shown in formula (1).
[0013] x=[x0,x1,x2,…,x a ,x a+1 ,x a+2 ,x a+3 ,…,x a+b ,x a+b+1 ,x a+b+2 ,…,x a+b+c ] (1)
[0014] Where [0, a] represents the time step of historical power generation data, [a+1, b] represents the number of meteorological data, and [b+1, c] represents the number of date data.
[0015] Among them, step 2, feature embedding: complete the above data identification work, here complete the feature embedding work. Considering the traditional word2vec static feature embedding, for fixed features, the same feature embedding vector is often generated, while ignoring the relevance of the current feature itself. Here, the transformer dynamic feature embedding algorithm is introduced. First, relevant identifiers are added to each input feature. The specific method is to add [CLS] at the beginning of the feature, and use [SEP] to insert the middle of different types of features, including historical power generation data and meteorological data. Since the number of features contained in different sites is different, the [PAD] padding identifier is added here to ensure the consistency of feature length. Thus, each feature is decomposed into a label embedding matrix, a mask embedding matrix, and a classification embedding matrix. In addition, the feature position encoding matrix needs to be introduced here. The calculation method is directly given here as shown in (2-3), which is specifically expressed as follows:
[0016]
[0017]
[0018] Among them, i represents the position of the embedding matrix, pos is the position of the relevant feature in the feature vector, and d k represents the input data dimension,
[0019] After completing the above feature processing, the corresponding data can be input into the transformer dynamic feature embedding algorithm model, where the dynamic embedding model is mainly composed of multiple encoders. Figure 1 As shown. The output R is the corresponding feature embedding vector. It should be emphasized here that R i is the feature embedding vector of the corresponding position, R clsis the overall embedding vector of the feature, where the encoder is the same as the encoder used in the transformer prediction model below, which will be described in detail later.
[0020] Among them, step 3, building a transformer prediction model, is as follows:
[0021] As a popular neural network structure in recent years, transformer has achieved relatively good results in the field of time series data prediction. The transformer network result mainly includes an encoder and a decoder. The feature embedding model used in the above application is the transformer encoder part. The prediction model constructed here also uses the transformer encoder part to mine the dependency relationship of wind power data, thereby improving the accuracy of model prediction. For this application, the transformer prediction model used is composed of a combination of multiple encoders. Since the structures of each encoder are the same, only one of them is described here. As shown in the figure, the transformer encoder mainly includes an attention sublayer and a feedforward network layer, wherein the attention sublayer includes a multi-head attention mechanism and residual connections and layer normalization, and the feedforward neural network sublayer includes a feedforward neural network and residual connections and normalization.
[0022] The attention mechanism is a resource allocation mechanism that enables the model to selectively focus on the key features of the input data, which is mainly expressed as follows:
[0023]
[0024] Where Q, K, and V are query vector matrices, key vector matrices, and value vector matrices, respectively. k is the dimension of the input matrix, softmax is the normalization function used to normalize the calculation results, and the multi-head attention mechanism is to concatenate the outputs of multiple attentions and obtain the final multi-head attention output through linear changes, as shown in formula (5).
[0025] R = concat(Z1,Z2,…,Z n )W0 (5)
[0026] Among them, Z represents the output of self-attention, n is a hyperparameter representing the number of multi-head attention heads, which is 8 for this application, and W0 is the weight matrix. After the above, the output of the attention mechanism is spliced to obtain the final attention matrix, and the above encoder modeling part is completed. By embedding the feature output R, adding the position code to the transformer encoder model, as shown Figure 3As shown, the dependency relationship between wind power prediction data and feature data is established, and finally the output h of the transformer encoder is input into the feedforward neural network to obtain the final wind power prediction data y pre , so far all modeling processes for client wind power generation prediction are completed.
[0027] Among them, step 4, federated learning joint training, is as follows: Taking into account the data privacy issues in multi-site wind power generation scenarios that make it impossible to implement data centralized training, this application uses a federated learning framework to perform distributed isolated training on multi-site wind power generation data, and completes the training of the overall model by only uploading model parameters. In traditional federated learning, when training in combination with servers and clients, each client only performs gradient calculations based on the model parameters sent by the server to update the corresponding model parameters. In this way, the importance of model parameters is consistent, which leads to huge changes in some important parameters. To address the above problems, the Memory Aware Synapses (MAS) algorithm is introduced here. By calculating the importance of each parameter, it avoids huge changes in important parameters during model fitting. While ensuring the normal fitting of model parameters, it avoids the forgetting of important parameters in the model. The specific algorithm description is as follows:
[0028] First, the server sends the model parameters to the client, completes the model training on the corresponding client, and calculates its model parameters θ ij The importance of this task ij , where the importance of each parameter is equivalent to the sensitivity of the parameter to the model output, based on which we can derive its calculation formula:
[0029]
[0030] From formula (6), we can get It is the partial derivative of the model for each parameter on the data set, and then the degree of parameter importance Ω is obtained ij It can be expressed as follows:
[0031]
[0032] Where M represents the number of client samples. According to the importance of each parameter of the above model, when a new client trains the model, in order to avoid the arrival of new data causing a huge change in the importance parameters, a regularization term will be added to its original loss function to limit the update range of each parameter.
[0033]
[0034] in, The model sent to the server is also the initial parameter of the nth client task, and λ is the regularization hyperparameter. Figure 4 As shown, it can be seen that the model training here is mainly divided into two parts, the feature embedding network model and the transformer prediction model. Considering that the feature embedding model only performs feature embedding, the model parameters are not uploaded during the federated learning model training, and only the transformer prediction model is uploaded and aggregated.
[0035] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multi-site wind power prediction method based on feature embedding-transformer federated learning is implemented.
[0036] A computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the multi-site wind power prediction method based on feature embedding-transformer federated learning.
[0037] Compared with the prior art, the present invention has the following advantages. This technical solution aims to solve the problem that the data features of various wind power stations are inconsistent and the data privacy between stations makes it difficult to complete the centralized training of the prediction model. This application adopts a feature embedding-transformer federated learning strategy to perform distributed training on wind power data from multiple stations. While ensuring data privacy, it fully utilizes the data of each station to complete the training of the model. While making full use of the internal data of the station, it ensures the consistency of the joint training data structure. In addition, considering that traditional deep learning networks are often limited by the time span of data when solving time series data prediction, for this application, whether in the feature embedding stage or the prediction model training stage, the transformer encoder network structure can fully utilize the correlation between different features and establish a dependency relationship between any two positions, thereby fully fitting the wind power data of each station and improving the accuracy of the overall operation of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 is a schematic diagram of feature embedding.
[0039] Figure 2 This is a schematic diagram of the transformer encoder structure.
[0040] Figure 3 Prediction structure diagram for transformer wind power generation
[0041] Figure 4Schematic diagram of the federated learning framework. DETAILED DESCRIPTION
[0042] In order to deepen the understanding of the present invention, the present embodiment is described in detail below with reference to the accompanying drawings.
[0043] Example 1: See Figure 1-Figure 4 A multi-station wind power prediction method based on feature embedding-transformer federated learning, the method comprising the following steps:
[0044] Step 1: Data preprocessing,
[0045] Step 2: Feature Embedding,
[0046] Step 3: Build the transformer prediction model,
[0047] Step 4: Federated learning joint training.
[0048] Among them, step 1 is as follows:
[0049] In order to prevent data errors from causing problems in the overall operation of the system, the data must be preprocessed before model training and prediction, including processing of missing values, singular values, etc. In addition, this application is different from other applications in that all data are discretized here, that is, all data are first identified, and the feature embedding task is completed by the identified data.
[0050] Specific implementation method For historical power generation data, since the installed capacity of different stations is different, the maximum installed capacity is taken as the data upper limit, and the historical power generation data is retained to two decimal places. The data is mapped according to the step size to obtain x 0～a , thus completing the discretization of historical power generation data. In addition, the meteorological data, date type data and the maximum historical power generation data are added to discretize the data to obtain x a+1～a+b With x a+b+1～a+b+c Thus, historical power generation data, meteorological data, and date type data are integrated into one type of data, providing preparation for subsequent data encoding, as shown in formula (1).
[0051] x=[x0,x1,x2,…,x a ,x a+1 ,x a+2 ,x a+3 ,…,x a+b ,x a+b+1 ,x a+b+2 ,…,x a+b+c ] (1)
[0052] Where [0, a] represents the time step of historical power generation data, [a+1, b] represents the number of meteorological data, and [b+1, c] represents the number of date data.
[0053] Among them, step 2, feature embedding: complete the above data identification work, here complete the feature embedding work. Considering the traditional word2vec static feature embedding, for fixed features, the same feature embedding vector is often generated, while ignoring the relevance of the current feature itself. Here, the transformer dynamic feature embedding algorithm is introduced. First, relevant identifiers are added to each input feature. The specific method is to add [CLS] at the beginning of the feature, and use [SEP] to insert the middle of different types of features, including historical power generation data and meteorological data. Since the number of features contained in different sites is different, the [PAD] padding identifier is added here to ensure the consistency of feature length. Thus, each feature is decomposed into a label embedding matrix, a mask embedding matrix, and a classification embedding matrix. In addition, the feature position encoding matrix needs to be introduced here. The calculation method is directly given here as shown in (2-3), which is specifically expressed as follows:
[0054]
[0055] Among them, i represents the position of the embedding matrix, pos is the position of the relevant feature in the feature vector, and d k represents the input data dimension,
[0056] After completing the above feature processing, the corresponding data can be input into the transformer dynamic feature embedding algorithm model, where the dynamic embedding model is mainly composed of multiple encoders. Figure 1 As shown. The output R is the corresponding feature embedding vector. It should be emphasized here that R i is the feature embedding vector of the corresponding position, R cls is the overall embedding vector of the feature, where the encoder is the same as the encoder used in the transformer prediction model below, which will be described in detail later.
[0057] Among them, step 3, building a transformer prediction model, is as follows:
[0058] As a popular neural network structure in recent years, transformer has achieved relatively good results in the field of time series data prediction. The transformer network result mainly includes encoder and decoder. The feature embedding model used in the above application is the transformer encoder part. The prediction model constructed here also uses the transformer encoder part to mine the dependency relationship of wind power data, thereby improving the accuracy of model prediction. For this application, the transformer prediction model used is composed of multiple encoders. Since the structures of each encoder are the same, only one of them is described here, such as Figure 2 As shown in the figure, the transformer encoder mainly includes an attention sublayer and a feedforward network layer. The attention sublayer includes a multi-head attention mechanism, residual connections, and layer normalization. The feedforward neural network sublayer includes a feedforward neural network, residual connections, and normalization.
[0059] The attention mechanism is a resource allocation mechanism that enables the model to selectively focus on the key features of the input data, which is mainly expressed as follows:
[0060]
[0061] Where Q, K, and V are query vector matrices, key vector matrices, and value vector matrices, respectively. k is the dimension of the input matrix, softmax is the normalization function used to normalize the calculation results, and the multi-head attention mechanism is to concatenate the outputs of multiple attentions and obtain the final multi-head attention output through linear changes, as shown in formula (5).
[0062] R = concat(Z1,Z2,…,Z n )W0 (5)
[0063] Among them, Z represents the output of self-attention, n is a hyperparameter representing the number of multi-head attention heads, which is 8 for this application, and W0 is the weight matrix. After the above, the output of the attention mechanism is spliced to obtain the final attention matrix, and the above encoder modeling part is completed. By embedding the feature output R, adding the position code to the transformer encoder model, as shown Figure 3 As shown, the dependency relationship between wind power prediction data and feature data is established, and finally the output h of the transformer encoder is input into the feedforward neural network to obtain the final wind power prediction data y pre , so far all modeling processes for client wind power generation prediction are completed.
[0064] Among them, step 4, federated learning joint training, is as follows: Taking into account the data privacy issues in multi-site wind power generation scenarios that make it impossible to implement data centralized training, this application uses a federated learning framework to perform distributed isolated training on multi-site wind power generation data, and completes the training of the overall model by only uploading model parameters. In traditional federated learning, when training in combination with servers and clients, each client only performs gradient calculations based on the model parameters sent by the server to update the corresponding model parameters. In this way, the importance of model parameters is consistent, which leads to huge changes in some important parameters. To address the above problems, the Memory Aware Synapses (MAS) algorithm is introduced here. By calculating the importance of each parameter, it avoids huge changes in important parameters during model fitting. While ensuring the normal fitting of model parameters, it avoids the forgetting of important parameters in the model. The specific algorithm description is as follows:
[0065] First, the server sends the model parameters to the client, completes the model training on the corresponding client, and calculates its model parameters θ ij The importance of this task ij , where the importance of each parameter is equivalent to the sensitivity of the parameter to the model output, based on which we can derive its calculation formula:
[0066]
[0067] From formula (6), we can get It is the partial derivative of the model for each parameter on the data set, and then the degree of parameter importance Ω is obtained ij It can be expressed as follows:
[0068]
[0069] Where M represents the number of client samples. According to the importance of each parameter of the above model, when a new client trains the model, in order to avoid the arrival of new data causing a huge change in the importance parameters, a regularization term will be added to its original loss function to limit the update range of each parameter.
[0070]
[0071] in, The model is sent to the server and is also the initial parameter of the nth client task. λ is the regularization hyperparameter. So far, the overall framework of federated learning has been built.
[0072] The overall federated learning training framework is as follows Figure 4As shown, it can be seen that the model training here is mainly divided into two parts, the feature embedding network model and the transformer prediction model. Considering that the feature embedding model only performs feature embedding, the model parameters are not uploaded during the federated learning model training, and only the transformer prediction model is uploaded and aggregated.
[0073] Embodiment 2: An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multi-site wind power prediction method based on feature embedding-transformer federated learning is implemented.
[0074] Embodiment 3: A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implements the multi-station wind power prediction method based on feature embedding-transformer federated learning.
[0075] Embodiment 4:
[0076] This article about feature embedding-transformer federated learning multi-site wind power prediction is mainly divided into data preprocessing module, model building module and federated learning model integration module.
[0077] Data preprocessing module: Before establishing the feature embedding model and transformer prediction model, each station needs to complete the data preprocessing work. In order to prevent the system from failing to operate normally due to problems such as missing data and singular values, these data need to be filled or lost. In addition, all features of this application are processed in a discrete data manner. Therefore, it is necessary to first discretize the historical wind power generation data, meteorological data, and date type data to obtain the final feature identification as shown in formula (1).
[0078] Model building module: The model building module is mainly divided into feature embedding model and transformer prediction model module. For the feature embedding model, after completing the above data processing to obtain the feature identifier, it is also necessary to add relevant identifiers to the feature, so as to calculate the label embedding matrix, position embedding matrix, classification embedding matrix and mask matrix corresponding to the feature. For specific methods, see the above step 2. At this point, all feature preprocessing is completed, and the data is input into the feature embedding model containing multiple encoders to obtain the final feature embedding matrix R; for the transformer prediction model, as shown in Figure (3), it contains 8 encoders and an output feedforward network layer. For the output R of the feature embedding model, the position matrix is added to complete the feature construction of the prediction model, and the prediction model is input to finally obtain the wind power generation prediction result. Federated learning online update module: Through the construction of the above model, the federated learning framework completes the model training. It should be noted here that in this application scenario, each wind power station contains two network models, but only the transformer wind power generation prediction model needs to be uploaded to the server, and the feature embedding model only performs the data embedding function within the station. The specific approach is that the server sends the parameters of the prediction model to each station client. The client combines its own data to complete feature embedding and prediction model training, and then uploads the prediction model parameters to the server. After the server receives the model parameters of all stations, it aggregates all model parameters and then sends them to the client. The above steps are repeated until the model meets the prediction requirements.
[0079] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions all fall within the protection scope of the claims of the present invention.
Claims
1. A multi-station wind power prediction method based on feature embedding-transformer federated learning, characterized in that: The method comprises the following steps: Step 1: Data preprocessing, Step 2: Feature Embedding, Step 3: Build the transformer prediction model, Step 4: Federated learning joint training.
2. The multi-station wind power prediction method based on feature embedding-transformer federated learning according to claim 1 is characterized in that: Step 1 is as follows: Before training and predicting the model, the data must be preprocessed, including the processing of missing values and singular values, and all data must be discretized, that is, all data must be identified first, and the feature embedding task is completed by the identified data. Take the maximum installed capacity as the data upper limit, retain the historical power generation data to two decimal places, map the data according to the step length, and get x 0～a , thus completing the discretization of historical power generation data. In addition, the meteorological data, date type data and the maximum historical power generation data are added to discretize the data to obtain x a+1～a+b With x a+b+1～a+b+c Thus, historical power generation data, meteorological data, and date type data are integrated into one type of data, providing preparation for subsequent data encoding, as shown in formula (1). x=[x0,x1,x2,…,x a ,x a+1 ,x a+2 , x a+3 ,…,x a+b ,x a+b+1 ,x a+b+2 ,…,x a+b+c ](1) Where [0, a] represents the time step of historical power generation data, [a+1, b] represents the number of meteorological data, and [b+1, c] represents the number of date data.
3. The multi-station wind power prediction method based on feature embedding-transformer federated learning according to claim 2 is characterized in that: Step 2, feature embedding: Introduce the transformer dynamic feature embedding algorithm. First, add relevant identifiers to each input feature. The specific method is to add [CLS] at the beginning of the feature and use [SEP] to insert it in the middle of different types of features, including historical power generation data and meteorological data. Each feature is decomposed into a label embedding matrix, a mask embedding matrix, and a classification embedding matrix. It is also necessary to introduce a feature position encoding matrix. The calculation method is directly given as shown in (2) and (3). The specific expression is as follows: Among them, i represents the position of the embedding matrix, pos is the position of the relevant feature in the feature vector, and d k represents the input data dimension, After completing the above feature processing, the corresponding data can be input into the transformer dynamic feature embedding algorithm model, where the dynamic embedding model is mainly composed of multiple encoders.
4. The multi-station wind power prediction method based on feature embedding-transformer federated learning according to claim 3 is characterized in that: Step 3: Build the transformer prediction model as follows: The transformer prediction model is composed of multiple encoders. The transformer encoder mainly includes an attention sublayer and a feedforward network layer. The attention sublayer includes a multi-head attention mechanism, residual connections, and layer normalization. The feedforward neural network sublayer includes a feedforward neural network, residual connections, and normalization. The attention mechanism is a resource allocation mechanism that enables the model to selectively focus on the key features of the input data, which is mainly expressed as follows: Where Q, K, and V are query vector matrices, key vector matrices, and value vector matrices, respectively. k is the dimension of the input matrix, softmax is a normalization function used to normalize the calculation results, and the multi-head attention mechanism is to concatenate the outputs of multiple attentions and obtain the final multi-head attention output through linear changes, as shown in formula (5). R=concat(Z1,Z2,…,Z n )W0 (5) Among them, Z represents the output of self-attention, n is a hyperparameter representing the number of multi-head attention heads, and W0 is the weight matrix. After the above, the output of the attention mechanism is spliced to obtain the final attention matrix. After completing the encoder modeling part, the feature embedding output R is added to the position encoding input transformer encoder model to establish the dependency relationship between wind power prediction data and feature data. Finally, the output h of the transformer encoder is input into the feedforward neural network to obtain the final wind power prediction data y. pre , so far all modeling processes for client wind power generation prediction are completed.
5. The multi-station wind power prediction method based on feature embedding-transformer federated learning according to claim 2 is characterized in that: Step 4: Federated learning joint training. The details are as follows: Introduce the Memory Aware Synapses (MAS) algorithm to calculate the importance of each parameter to avoid drastic changes in important parameters during model fitting. While ensuring the normal fitting of model parameters, it avoids the forgetting of important parameters in the model. The specific algorithm is described as follows: First, the server sends the model parameters to the client, completes the model training on the corresponding client, and calculates its model parameters θ ij The importance of this task ij , where the importance of each parameter is equivalent to the sensitivity of the parameter to the model output, and the calculation formula is obtained: From formula (6), we can get It is the partial derivative of the model for each parameter on the data set, and then the degree of parameter importance Ω is obtained ij It can be expressed as follows: Where M represents the number of client samples. According to the importance of each parameter of the above model, when a new client trains the model, a regularization term is added to its original loss function to limit the update range of each parameter. in, The model is sent to the server and is also the initial parameter of the nth client task. λ is the regularization hyperparameter.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the multi-site wind power prediction method based on feature embedding-transformer federated learning as described in any one of claims 1 to 5 above is implemented.
7. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by the processor, the multi-site wind power prediction method based on feature embedding-transformer federated learning as described in any one of claims 1 to 5 is implemented.