Privacy protection model training method and system suitable for distributed new energy station

Abstract

The invention discloses a privacy protection model training method and system suitable for a distributed new energy station, and the method comprises the steps: judging whether the model performance of a local model of each new energy station in a current round is improved compared with that in a previous round in the federal training process of a second round and a subsequent round; if the new energy station is lifted, the new energy station is used as a participant of the current round of model aggregation operation, otherwise, the new energy station is empty in the current round, and the new energy station with improved performance is screened to participate in model aggregation, so that the communication cost waste caused by sending highly similar local model weight parameters is effectively avoided, and the communication efficiency is improved. The data volume needing to be processed by model aggregation is reduced, the model aggregation process is simplified, and the training speed is increased.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular, to a privacy protection model training method and system applicable to distributed new energy sites, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of new energy technologies, more and more distributed new energy sites and terminal devices are widely deployed. The data generated by these terminal devices contain a large amount of sensitive information, such as energy production, consumption, and related environmental parameters. However, with the increase of data and the increase in sharing needs, privacy protection has become an important challenge. Traditional centralized data training models may involve centralized storage and processing of data, which may lead to risks of privacy leakage and data security. In order to solve these problems, the recent research trend is to introduce federated learning technology into the new energy field. Federated learning is a distributed machine learning method that allows local model training on terminal devices without transmitting the original data to the central server for model training. After each round of training, the terminal device only needs to transmit its local model parameters to the central server for model aggregation to optimize and update the global model. This distributed learning method can protect the privacy of user data and reduce the need for data transmission. However, due to the large number of distributed new energy sites, all distributed new energy sites need to transmit their local model parameters to the central server, which has a large communication cost, and will also make the model aggregation process more complicated and the model training speed slower. Summary of the invention

[0003] The present invention provides a privacy protection model training method and system suitable for distributed new energy stations, an electronic device, and a computer-readable storage medium, which can effectively avoid the waste of communication costs caused by sending highly similar local model weight parameters, reduce the amount of data required for model aggregation, simplify the model aggregation process, and improve the training speed.

[0004] According to one aspect of the present invention, a privacy protection model training method applicable to distributed new energy stations is provided, comprising the following contents:

[0005] Initialize the global model of the server and distribute it to all new energy sites to be trained as the local model of each new energy site. Each new energy site conducts the first round of federated training based on the local model and the server conducts the first round of federated training based on the global model.

[0006] In the second and subsequent rounds of federated training, determine whether the model performance of the local model of each new energy station in the current round has improved compared with the previous round. If so, the new energy station will be included as a participant in the current round of model aggregation operations. Otherwise, the new energy station will not participate in this round.

[0007] Multiple rounds of federated training are performed until the global optimization loss function of the global model converges, and the training ends.

[0008] Furthermore, in the second and subsequent rounds of federated training, each round of federated training includes the following:

[0009] Calculate the aggregated contribution of each participant based on the distribution difference aggregation and weight parameter similarity aggregation, and send the aggregated contribution and local model weight parameters of each participant to the server;

[0010] The global model weight parameters are calculated based on the aggregated contribution of all participants and the local model weight parameters to optimize and update the global model of the server, and the global model weight parameters of the server and the local model weight parameters of all participants are distributed to all new energy stations;

[0011] Each new energy station updates and trains its own local model based on the global model weight parameters and the local model weight parameters of all participants.

[0012] Furthermore, the distribution difference aggregation of each participant is calculated based on the following formula:

[0013] d j =KL(D j ,D global )

[0014]

[0015] Among them, d j represents the difference between the local data distribution of the jth participant and the data distribution of the server, D j represents the local data distribution of the jth participant, D global represents the data distribution of the server, which is uniformly distributed. KL() represents the KL divergence calculation function. n j represents the relative size of the dataset of the jth participant, N j represents the dataset size of the jth participant, C represents the number of participants, and N i represents the dataset size of the i-th participant, q j represents the distribution difference aggregation of the jth participant, Max() represents the maximum value function, d irepresents the difference between the local data distribution of the ith participant and the data distribution of the server, and a represents a hyperparameter.

[0016] Furthermore, the weight parameter similarity aggregation of each participant is calculated based on the following formula:

[0017]

[0018] in, represents the weight parameter similarity aggregation of the jth participant in the tth round, β represents the hyperparameter, cosine() represents the cosine similarity calculation function, and denote the local model weight parameters of the jth participant in rounds t-1 and t, respectively. and denote the local model weight parameters of the kth participant in round t-1 and round t, respectively, and C t represents the number of participants in round t.

[0019] Furthermore, the process of updating and training the local model of each new energy station based on the global model weight parameters and the local model weight parameters of all participants includes the following:

[0020] Each new energy site calculates the similarity between its own local model weight parameters and the local model weight parameters of all participants, selects the local model weight parameters with the greatest similarity, and performs weighted summation on them with the global model weight parameters to update and train its own local model.

[0021] Furthermore, the global optimization loss function value of the global model is calculated based on the following formula:

[0022]

[0023] Among them, Loss represents the value of the global optimization loss function, M represents the number of new energy stations, and q i represents the distribution difference aggregation of the i-th new energy station, L i Represents the loss function value of the i-th new energy station.

[0024] Furthermore, when the number of byes of a certain new energy station is greater than a preset threshold, the new energy station will be used as a participant in the model aggregation operation in the next round of federated training.

[0025] In addition, the present invention also provides a privacy protection model training system applicable to distributed new energy stations, including:

[0026] The initialization module is used to initialize the global model of the server and distribute it to all new energy sites to be trained as the local model of each new energy site. Each new energy site conducts the first round of federated training based on the local model and the server conducts the first round of federated training based on the global model.

[0027] The screening module is used to determine whether the model performance of the local model of each new energy station in the current round has improved compared with the previous round during the second and subsequent rounds of federated training. If so, the new energy station will be included as a participant in the current round of model aggregation operations; otherwise, the new energy station will be left blank in this round.

[0028] The iterative training module is used to perform multiple rounds of federated training until the global optimization loss function of the global model converges, and the training ends.

[0029] Furthermore, during the second and subsequent rounds of federated training, the screening module calculates the aggregate contribution of each participant based on the distribution difference aggregation and weight parameter similarity aggregation, and sends the aggregate contribution and local model weight parameters of each participant to the server, and calculates the global model weight parameters based on the aggregate contribution and local model weight parameters of all participants to optimize and update the global model of the server, and distributes the global model weight parameters of the server and the local model weight parameters of all participants to all new energy stations, and each new energy station updates and trains its own local model based on the global model weight parameters and the local model weight parameters of all participants.

[0030] Furthermore, the screening module calculates the distribution difference aggregation of each participant based on the following formula:

[0031] d j =KL(D j ,D global )

[0032]

[0033] Among them, d j represents the difference between the local data distribution of the jth participant and the data distribution of the server, D j represents the local data distribution of the jth participant, D global represents the data distribution of the server, which is uniformly distributed. KL() represents the KL divergence calculation function. n j represents the relative size of the dataset of the jth participant, N j represents the dataset size of the jth participant, C represents the number of participants, and N i represents the dataset size of the i-th participant, q jrepresents the distribution difference aggregation of the jth participant, Max() represents the maximum value function, d i represents the difference between the local data distribution of the ith participant and the data distribution of the server, and a represents a hyperparameter.

[0034] Furthermore, the screening module calculates the weight parameter similarity aggregation of each participant based on the following formula:

[0035]

[0036] in, represents the weight parameter similarity aggregation of the jth participant in the tth round, β represents the hyperparameter, cosine() represents the cosine similarity calculation function, and denote the local model weight parameters of the jth participant in rounds t-1 and t, respectively. and denote the local model weight parameters of the kth participant in rounds t-1 and t, respectively, and c t represents the number of participants in round t.

[0037] Furthermore, each new energy site calculates the similarity between its own local model weight parameters and the local model weight parameters of all participants, selects the local model weight parameters with the greatest similarity, and performs weighted summation on them with the global model weight parameters to update and train its own local model.

[0038] Furthermore, the iterative training module calculates the global optimization loss function value of the global model based on the following formula:

[0039]

[0040] Among them, Loss represents the value of the global optimization loss function, M represents the number of new energy stations, and q i represents the distribution difference aggregation of the i-th new energy station, L i Represents the loss function value of the i-th new energy station.

[0041] Furthermore, when the number of byes of a certain new energy station is greater than a preset threshold, the new energy station will be used as a participant in the model aggregation operation in the next round of federated training.

[0042] In addition, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the above method by calling the computer program stored in the memory.

[0043] In addition, the present invention also provides a computer-readable storage medium for storing a computer program for training a privacy protection model suitable for distributed new energy stations, and the computer program executes the steps of the method described above when running on a computer.

[0044] The present invention has the following beneficial effects:

[0045] The privacy protection model training method applicable to distributed new energy stations of the present invention first initializes the global model of the server and distributes it to all new energy stations as the local model of each new energy station. Each new energy station uses its own data to train the local model, and transmits the trained local model weight parameters to the server to update the global model weight parameters, thereby completing the first round of federated training. Then, in the second and subsequent rounds of federated training, by judging whether the model performance of the local model of each new energy station in the current round is improved compared with the previous round, if there is an improvement, the new energy station is used as a participant in the current round of model aggregation operation, otherwise the new energy station is by-passed in this round, and the new energy station with improved performance is selected to participate in the model aggregation, which effectively avoids the waste of communication costs caused by sending highly similar local model weight parameters, reduces the amount of data required for model aggregation, simplifies the model aggregation process, and improves the training speed. Finally, through multiple rounds of federated training, until the global optimization loss function of the global model converges, the training ends.

[0046] In addition, the privacy protection model training system applicable to distributed new energy stations of the present invention also has the above advantages.

[0047] In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0049] Figure 1 It is a flow chart of a privacy protection model training method applicable to distributed new energy stations according to a preferred embodiment of the present invention.

[0050] Figure 2 It is a flowchart of the second and subsequent rounds of federated training process in the preferred embodiment of the present invention.

[0051] Figure 3It is a schematic diagram of the module structure of a privacy protection model training system applicable to distributed new energy stations according to another embodiment of the present invention. DETAILED DESCRIPTION

[0052] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0053] Reference Figure 1 The preferred embodiment of the present invention provides a privacy protection model training method applicable to distributed new energy stations, including the following contents:

[0054] Step S1: Initialize the global model of the server and distribute it to all new energy sites to be trained as the local model of each new energy site. Each new energy site performs the first round of federated training based on the local model and the server performs the first round of federated training based on the global model.

[0055] Step S2: During the second and subsequent rounds of federated training, determine whether the model performance of the local model of each new energy station in the current round has improved compared with the previous round. If so, the new energy station will be used as a participant in the current round of model aggregation operation; otherwise, the new energy station will not participate in the current round;

[0056] Step S3: Perform multiple rounds of federated training until the global optimization loss function of the global model converges, and the training ends.

[0057] It can be understood that the privacy protection model training method applicable to distributed new energy stations in this embodiment first initializes the global model of the server and distributes it to all new energy stations as the local model of each new energy station. Each new energy station uses its own data to train the local model, and transmits the trained local model weight parameters to the server to update the global model weight parameters, thereby completing the first round of federated training. Then, in the second and subsequent rounds of federated training, by judging whether the model performance of the local model of each new energy station in the current round is improved compared with the previous round, if there is an improvement, the new energy station is used as a participant in the current round of model aggregation operation, otherwise the new energy station is not in the current round, and by screening new energy stations with improved performance to participate in model aggregation, the communication cost waste caused by sending highly similar local model weight parameters is effectively avoided, the amount of data required for model aggregation is reduced, the model aggregation process is simplified, and the training speed is improved. Finally, through multiple rounds of federated training, until the global optimization loss function of the global model converges, the training ends.

[0058] It can be understood that in step S1, the global model on the server is initialized, and the initialized global model is encrypted and distributed to all new energy stations. After each new energy station decrypts, the initialized global model is used as its own local model, and the local model is trained using its own local data. The trained local model weight parameters are then transmitted to the server to update the global model weight parameters, thereby completing the first round of federal training. Therefore, in the first round of federal training, all new energy stations participate in the model aggregation operation. Among them, the global model weight parameters are calculated based on the following formula:

[0059]

[0060] Among them, E() represents the CKKS homomorphic encryption calculation function, W t+1 represents the global model weight parameter of the t+1th round, represents the aggregate contribution of the jth participant in round t, represents the local model weight parameter of the jth participant in round t, c t represents the number of participants in round t.

[0061] Optionally, in the first round of federated training, each new energy station uses distribution difference aggregation as its aggregate contribution, where the distribution difference aggregation of each participant is calculated based on the following formula:

[0062] d j =KL(D j ,D global )

[0063]

[0064] Among them, d j represents the difference between the local data distribution of the jth participant and the data distribution of the server, D j represents the local data distribution of the jth participant, D global represents the data distribution of the server, which is assumed to be uniformly distributed based on prior knowledge, KL() represents the KL divergence calculation function, n j represents the relative size of the dataset of the jth participant, N j represents the dataset size of the jth participant, C represents the number of participants, and N i represents the dataset size of the i-th participant, q j represents the distribution difference aggregation of the jth participant, Max() represents the maximum value function, d i represents the difference between the local data distribution of the ith participant and the data distribution of the server, and a represents a hyperparameter.

[0065] It can be understood that when performing model aggregation, the prior art usually sets the aggregation contribution of each local model parameter to a fixed value, resulting in poor generalization ability of the global model. The present invention calculates the difference between the data distribution of each participant and the server through KL divergence, and assigns different distribution difference aggregation values ​​to each participant in combination with the relative size of each participant's data set, so that the distribution difference aggregation assigned to participants with larger data sets and smaller distribution differences is higher, thereby realizing dynamic adjustment of the aggregation contribution and improving the generalization ability and accuracy of the global model. Of course, in other embodiments of the present invention, during the first round of federated training, a fixed aggregation contribution can also be set for all new energy sites.

[0066] It can be understood that in the entire federated learning process, the distribution difference aggregation of each participant only needs to be calculated once.

[0067] It can be understood that in step S2, in the second and subsequent rounds of federated training, it is necessary to screen new energy stations, and all new energy stations are no longer used as participants in the model aggregation operation. Specifically, it is determined whether the model performance of the local model of each new energy station in the current round has been improved compared with the previous round. If so, the new energy station is used as a participant in the current round of model aggregation operations, otherwise the new energy station is not selected in this round, that is, it is eliminated in this round. Therefore, the present invention effectively avoids the waste of communication costs caused by sending highly similar local model weight parameters by screening new energy stations with improved performance to participate in model aggregation, reduces the amount of data required to be processed for model aggregation, simplifies the model aggregation process, and improves the training speed.

[0068] Optionally, during the entire federated learning process, when the number of byes of a new energy station is greater than a preset threshold, the new energy station will be used as a participant in the model aggregation operation in the next round of federated training. By setting a threshold for the number of byes, when the number of byes of a new energy station exceeds the threshold, it will be used as a participant in the model aggregation operation again, ensuring that all new energy stations have the opportunity to participate in training, thereby enhancing the fairness and participation of the training system.

[0069] In addition, if Figure 2 As shown in the figure, in the second and subsequent rounds of federated training, each round of federated training includes the following:

[0070] Step S21: Calculate the aggregated contribution of each participant based on the distribution difference aggregation and weight parameter similarity aggregation, and send the aggregated contribution and local model weight parameter of each participant to the server;

[0071] Step S22: Calculate the global model weight parameters based on the aggregated contribution of all participants and the local model weight parameters to optimize and update the global model of the server, and distribute the global model weight parameters of the server and the local model weight parameters of all participants to all new energy stations;

[0072] Step S23: Each new energy station updates and trains its own local model based on the global model weight parameters and the local model weight parameters of all participants.

[0073] Specifically, in the second and subsequent rounds of federated training, the aggregated contribution of each participant is calculated based on the weighted sum of the distribution difference aggregation and weight parameter similarity aggregation. The distribution difference aggregation has been calculated in the first federated learning process and will not be repeated here. The weight parameter similarity aggregation of each participant is calculated based on the following formula:

[0074]

[0075] in, represents the weight parameter similarity aggregation of the jth participant in the tth round, β represents the hyperparameter, cosine() represents the cosine similarity calculation function, and denote the local model weight parameters of the jth participant in rounds t-1 and t, respectively. and denote the local model weight parameters of the kth participant in rounds t-1 and t, respectively, and c t Represents the number of participants in the tth round. The present invention takes into account the differences in the weight parameters of the local models of the participants in two adjacent rounds, and calculates the similarity of the weight parameters of the local models in two adjacent rounds by cosine similarity, thereby assigning different weight parameter similarity aggregation values ​​to each participant, so that the weight parameter similarity aggregation assigned to the participants with lower similarity of the local model weight parameters in two adjacent rounds is higher, further improving the generalization ability and accuracy of the global model, and maintaining the balance and fairness among the participants.

[0076] In addition, the aggregate contribution is calculated based on the following formula: in, represents the aggregate contribution of the jth participant in round t, q j represents the distribution difference aggregation of the jth participant, represents the weight parameter similarity aggregation of the jth participant in the tth round, γ, represents the weight coefficient,

[0077] It can be understood that in the second and subsequent rounds of federated training, the aggregate contribution of each participant is calculated based on the weighted sum of the distribution difference aggregation and weight parameter similarity aggregation, which not only takes into account the novelty of each participant's local model update, but also takes into account the diversity of data distribution, thereby greatly improving the generalization ability and accuracy of the global model.

[0078] Then, each participant sends its aggregated contribution and local model weight parameters to the server after homomorphic encryption by CKKS. The server decrypts them based on the formula: The global model weight parameters are calculated to optimize and update the global model, and the global model weight parameters and the local model weight parameters of all participants are distributed to all new energy stations after being homomorphically encrypted by CKKS. The present invention introduces the CKKS homomorphic encryption method to encrypt the model updates of new energy stations, ensuring secure model aggregation without leaking the data privacy of new energy stations. This measure significantly improves the security of the system and combats potential data leakage risks.

[0079] Finally, after decryption, each new energy station obtains the global model weight parameters and the local model weight parameters of all participants. Each new energy station calculates the similarity between its own local model weight parameters and the local model weight parameters of all participants, selects the local model weight parameters with the greatest similarity, and performs weighted summation with the global model weight parameters to update and train its own local model. It can be understood that the global model weight parameters need to adapt to the data of all participants. By selecting the local model weight parameters with the greatest similarity and performing weighted summation with the global model weight parameters, each new energy station can not only learn the data characteristics of other similar stations, but also avoid overfitting the global model weight parameters.

[0080] It can be understood that in step S3, multiple rounds of federated training are performed until the global optimization loss function of the global model converges, then the federated learning ends, and the final global model is a privacy protection model suitable for distributed new energy stations. The global optimization loss function value of the global model is calculated based on the following formula:

[0081]

[0082] Among them, Loss represents the value of the global optimization loss function, M represents the number of new energy stations, and q i represents the distribution difference aggregation of the i-th new energy station, L i Represents the loss function value of the i-th new energy station. In addition, the specific loss function type can be a distillation loss function, a cross entropy loss function, a Hinge loss function, etc., which are not specifically limited here.

[0083] It can be understood that the present invention adopts the distribution difference aggregation of each new energy station as the weight coefficient, comprehensively calculates the global loss based on the loss function value of all new energy stations, combines the diversity of data distribution to consider global convergence, and realizes accurate convergence of the global model.

[0084] In addition, if Figure 3 As shown, another embodiment of the present invention further provides a privacy protection model training system applicable to distributed new energy stations, preferably using the privacy protection model training method as described above, including:

[0085] The initialization module is used to initialize the global model of the server and distribute it to all new energy sites to be trained as the local model of each new energy site. Each new energy site conducts the first round of federated training based on the local model and the server conducts the first round of federated training based on the global model.

[0086] The screening module is used to determine whether the model performance of the local model of each new energy station in the current round has improved compared with the previous round during the second and subsequent rounds of federated training. If so, the new energy station will be included as a participant in the current round of model aggregation operations; otherwise, the new energy station will be left blank in this round.

[0087] The iterative training module is used to perform multiple rounds of federated training until the global optimization loss function of the global model converges, and the training ends.

[0088] It can be understood that the privacy protection model training system for distributed new energy stations in this embodiment first initializes the global model of the server and distributes it to all new energy stations as the local model of each new energy station. Each new energy station uses its own data to train the local model, and transmits the trained local model weight parameters to the server to update the global model weight parameters, thereby completing the first round of federated training. Then, in the second and subsequent rounds of federated training, by judging whether the model performance of the local model of each new energy station in the current round is improved compared with the previous round, if there is an improvement, the new energy station is used as a participant in the current round of model aggregation operation, otherwise the new energy station is not in the current round, and by screening new energy stations with improved performance to participate in model aggregation, the communication cost waste caused by sending highly similar local model weight parameters is effectively avoided, the amount of data required for model aggregation is reduced, the model aggregation process is simplified, and the training speed is improved. Finally, through multiple rounds of federated training, until the global optimization loss function of the global model converges, the training ends.

[0089] Among them, during the second and subsequent rounds of federated training, the screening module calculates the aggregate contribution of each participant based on the distribution difference aggregation and weight parameter similarity aggregation, and sends the aggregate contribution and local model weight parameters of each participant to the server, and calculates the global model weight parameters based on the aggregate contribution and local model weight parameters of all participants to optimize and update the global model of the server, and distributes the global model weight parameters of the server and the local model weight parameters of all participants to all new energy stations, and each new energy station updates and trains its own local model based on the global model weight parameters and the local model weight parameters of all participants.

[0090] In addition, the screening module calculates the distribution difference aggregation for each participant based on the following formula:

[0091] d j =KL(D j ,D global )

[0092]

[0093] Among them, d j represents the difference between the local data distribution of the jth participant and the data distribution of the server, D j represents the local data distribution of the jth participant, D global represents the data distribution of the server, which is uniformly distributed. KL() represents the KL divergence calculation function. n j represents the relative size of the dataset of the jth participant, N j represents the dataset size of the jth participant, C represents the number of participants, and N i represents the dataset size of the i-th participant, q j represents the distribution difference aggregation of the jth participant, Max() represents the maximum value function, d i represents the difference between the local data distribution of the ith participant and the data distribution of the server, and a represents a hyperparameter.

[0094] In addition, the screening module calculates the weight parameter similarity aggregation of each participant based on the following formula:

[0095]

[0096] in, represents the weight parameter similarity aggregation of the jth participant in the tth round, β represents the hyperparameter, cosine() represents the cosine similarity calculation function, and denote the local model weight parameters of the jth participant in rounds t-1 and t, respectively. and denote the local model weight parameters of the kth participant in rounds t-1 and t, respectively, and c t represents the number of participants in round t.

[0097] In addition, each new energy station calculates the similarity between its own local model weight parameters and the local model weight parameters of all participants, selects the local model weight parameters with the greatest similarity, and performs weighted summation on them with the global model weight parameters to update and train its own local model.

[0098] In addition, the iterative training module calculates the global optimization loss function value of the global model based on the following formula:

[0099]

[0100] Among them, Loss represents the value of the global optimization loss function, M represents the number of new energy stations, and q i represents the distribution difference aggregation of the i-th new energy station, L i Represents the loss function value of the i-th new energy station.

[0101] In addition, when the number of byes of a new energy station is greater than the preset threshold, the new energy station will be used as a participant in the model aggregation operation in the next round of federated training.

[0102] In addition, another embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the above method by calling the computer program stored in the memory.

[0103] In addition, another embodiment of the present invention also provides a computer-readable storage medium for storing a computer program for training a privacy protection model suitable for distributed new energy stations, and the computer program executes the steps of the method described above when running on a computer.

[0104] The general form of computer readable storage media includes: floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with a pattern of holes, random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), flash erasable programmable read-only memory (FLASH-EPROM), any other memory chip or cartridge, or any other medium that can be read by a computer. The instructions can further be transmitted or received by a transmission medium. The term transmission medium can include any tangible or intangible medium that can be used to store, encode or carry instructions for execution by a machine, and includes digital or analog communication signals or other intangible media that facilitate the communication of the above instructions. Transmission media include coaxial cables, copper wires and optical fibers, which include the wires of a bus used to transmit a computer data signal.

[0105] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0106] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0109] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0110] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A privacy protection model training method applicable to distributed new energy stations, characterized in that: Includes the following: Initialize the global model of the server and distribute it to all new energy sites to be trained as the local model of each new energy site. Each new energy site conducts the first round of federated training based on the local model and the server conducts the first round of federated training based on the global model. In the second and subsequent rounds of federated training, determine whether the model performance of the local model of each new energy station in the current round has improved compared with the previous round. If so, the new energy station will be included as a participant in the current round of model aggregation operations. Otherwise, the new energy station will not participate in this round. Multiple rounds of federated training are performed until the global optimization loss function of the global model converges, and the training ends.

2. The privacy protection model training method applicable to distributed new energy stations according to claim 1, characterized in that: In the second and subsequent rounds of federated training, each round of federated training includes the following: Calculate the aggregated contribution of each participant based on the distribution difference aggregation and weight parameter similarity aggregation, and send the aggregated contribution and local model weight parameters of each participant to the server; The global model weight parameters are calculated based on the aggregated contribution of all participants and the local model weight parameters to optimize and update the global model of the server, and the global model weight parameters of the server and the local model weight parameters of all participants are distributed to all new energy stations; Each new energy station updates and trains its own local model based on the global model weight parameters and the local model weight parameters of all participants.

3. The privacy protection model training method applicable to distributed new energy stations according to claim 2, characterized in that: The distribution difference aggregate for each participant is calculated based on the following formula: d j =KL(D j ,D global ) Among them, d j represents the difference between the local data distribution of the jth participant and the data distribution of the server, D j represents the local data distribution of the jth participant, D global represents the data distribution of the server, which is uniformly distributed. KL() represents the KL divergence calculation function. n j represents the relative size of the dataset of the jth participant, N j represents the dataset size of the jth participant, C represents the number of participants, and N i represents the dataset size of the i-th participant, q j represents the distribution difference aggregation of the jth participant, Max() represents the maximum value function, d i represents the difference between the local data distribution of the ith participant and the data distribution of the server, and a represents a hyperparameter.

4. The privacy protection model training method applicable to distributed new energy stations according to claim 2, characterized in that: The weight parameter similarity aggregation of each participant is calculated based on the following formula: in, represents the weight parameter similarity aggregation of the jth participant in the tth round, β represents the hyperparameter, cosine() represents the cosine similarity calculation function, and denote the local model weight parameters of the jth participant in rounds t-1 and t, respectively. and denote the local model weight parameters of the kth participant in round t-1 and round t, respectively, and C t represents the number of participants in round t.

5. The privacy protection model training method applicable to distributed new energy stations according to claim 2, characterized in that: The process of updating and training the local model of each new energy station based on the global model weight parameters and the local model weight parameters of all participants includes the following: Each new energy site calculates the similarity between its own local model weight parameters and the local model weight parameters of all participants, selects the local model weight parameters with the greatest similarity, and performs weighted summation on them with the global model weight parameters to update and train its own local model.

6. The privacy protection model training method applicable to distributed new energy stations according to claim 1, characterized in that: The global optimization loss function value of the global model is calculated based on the following formula: Among them, Loss represents the value of the global optimization loss function, M represents the number of new energy stations, and q i represents the distribution difference aggregation of the i-th new energy station, L i Represents the loss function value of the i-th new energy station.

7. The privacy protection model training method applicable to distributed new energy stations according to claim 1, characterized in that: When the number of byes of a new energy station is greater than the preset threshold, the new energy station will be used as a participant in the model aggregation operation in the next round of federated training.

8. A privacy protection model training system suitable for distributed new energy stations, characterized in that: include: The initialization module is used to initialize the global model of the server and distribute it to all new energy sites to be trained as the local model of each new energy site. Each new energy site conducts the first round of federated training based on the local model and the server conducts the first round of federated training based on the global model. The screening module is used to determine whether the model performance of the local model of each new energy station in the current round has improved compared with the previous round during the second and subsequent rounds of federated training. If so, the new energy station will be included as a participant in the current round of model aggregation operations; otherwise, the new energy station will be left blank in this round. The iterative training module is used to perform multiple rounds of federated training until the global optimization loss function of the global model converges, and the training ends.

9. The privacy protection model training system applicable to distributed new energy stations according to claim 8, characterized in that: During the second and subsequent rounds of federated training, the screening module calculates the aggregate contribution of each participant based on the distribution difference aggregation and weight parameter similarity aggregation, and sends the aggregate contribution and local model weight parameters of each participant to the server, and calculates the global model weight parameters based on the aggregate contribution and local model weight parameters of all participants to optimize and update the global model of the server, and distributes the global model weight parameters of the server and the local model weight parameters of all participants to all new energy stations, and each new energy station updates and trains its own local model based on the global model weight parameters and the local model weight parameters of all participants.

10. The privacy protection model training system applicable to distributed new energy stations according to claim 9, characterized in that: The screening module calculates the distribution difference aggregation for each participant based on the following formula: d j =KL(D j ,D global ) Among them, d j represents the difference between the local data distribution of the jth participant and the data distribution of the server, D j represents the local data distribution of the jth participant, D global represents the data distribution of the server, which is uniformly distributed. KL() represents the KL divergence calculation function. n j represents the relative size of the dataset of the jth participant, N j represents the dataset size of the jth participant, C represents the number of participants, and N i represents the dataset size of the i-th participant, q j represents the distribution difference aggregation of the jth participant, Max() represents the maximum value function, d i represents the difference between the local data distribution of the ith participant and the data distribution of the server, and a represents a hyperparameter.

11. The privacy protection model training system applicable to distributed new energy stations according to claim 9, characterized in that: The screening module calculates the weight parameter similarity aggregation of each participant based on the following formula: in, represents the weight parameter similarity aggregation of the jth participant in the tth round, β represents the hyperparameter, cosine() represents the cosine similarity calculation function, and denote the local model weight parameters of the jth participant in rounds t-1 and t, respectively. and denote the local model weight parameters of the kth participant in round t-1 and round t, respectively, and C t represents the number of participants in round t.

12. The privacy protection model training system applicable to distributed new energy stations according to claim 9, characterized in that: Each new energy site calculates the similarity between its own local model weight parameters and the local model weight parameters of all participants, selects the local model weight parameters with the greatest similarity, and performs weighted summation on them with the global model weight parameters to update and train its own local model.

13. The privacy protection model training system applicable to distributed new energy stations according to claim 8, characterized in that: The iterative training module calculates the global optimization loss function value of the global model based on the following formula: Among them, Loss represents the value of the global optimization loss function, M represents the number of new energy stations, and q i represents the distribution difference aggregation of the i-th new energy station, L i Represents the loss function value of the i-th new energy station.

14. The privacy protection model training system applicable to distributed new energy stations according to claim 8, characterized in that: When the number of byes of a new energy station is greater than the preset threshold, the new energy station will be used as a participant in the model aggregation operation in the next round of federated training.

15. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of the method according to any one of claims 1 to 7 by calling the computer program stored in the memory.

16. A computer-readable storage medium for storing a computer program for training a privacy protection model for distributed new energy stations, characterized in that: When the computer program is run on a computer, the steps of the method according to any one of claims 1 to 7 are executed.

Images