Source, network, load, storage and use multi-source heterogeneous data fusion method based on cross-feature federated learning

Abstract

The invention relates to the field of electric power, in particular to a source network load storage multi-source heterogeneous data fusion method based on cross-feature federated learning, comprising the following steps: S1, collecting participant data, and preprocessing the participant data, the preprocessing comprising heterogeneous data feature extraction, secret generation, shared encryption, and model training and evaluation; performing data fusion processing on the preprocessed participant data to obtain fused participant data; and S2, carrying out decryption recovery based on the processed participant data to obtain updated global model parameters, and sending the updated global model parameters to the participant to generate new participant data for next iteration. The multi-source heterogeneous data fusion method has the advantages that the multi-source heterogeneous data can be safely fused on the premise of not leaking the privacy data of each participant by performing feature extraction on the heterogeneous data and based on the privacy computing technologies such as homomorphic encryption and secret sharing, and the data security is effectively improved.

Description

[Technical field]

[0001] The present invention relates to the field of electric power, and in particular to a method for fusion of multi-source heterogeneous data of source, grid, load and storage based on cross-feature federated learning. [Background technology]

[0002] The new power systems are located in different regions and provinces. There are large data barriers between them in terms of data transmission, which increases the difficulty of collecting and processing power-related data and leads to a significant data island effect.

[0003] In view of the fact that the parties involved cannot directly exchange local private data, the online transmission method inevitably has the risk of privacy leakage, which brings severe challenges to the collection and processing of user electricity consumption data in the power system and the flow of plain text data between participants. However, maintaining data security is very important. The user electricity privacy information in the power data and the confidential information of power generation, transmission, transformation and distribution are national security resources, and it is urgent to take sufficient and necessary data security measures.

[0004] Especially for the single point failure problem of aggregation servers caused by the simplification of federated learning, and the susceptibility to collusion attacks between malicious aggregation servers and participants, there is an urgent need for a multi-source heterogeneous data fusion solution for data and model privacy protection.

[0005] Therefore, a multi-source heterogeneous data fusion method for source, grid, load and storage based on cross-feature federated learning is adopted. By extracting features from heterogeneous data and based on privacy computing technologies such as homomorphic encryption and secret sharing, the secure fusion of multi-source heterogeneous data is achieved without leaking the privacy data of each participant, thereby effectively improving data security. [Summary of the invention]

[0006] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a multi-source heterogeneous data fusion method for source, grid, load and storage based on cross-feature federated learning. It can extract features from heterogeneous data and, based on privacy computing technologies such as homomorphic encryption and secret sharing, achieve secure fusion of multi-source heterogeneous data without leaking the privacy data of each participant, thereby effectively improving data security.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] A method for fusion of multi-source heterogeneous data for source, grid, load and storage based on cross-feature federated learning, comprising the following steps:

[0009] S1. Collect data from participants and preprocess the data, including heterogeneous data feature extraction, secret generation, shared encryption, model training and evaluation, and perform data fusion processing on the preprocessed data to obtain fused data of participants;

[0010] S2. Perform decryption and recovery based on the processed participant data to obtain updated global model parameters, and send the updated global model parameters to the participants to generate new participant data for the next iteration.

[0011] Preferably, S1 specifically includes:

[0012] S11: Randomly initialize parameters and generate key pairs (sk, pk), and send global initialization parameters and public key pk to each participant;

[0013] S12: Perform model training, extract features from the data of the participants and perform model training, use the Shamir secret sharing scheme and homomorphic encryption technology to generate shared fragments, and upload them to the aggregation server for model aggregation;

[0014] S13: Perform model aggregation, the aggregation server locally sums up each encrypted shared fragment, and then sends the total sum information to the central server.

[0015] Preferably, S2 specifically includes:

[0016] S21: Perform model update. The central server decrypts and recovers the total sum information to obtain updated global model parameters, and resends the updated global model parameters to the participants for the next round of iteration.

[0017] Preferably, a high-order tensor W is introduced as a memory unit, each mode of which corresponds to a spatial mapping of a data feature of a participant.

[0018] Preferably, the participant data includes a training set, a validation set and a test set, and a loss function is calculated. By calculating the gradient of the loss function, each participant data generates gradient information about the model parameters.

[0019] As a preference, the formula is calculated based on the number of training rounds: D train ,D valid ,D test = split_dataset(D i )

[0020]

[0021]

[0022] <g>=ShamirSecretSharing(g,t,m)

[0023] Among them, D i represents the data set, D train represents the training set, D valid represents the validation set, D test represents the test set, (x l ,y l ) is D train Specific data set, M is the model, and b is the batch size.

[0024] Preferably, the Lagrange interpolation method is used to recover the complete average gradient, based on the learning rate α and the calculated gradient G*,

[0025] [<G i >]=[<g i1 >]+[<g i2 >]+...+[<g im >]

[0026] G=Lagrange(<G1>,<G2>,...,<G j >)

[0027]

[0028] M=M-α×G *

[0029] Among them, [<g im >] is the shared fragment gradient parameter, α is the learning rate, and M is the model.

[0030] Preferably, the fused participant data is subjected to adaptive denoising processing to obtain noise-filtered participant data.

[0031] A multi-source heterogeneous data fusion system for source, grid, load and storage based on cross-feature federated learning. The system uses a multi-source heterogeneous data fusion method for source, grid, load and storage based on cross-feature federated learning. The system includes a participant module, an aggregation server and a central server.

[0032] The participant module includes heterogeneous data feature extraction, secret generation, shared encryption, model training and evaluation function modules, which are used to randomly initialize parameters and generate key pairs (sk, pk), send global initialization parameters and public key pk to each participant, and perform model training, extract features from participant data and perform model training, use Shamir secret sharing scheme and homomorphic encryption technology to generate shared fragments, and upload them to the aggregation server for model aggregation;

[0033] The aggregation server includes a shared summing module for performing model aggregation. The aggregation server locally sums each encrypted shared fragment and then sends the total sum information to the central server.

[0034] The central server includes a decryption recovery module and a key generation module to perform model update. The central server decrypts and recovers the total sum information to obtain updated global model parameters, and resends the updated global model parameters to the participants for the next round of iteration.

[0035] A processor is used to run a program, wherein the program executes the above-mentioned multi-source heterogeneous data fusion method for source, grid, load and storage based on cross-feature federated learning when running.

[0036] The beneficial effects of the present invention are as follows:

[0037] 1. In view of the situation where the participants cannot directly exchange local private data, by extracting features from heterogeneous data and using homomorphic encryption technology, multi-source heterogeneous data can be securely integrated without leaking the privacy data of each participant, thereby improving the security of data transmission.

[0038] 2. In each training round, the data of the participants include training set, validation set and test set, and the loss function is calculated. By calculating the gradient of the loss function, the data of each participant generates the gradient information about the model parameters. To ensure data privacy, the Shamir secret sharing scheme is used to split the gradient information into multiple copies.

[0039] Secondly, each gradient shard is encrypted using homomorphic encryption technology to generate encrypted gradient shards and send them to the aggregation server. After receiving the encrypted gradient shards from each participant, the aggregation server can perform the update operation of the model parameters without exposing the original gradient information, thereby ensuring data privacy and model security.

[0040] 3. By extracting features from heterogeneous data and based on homomorphic encryption and Shamir secret sharing technology, secure fusion of multi-source heterogeneous data is achieved without leaking the privacy data of each participant. This solution can not only achieve the effect of model training on a single machine, but also effectively protect the privacy of the participants, and has better performance than the single-machine plaintext case. This provides a feasible solution for data factorization in new power systems and has the potential to promote the integration and comprehensive deployment of power flow, information flow and value flow.

[0041] These features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings.

Brief Description of the Drawings

[0042] The present invention will be further described below in conjunction with the accompanying drawings:

[0043] Figure 1 It is a schematic diagram of the process of the present invention;

[0044] Figure 2 Schematic diagram of the learning process;

[0045] Figure 3 Design schematics for the modules;

[0046] Figure 4 It is a schematic diagram of the iterative process;

[0047] Figure 5 This is a schematic diagram of model training;

[0048] Figure 6 Schematic diagram of model aggregation and model update process. [Specific implementation method]

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The following first describes the concepts involved in this application in conjunction with the accompanying drawings. It should be noted that the following description of each concept is only to make the content of this application easier to understand and does not limit the scope of protection of this application.

[0051] Embodiment 1:

[0052] A multi-source heterogeneous data fusion method for source, grid, load and storage based on cross-feature federated learning, such as Figure 1 As shown, the steps include the following:

[0053] S1. Collect data from participants and preprocess the data, including heterogeneous data feature extraction, secret generation, shared encryption, model training and evaluation, and perform data fusion processing on the preprocessed data to obtain fused data of participants;

[0054] S1 specifically includes:

[0055] S11: Randomly initialize parameters and generate key pairs (sk, pk), and send global initialization parameters and public key pk to each participant;

[0056] S12: Perform model training, extract features from the data of the participants and perform model training, use the Shamir secret sharing scheme and homomorphic encryption technology to generate shared fragments, and upload them to the aggregation server for model aggregation;

[0057] like Figures 2 to 4 As shown in Figure 1, in the model training phase, participant i uses the feature extraction module to extract the features of heterogeneous data to obtain feature data in each training, and uses the extracted feature data to train the model. For the trained model, the public key obtained from the central server is used to encrypt these secret fragments to obtain encrypted shared fragments.

[0058] In view of the situation where the participants cannot directly exchange local private data, multi-source heterogeneous data can be securely integrated without leaking the privacy data of the participants by extracting features from heterogeneous data and adopting homomorphic encryption technology.

[0059] S13: Perform model aggregation, the aggregation server locally sums up each encrypted shared fragment, and then sends the total sum information to the central server.

[0060] S2. Perform decryption and recovery based on the processed participant data to obtain updated global model parameters, and send the updated global model parameters to the participants to generate new participant data for the next iteration.

[0061] In order to address the security bottleneck that is easily formed by a single aggregation server, secret sharing technology is used to distribute the secret fragments of local model parameters to different aggregation servers, and finally form global model parameters on a reliable central server, preventing fusion failure caused by device offline and increasing system stability.

[0062] S2 specifically includes:

[0063] S21: Perform model update. The central server decrypts and recovers the total sum information to obtain updated global model parameters, and resends the updated global model parameters to the participants for the next round of iteration.

[0064] A high-order tensor W is introduced as a memory unit, each mode of which corresponds to a spatial mapping of the data features of a participant.

[0065] Therefore, when fusing each heterogeneous data feature, the high-order tensor W can not only introduce the features of the remaining heterogeneous data modalities for correction, but also memorize the ongoing heterogeneous data modal features. When the heterogeneous data features to be processed are z1, z2, and z3, W corresponds to a 3rd-order tensor, and the 3 dimensions of this tensor correspond to the feature spaces of the 3 heterogeneous data features z1, z2, and z3. In heterogeneous data feature fusion, by modularly multiplying the heterogeneous data features with the feature space corresponding to the memory unit, a memory unit with the heterogeneous data features can be obtained, and further feature fusion operations can be performed.

[0066] In each training round, the data of the participants include training set, validation set and test set, and the loss function is calculated. By calculating the gradient of the loss function, the data of each participant generates the gradient information about the model parameters. To ensure data privacy, the Shamir secret sharing scheme is used to split the gradient information into multiple copies.

[0067] Secondly, each gradient shard is encrypted using homomorphic encryption technology to generate encrypted gradient shards and send them to the aggregation server. After receiving the encrypted gradient shards from each participant, the aggregation server can perform the update operation of the model parameters without exposing the original gradient information, thereby ensuring data privacy and model security.

[0068] In each training round, each aggregation server first adds the encrypted shared gradient parameters to obtain the encrypted cumulative gradient. Then, these summed gradients are sent to the central server. After receiving the data, the central server uses the homomorphic decryption method to decrypt the original secret shared fragments, and uses the Lagrange interpolation method to restore the complete average gradient. Finally, the model M is updated according to the learning rate α and the calculated gradient G*. After the set training rounds, the updated model M is output.

[0069] like Figure 5 , Figure 6 As shown, the formula is calculated according to the number of training rounds:

[0070] for epoch=1 to epochs: mainly used for cyclic calculation;

[0071] D train ,D valid ,D test = split_dataset(D i ) is mainly used to divide data sets;

[0072] Mainly used to calculate loss value;

[0073] Mainly used to calculate gradient values;

[0074] <g> = ShamirSecretSharing(g,t,m) is mainly used to generate (t,m), Shamir secret sharing. forj = 1 to m:

[0075] [<g ij >]=HE.enc(<g ij >) Homomorphically encrypt the secret sharing fragments sent to the aggregation server j and send [<g ij >] to aggregation server j;

[0076] End for

[0077] Return[<g i1 >],[<g i2 >],...,[<g im >];

[0078] End for

[0079] Among them, D i represents the data set, D train represents the training set, D valid represents the validation set, D test represents the test set, (x l ,y l ) is D train For a specific data set, M is the model, b is the batch size, and epochs represents the number of training rounds.

[0080] for epoch=1 to epochs: mainly used for cyclic calculation;

[0081] For an aggregation server:

[0082] for j=1 to m:

[0083] [<G i >]=[<g i1 >]+[<g i2 >]+...+[<g im >] is mainly used for sharing and adding, sending [<G i >] to the central server;

[0084] End for

[0085] For the central server:

[0086] [<G1>],[<G2>],...,[<G j >] is mainly used to receive the shared summation results from the aggregation server;

[0087] dec([<G1>],[<G2>],...,[<G j >]) is mainly used for homomorphic decryption;

[0088] G=Lagrange(<G1>,<G2>,...,<G j >) is mainly used to restore gradient using Lagrange interpolation;

[0089] Mainly used to calculate the average gradient;

[0090] M=M-α×G * Mainly used to update model M;

[0091] End for

[0092] ReturnM

[0093] Among them, [<g im >] represents the shared fragment gradient parameter of the mth power equipment group in the ith group, α is the learning rate, M is the model, and epochs represents the number of training rounds.

[0094] Lagrange interpolation is a polynomial interpolation technique used in numerical analysis. It approximates a continuous function by constructing a polynomial so that the function value of the polynomial at given data points is equal to the actual function value at these points.

[0095] The interpolation basis functions are:

[0096]

[0097] Similarly,

[0098] By extracting features from heterogeneous data and based on homomorphic encryption and Shamir secret sharing technology, secure fusion of multi-source heterogeneous data is achieved without leaking the privacy data of each participant. This solution can not only achieve the effect of model training on a single machine, but also effectively protect the privacy of the participants, and has better performance than the single-machine plaintext case. This provides a feasible solution for data factorization in new power systems and has the potential to promote the integration and comprehensive deployment of power flow, information flow and value flow.

[0099] The fused participant data is subjected to adaptive denoising to obtain the noise-filtered participant data.

[0100]

[0101] Among them, xf (x, y) is the fused M data, is the number of cycles of the sth layer of model i after preliminary denoising filtering under subsets x and y, and N is the total number of cycles. Through multi-dimensional weight optimization, the information of each data set is integrated to ensure the accuracy and consistency of the data.

[0102] A multi-source heterogeneous data fusion system for source, grid, load and storage based on cross-feature federated learning, the system uses a multi-source heterogeneous data fusion method for source, grid, load and storage based on cross-feature federated learning, the system includes a participant module, an aggregation server and a central server,

[0103] The participant module includes heterogeneous data feature extraction, secret generation, shared encryption, model training and evaluation function modules, which are used to randomly initialize parameters and generate key pairs (sk, pk), send global initialization parameters and public key pk to each participant, and perform model training, extract features from participant data and perform model training, use Shamir secret sharing scheme and homomorphic encryption technology to generate shared fragments, and upload them to the aggregation server for model aggregation;

[0104] The aggregation server includes a shared summing module for performing model aggregation. The aggregation server locally sums each encrypted shared fragment and then sends the total sum information to the central server.

[0105] The central server includes a decryption recovery module and a key generation module to perform model update. The central server decrypts and recovers the total sum information to obtain updated global model parameters, and resends the updated global model parameters to the participants for the next round of iteration.

[0106] A processor is used to run a program, wherein the program executes the above-mentioned multi-source heterogeneous data fusion method for source, grid, load and storage based on cross-feature federated learning when running.

[0107] Those of ordinary skill in the art will appreciate that the units (or modules, steps, etc., the same below) of the examples described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0108] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored, etc.

[0109] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0110] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nly Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A multi-source heterogeneous data fusion method for source, grid, load and storage based on cross-feature federated learning, characterized in that: The steps include the following: S1. Collect data from participants and preprocess the data, including heterogeneous data feature extraction, secret generation, shared encryption, model training and evaluation, and perform data fusion processing on the preprocessed data to obtain fused data of participants; S2. Perform decryption and recovery based on the processed participant data to obtain updated global model parameters, and send the updated global model parameters to the participants to generate new participant data for the next iteration.

2. A method for fusion of multi-source heterogeneous data for source, grid, load and storage based on cross-feature federated learning as claimed in claim 1, characterized in that: S1 specifically includes: S11: Randomly initialize parameters and generate key pairs (sk, pk), and send global initialization parameters and public key pk to each participant; S12: Perform model training, extract features from the data of the participants and perform model training, use the Shamir secret sharing scheme and homomorphic encryption technology to generate shared fragments, and upload them to the aggregation server for model aggregation; S13: Perform model aggregation, the aggregation server locally sums up each encrypted shared fragment, and then sends the total sum information to the central server.

3. The method for fusion of multi-source heterogeneous data for source, grid, load and storage based on cross-feature federated learning as claimed in claim 1, characterized in that: S2 specifically includes: S21: Perform model update. The central server decrypts and recovers the total sum information to obtain updated global model parameters, and resends the updated global model parameters to the participants for the next round of iteration.

4. The method for fusion of multi-source heterogeneous data for source, grid, load and storage based on cross-feature federated learning as claimed in claim 2, characterized in that: A high-order tensor W is introduced as a memory unit, each mode of which corresponds to a spatial mapping of the data features of a participant.

5. The method for fusion of multi-source heterogeneous data for source, grid, load and storage based on cross-feature federated learning as claimed in claim 2, characterized in that: The participant data includes a training set, a validation set and a test set, and a loss function is calculated. By calculating the gradient of the loss function, each participant data generates gradient information about the model parameters.

6. A method for fusion of multi-source heterogeneous data for source, grid, load and storage based on cross-feature federated learning as claimed in claim 5, characterized in that: Formula calculation based on the number of training rounds: D train ,D valid ,D test = split_dataset(D i ) <g>=ShamirSecretSharing(g,t,m) Among them, D i represents the data set, D train represents the training set, D valid represents the validation set, D test represents the test set, (x l ,y l ) is D train Specific data set, M is the model, and b is the batch size.

7. The method for fusion of multi-source heterogeneous data for source, grid, load and storage based on cross-feature federated learning as claimed in claim 1, characterized in that: Use Lagrange interpolation to restore the complete average gradient, based on the learning rate α and the calculated gradient G*, [<G i >]=[<g i1 >]+[<g i2 >]+...+[<g im >] G=Lagrange(<G1>,<G2>,...,<G j >) M=M-α×G * Among them, [<g im >] is the shared fragment gradient parameter, α is the learning rate, and M is the model.

8. The method for fusion of multi-source heterogeneous data for source, grid, load and storage based on cross-feature federated learning as claimed in claim 1, characterized in that: The fused participant data is subjected to adaptive denoising to obtain the noise-filtered participant data.

9. A multi-source heterogeneous data fusion system for source, grid, load and storage based on cross-feature federated learning as claimed in claim 9, characterized in that: The system uses a multi-source heterogeneous data fusion method for source, grid, load and storage based on cross-feature federated learning. The system includes a participant module, an aggregation server and a central server. The participant module includes heterogeneous data feature extraction, secret generation, shared encryption, model training and evaluation function modules, which are used to randomly initialize parameters and generate key pairs (sk, pk), send global initialization parameters and public key pk to each participant, and perform model training, extract features from participant data and perform model training, use Shamir secret sharing scheme and homomorphic encryption technology to generate shared fragments, and upload them to the aggregation server for model aggregation; The aggregation server includes a shared summing module for performing model aggregation. The aggregation server locally sums each encrypted shared fragment and then sends the total sum information to the central server. The central server includes a decryption recovery module and a key generation module to perform model update. The central server decrypts and recovers the total sum information to obtain updated global model parameters, and resends the updated global model parameters to the participants for the next round of iteration.

10. A processor, characterized in that: The processor is used to run the program, wherein when the program is running, the source-grid-load-storage multi-source heterogeneous data fusion method based on cross-feature federated learning of any one of claims 1 to 8 is executed.

Images