Distributed privacy protection method and system based on federal learning

Abstract

The invention discloses a distributed privacy protection method and system based on federated learning, and relates to the related field of privacy protection, and the method comprises the steps: a hospital end carries out the homomorphic encryption of patient data, calls an initial model through the encrypted data, and carries out the incremental training, and generates a first model parameter set; sending to a central node, aggregating by using a secure multi-party computing protocol, and generating a first aggregation parameter group; after differential privacy noise is added, the data is transmitted back to a hospital end, local patient data is continuously called to carry out incremental learning after homomorphic encryption, and so on until model prediction precision meets constraints, and a target model parameter set is generated; and after differential privacy noise is added, the data is transmitted back to the hospital end for target model generation. The technical problems that existing privacy protection is large in calculation overhead and high in communication cost, and accurate aggregation of model parameters is difficult to achieve while privacy is guaranteed are solved, and the technical effects of small calculation overhead, low communication cost, privacy protection and accurate aggregation of the model parameters are achieved.

Description

Technical Field

[0001] This application relates to the field of privacy protection, and particularly to a distributed privacy protection method and system based on federated learning. Background Art

[0002] In the medical field, the privacy protection of patient data is of crucial importance, while distributed data learning faces the risks of data leakage and privacy infringement. How to effectively utilize the patient data of each hospital for model training while ensuring data privacy and improving the model prediction accuracy is an urgent problem to be solved currently. At present, in order to protect data privacy, traditional methods such as data desensitization and encrypted transmission are usually adopted in data sharing and model training by each hospital. Although this can protect data privacy to a certain extent, in the scenario of federated learning, due to the need for multiple iterative training and model optimization, these methods will lead to large computational overhead, high communication cost, and it is difficult to accurately aggregate model parameters while ensuring privacy, especially when the data distribution is uneven or the data volume is large, these problems are particularly prominent.

[0003] In the related technologies at the present stage, there are technical problems in distributed privacy protection based on federated learning, such as large computational overhead, high communication cost, and difficulty in accurately aggregating model parameters while ensuring privacy. Summary of the Invention

[0004] This application provides a distributed privacy protection method and system based on federated learning. Each hospital end homomorphically encrypts the patient data, and uses the encrypted data to train an initial model to generate a first set of model parameters, sends the first set of model parameters to the central node, uses a secure multi-party computation protocol for aggregation to generate a first set of aggregated parameters, the central node adds differential privacy noise to the aggregated parameter set and then sends it back to the hospital end, the hospital end continues encrypted training based on the sent-back aggregated parameters, iterates multiple rounds until the model accuracy meets the requirements to generate a set of target model parameters, adds differential privacy noise to the set of target model parameters and then sends it back to the hospital end to generate a target model. By means of homomorphic encryption, cloud computing aggregation, differential privacy, and secure multi-party computation and other means, the technical effects of small computational overhead, low communication cost, privacy protection, and accurate aggregation of model parameters are achieved.

[0005] The present application provides a distributed privacy protection method based on federated learning, including: homomorphically encrypting the respective patient data by each hospital side, and using the encrypted patient data to call an initial model for incremental training to generate respective first model parameter groups corresponding to each hospital side; sending the respective first model parameter groups to a central node, and at the central node, aggregating the respective first model parameter groups using a secure multi-party computation protocol to generate a first aggregated parameter group; adding differential privacy noise to the first aggregated parameter group at the central node and then sending it back to each hospital side, and each hospital side continues to call the local patient data for incremental learning after homomorphic encryption based on the sent-back aggregated parameter, and so on, performing multiple rounds of training optimization until the model prediction accuracy meets a preset accuracy constraint to generate a target model parameter group; adding differential privacy noise to the target model parameter group and then sending it back to each hospital side for generating a target model.

[0006] In a possible implementation, the following processing is performed: the initial model is a model shared by each hospital side, and the initial model is constructed by any one of the hospital sides and then the initial model structure and initial model parameters are shared with other hospital sides.

[0007] In a possible implementation, when sending the respective first model parameter groups to the central node, and at the central node, aggregating the respective first model parameter groups using a secure multi-party computation protocol to generate a first aggregated parameter group, the following processing is performed: configuring the secure multi-party computation protocol applicable to N-party computation, where N is an integer greater than 2; each hospital side sends the encrypted respective first model parameter groups to the central node, and based on the secure multi-party computation protocol, performs weighted calculation on the respective first model parameter groups in the encrypted state to generate the first aggregated parameter group.

[0008] In a possible implementation, the following processing is performed: when each hospital side sends the encrypted respective first model parameter groups to the central node, it also sends local data scale information, and based on the local data scale information of each hospital side, sets the model aggregation weights of each hospital side in a relationship where the weight is proportional to the data scale, and performs weighted calculation on the respective first model parameter groups in the encrypted state using the model aggregation weights.

[0009] In a possible implementation, after adding differential privacy noise to the first aggregated parameter group at the central node, it is sent back to each hospital end, and the following processing is performed: receiving the respective privacy budget information of each hospital end; performing differential privacy noise intensity matching based on the respective privacy budget information to generate respective noise intensity information; after adding differential privacy noise to the first aggregated parameter group with the respective noise intensity information, it is sent back to each hospital end according to the corresponding relationship between the noise intensity and the hospital end.

[0010] In a possible implementation, for performing differential privacy noise intensity matching based on the respective privacy budget information to generate respective noise intensity information, the following processing is performed: based on the Laplace mechanism, calculating the noise addition intensity according to the respective privacy budget information to generate respective Laplace noise scale parameters; generating the respective noise intensity information with the respective Laplace noise scale parameters.

[0011] In a possible implementation, after adding differential privacy noise to the first aggregated parameter group at the central node and sending it back to each hospital end, each hospital end continues to perform incremental learning after homomorphic encryption of local patient data with the sent-back aggregated parameters, and so on, performing multiple rounds of training optimization until the model prediction accuracy meets the preset accuracy constraint to generate a target model parameter group, and the following processing is performed: updating the initial model of each hospital end with the sent-back aggregated parameters to generate respective local updated models; based on the respective local updated models, continuing to perform incremental learning after homomorphic encryption of local patient data to generate respective second model parameter groups, and sending them to the central node for aggregation and adding differential privacy noise and then sending them back to each hospital end; and so on, performing multiple rounds of training optimization until the model accuracy test results of each model after updating the model with the aggregated model parameter group sent back to each hospital end all meet the preset accuracy constraint, stopping the training, and generating a target model parameter group.

[0012] In a possible implementation, when performing multiple rounds of training optimization, the following processing is also performed: during the multiple rounds of training optimization, if in any round of optimization, the model accuracy test result of any hospital end after updating the model with the aggregated model parameter group meets the preset accuracy constraint, marking the any hospital end as an exiting party; after extracting the encrypted summary based on gradient statistics for the exiting party and uploading the encrypted summary to the central node, deleting the exiting party from each hospital end; and the central node performing global aggregation with reference to the historical aggregated contribution of the exiting party based on the encrypted summary.

[0013] In a possible implementation, an encrypted digest based on gradient statistics is extracted from the exiting party, and the following processing is performed: the historical model gradient sequence of the exiting party is extracted with a historical round limit; the mean and variance of the gradients are calculated based on the historical model gradient sequence to form a statistical digest; the statistical digest is encrypted and uploaded to the server through homomorphic encryption to generate the encrypted digest.

[0014] The present application also provides a distributed privacy protection system based on federated learning, including: a first model parameter group generation module, configured to perform homomorphic encryption on the respective patient data of each hospital end, and use the encrypted patient data to call an initial model for incremental training to generate respective first model parameter groups corresponding to each hospital end; a first aggregated parameter group generation module, configured to send the respective first model parameter groups to a central node, and at the central node, use a secure multi-party computation protocol to aggregate the respective first model parameter groups to generate a first aggregated parameter group; a target model parameter group generation module, configured to add differential privacy noise to the first aggregated parameter group at the central node and then send it back to each hospital end, and each hospital end continues to call the local patient data for incremental learning after homomorphic encryption based on the sent-back aggregated parameter, and so on, perform multiple rounds of training optimization until the model prediction accuracy meets a preset accuracy constraint to generate a target model parameter group; a target model generation module, configured to add differential privacy noise to the target model parameter group and then send it back to each hospital end for generating a target model.

[0015] It is intended to achieve, through the distributed privacy protection method and system based on federated learning proposed in the present application, first, perform homomorphic encryption on the respective patient data of each hospital end, use the encrypted patient data to call an initial model for incremental training to generate respective first model parameter groups corresponding to each hospital end, then send the respective first model parameter groups to a central node, and at the central node, use a secure multi-party computation protocol to aggregate the respective first model parameter groups to generate a first aggregated parameter group, then add differential privacy noise to the first aggregated parameter group at the central node and send it back to each hospital end, and each hospital end continues to call the local patient data for incremental learning after homomorphic encryption based on the sent-back aggregated parameter, and so on, perform multiple rounds of training optimization until the model prediction accuracy meets a preset accuracy constraint to generate a target model parameter group, and finally add differential privacy noise to the target model parameter group and send it back to each hospital end for generating a target model. The technical effects of small computational overhead, low communication cost, privacy protection, and precise aggregation of model parameters are achieved through means such as homomorphic encryption, cloud computing aggregation, differential privacy, and secure multi-party computation. Description of the Drawings

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly introduced below. Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of this application. It should be understood that the operations described above or below do not necessarily need to be performed precisely in sequence. On the contrary, as needed, various steps can be performed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 It is a schematic flowchart of the distributed privacy protection method based on federated learning provided by the embodiments of this application.

[0018] Figure 2 It is a schematic structural diagram of the distributed privacy protection system based on federated learning provided by the embodiments of this application.

[0019] Explanation of reference numerals: The first model parameter group generation module 10, the first aggregated parameter group generation module 20, the target model parameter group generation module 30, and the target model generation module 40. Detailed implementation manners

[0020] The above description is only an overview of the technical solutions of this application. In order to be able to understand the technical means of this application more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features, and advantages of this application more obvious and understandable, the specific implementation manners of this application are specifically given below.

[0021] In order to make the purpose, technical solutions, and advantages of this application clearer, the present application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

[0022] In the following description, "some embodiments" are involved, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. The terms "first / second" involved are only used to distinguish similar objects and do not represent a specific order for the objects. The terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that comprises a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application.

[0023] The embodiments of this application provide a distributed privacy protection method based on federated learning, as Figure 1 shown, the method includes:

[0024] Step S100, homomorphically encrypt the respective patient data by each hospital end, and use the encrypted patient data to call the initial model for incremental training to generate respective first model parameter groups corresponding to each hospital end.

[0025] Specifically, the homomorphic encryption algorithm is used to encrypt the patient data of each hospital end. Homomorphic encryption is a special encryption method that allows specific algebraic operations to be performed on ciphertext, and the result after decryption is the same as the result of performing the same operation on the plaintext. For example, using the additive homomorphic encryption algorithm, for plaintext data a and b, and the corresponding ciphertexts E(a) and E(b), it satisfies E(a)+E(b)=E(a + b). Each hospital end uses the encrypted patient data to perform incremental training on the initial model. Incremental training means that on the basis of an existing model, the model is updated and trained using new data, rather than training from scratch. For example, the initial model is a neural network. Each hospital end calculates the gradient update values of the model parameters locally using the encrypted patient data, and then adjusts the initial model according to these gradient update values to generate their respective first model parameter groups. For example, assume that a certain hospital end has a patient dataset D and the initial model is M0. The dataset D is encrypted as E(D) through the homomorphic encryption algorithm, and then E(D) is used to perform incremental training on M0. The specific process is: calculate the gradient update value ΔM of the model parameters locally, and update M0 according to ΔM to obtain the first model parameter group M1 of this hospital end.

[0026] In a possible implementation, step S100 further includes step S110. The initial model is a model shared by each hospital terminal, and the initial model is constructed by any one of the hospital terminals and then the initial model structure and initial model parameters are shared with other hospital terminals.

[0027] Specifically, the initial model is constructed by any one of the hospital terminals and then shared with other hospital terminals. The hospital terminal that constructs the initial model serializes the architecture information of the model (such as the number of layers of the neural network, the number of nodes in each layer, the activation function, etc.) into a transmissible format, such as JSON or Protobuf. The initial parameters of the model (such as weights and biases) are also serialized and transmitted to other hospital terminals through a secure communication protocol (such as TLS / SSL). For example, assume that Hospital A constructs a neural network model for disease diagnosis, and its structure includes an input layer, two hidden layers, and an output layer. Hospital A serializes the model structure and initial parameters into JSON format and sends the above JSON data to other hospitals (such as Hospital B and Hospital C) through an HTTP request encrypted by TLS. After receiving the data, these hospitals deserialize and load the model structure and initial parameters for subsequent local training. The role of this implementation method in the entire solution is to realize the sharing of the initial model and provide a basis for subsequent distributed training. By allowing one hospital to construct and share the initial model, it can ensure that all participating hospitals start training from the same starting point, thereby improving the consistency and convergence speed of the model. In addition, this method reduces the resource consumption of each hospital constructing the model alone and improves the efficiency of the entire federated learning system.

[0028] Step S200: Send each first model parameter group to the central node. At the central node, use a secure multi-party computation protocol to aggregate each first model parameter group to generate a first aggregated parameter group.

[0029] Specifically, the central node is located in the cloud, and each hospital terminal sends the generated first model parameter group to the central node through a secure communication channel. Use a secure multi-party computation protocol to aggregate the first model parameter groups of each hospital terminal at the central node. The secure multi-party computation protocol allows multiple participants to jointly complete the calculation of a certain function without revealing their respective inputs. For example, adopt a secret sharing scheme, split each hospital terminal's first model parameter group into multiple secret shares and distribute them to different participants, and then through a specific calculation and interaction process, recover the aggregated first aggregated parameter group. For example, assume there are three hospitals, and their first model parameter groups are M 1a 、M 1b and M 1c . Using the secret sharing scheme, split M1a into S a1 、S a2 and Sa3 , split M 1b into S b1 , S b2 and S b3 , split M 1c into S c1 , S c2 and S c3 . Then send S a1 , S b1 and S c1 to Party 1, send S a2 , S b2 and S c2 to Party 2, send S a3 , S b3 and S c3 to Party 3. Party 1, Party 2, and Party 3 respectively calculate and interact on the received secret shares, and finally recover the first aggregation parameter group M agg1 .

[0030] In a possible implementation, send the respective first model parameter groups to a central node. At the central node, use a secure multi-party computation protocol to aggregate the respective first model parameter groups to generate a first aggregation parameter group. Step S200 further includes step S210 of configuring the secure multi-party computation protocol applicable to N-party computation, where N is an integer greater than 2. Specifically, configure a secure multi-party computation (SMPC) protocol applicable to multiple parties (N parties, N > 2), such as a protocol based on secret sharing. Such a protocol allows data to be split into multiple parts, each party holds a part, and the final result is recovered through specific interactions and calculations without exposing the original data. The Shamir secret sharing scheme can be adopted. This scheme splits the secret into multiple shares and distributes them to the parties. Only when a certain number of shares are reached can the original secret be recovered. Set the parameters of the protocol according to the number of parties and security requirements, such as the threshold of secret sharing (i.e., the minimum number of shares required to recover the secret). Determine the communication mechanism between the parties to ensure the security and efficiency of data transmission.

[0031] Step S220: Each hospital end sends the encrypted first model parameter groups to the central node, and based on the secure multi-party computation protocol, performs weighted computation on the encrypted first model parameter groups to generate the first aggregated parameter group. Specifically, each hospital end uses a homomorphic encryption algorithm to encrypt its own first model parameter group. Homomorphic encryption allows computations to be directly performed on encrypted data, and the decrypted result is the same as the result of the computation on the original data. For example, using the Paillier homomorphic encryption algorithm, addition and scalar multiplication operations can be performed on the encrypted data. Based on the secure multi-party computation protocol, weighted computation is performed on the encrypted first model parameter groups. Each participant processes the local encrypted parameters according to the protocol rules and interacts with other participants to obtain the computation results. For example, when using the secret sharing scheme, each participant splits the local encrypted parameters into multiple shares and distributes them to other participants, and then through the computation and interaction processes specified by the protocol, the weighted aggregated result is obtained. Through the above weighted computation process, the first aggregated parameter group is generated. This parameter group is the result in the encrypted state. The central node cannot directly obtain the specific parameters of each hospital end, but it can be used for subsequent model updates. For example, assume that hospitals A, B, and C respectively generate the encrypted first model parameter groups E(M 1a ), E(M 1b ), and E(M 1c ). Using the Shamir secret sharing scheme, each hospital splits its local parameters into three shares and distributes them to other hospitals. Through the computation and interaction specified by the protocol, the encrypted first aggregated parameter group E(M agg1 ) is finally generated at the central node. This parameter group is the weighted result of the parameters of the three hospitals. The role of this implementation method in the entire solution is to ensure that the data privacy of each participant is protected while aggregating the model parameters, and at the same time, the aggregated result can be efficiently generated. By configuring a secure multi-party computation protocol applicable to multiple participants, weighted computation in the encrypted state can be achieved, avoiding the central node directly obtaining the specific parameters of each hospital end, thereby enhancing the privacy protection ability of the entire federated learning system.

[0032] In a possible implementation, step S220 further includes step S221. When each hospital end sends the encrypted first model parameter groups to the central node, it also sends the local data scale information. Based on the local data scale information of each hospital end, the model aggregation weights of each hospital end are set in a relationship where the weight is proportional to the data scale, and weighted computation in the encrypted state is performed on the first model parameter groups using the model aggregation weights.

[0033] Specifically, while each hospital side sends the encrypted first model parameter group, it collects and sends local data scale information. The data scale information can be the quantity of patient data, the dimension of the data, etc., which is used to reflect the scale of data at each hospital side. For example, Hospital A has 1000 patient data, Hospital B has 800 patient data, and Hospital C has 1200 patient data. According to the local data scale information of each hospital side, model aggregation weights are set. The weight is proportional to the data scale, that is, the larger the data scale, the higher the weight. For example, if based on the quantity of patient data, the weights of Hospital A, B, and C are 1000, 800, and 1200 respectively. For the convenience of calculation, the weights can be normalized. For example, the weight of Hospital A is 0.33, Hospital B is 0.26, and Hospital C is 0.41. Using a secure multi-party computation protocol, a weighted calculation is performed on the encrypted first model parameter group. In the encrypted state, the model parameter groups of each hospital side are weighted and averaged according to the set weights.

[0034] For example, assume that the encrypted model parameters of Hospital A, B, and C are E(M 1a )、E(M 1b ) and E(M 1c ), and the weights are 0.33, 0.26, and 0.41 respectively. Then the aggregated parameter group after weighted calculation is E(0.33×M 1a +0.26×M 1b +0.41×M 1c ), and this parameter group is the weighted result of the parameters of the three hospitals. This implementation method can avoid the situation where hospitals with smaller data scales have too much influence on the aggregation result by setting weights according to the data scale, while ensuring that the contributions of hospitals with larger data scales are reasonably reflected. This helps to make full use of the data resources of each hospital on the premise of privacy protection, optimize the performance of the federated learning model, and ultimately achieve a balance between privacy protection and model effect.

[0035] Step S300, after adding differential privacy noise to the first aggregated parameter group at the central node, it is sent back to each hospital side. Each hospital side continues to call the local patient data for incremental learning after homomorphic encryption based on the sent-back aggregated parameters, and so on. Multiple rounds of training optimization are performed until the model prediction accuracy meets the preset accuracy constraint, and a target model parameter group is generated.

[0036] Specifically, the central node adds differential privacy noise to the first aggregated parameter group. Differential privacy is a privacy protection mechanism that adds noise to data or model parameters, making it difficult for attackers to infer information about individual individuals from the results. For example, the Laplace noise mechanism is adopted. According to the preset privacy budget ∈, the parameters of the noise are calculated, and then the noise is added to each parameter of the first aggregated parameter group. The first aggregated parameter group with added noise is sent back to each hospital end, and each hospital end continues to perform incremental learning using the locally encrypted patient data. For example, after each hospital end receives the sent-back aggregated parameter group, it uses it as the new initial model parameter group, calculates the gradient update value again using the locally encrypted patient data, and adjusts the model parameters according to the update value to generate a new model parameter group and continue the next round of training. This process is repeated for multiple rounds, and a new model parameter group is generated in each round and sent to the central node for aggregation and differential privacy noise addition. When the model prediction accuracy meets the preset accuracy constraint, a target model parameter group is generated, which can be determined by evaluating the model performance on the validation set.

[0037] For example, the first aggregated parameter group is M agg1 , and the privacy budget ∈ = 1. According to the Laplace noise mechanism, the noise parameters are calculated, and corresponding noise is added to each parameter of M agg1 to obtain the parameter group M' with added noise agg1 . M' agg1 is sent back to each hospital end, and each hospital end uses the locally encrypted patient data to perform incremental training on M' agg1 to generate new model parameter groups, such as M 2a , M 2b and M 2c , and continue the next round of training.

[0038] In a possible implementation manner, after adding differential privacy noise to the first aggregated parameter group at the central node and sending it back to each hospital end, step S300 further includes step S310 of receiving the privacy budget information of each hospital end. Specifically, the central node receives the privacy budget information (∈ value) sent by each hospital end. The privacy budget ∈ is an important parameter in differential privacy, which is used to quantify the strength of privacy protection. A smaller ∈ value represents stronger privacy protection. For example, hospitals A, B, and C respectively set the privacy budgets ∈ A = 1.0, ∈ B = 0.8, ∈ C = 1.2.

[0039] Step S320: Based on the respective privacy budget information, perform differential privacy noise intensity matching to generate respective noise intensity information. Specifically, according to the privacy budget ∈ of each hospital side, calculate the corresponding noise intensity. The noise intensity is inversely proportional to ∈, that is, the smaller the privacy budget, the greater the noise intensity to be added.

[0040] For example, using the Laplace noise mechanism, the noise intensity λ can be calculated by the formula where Δf is the sensitivity of the model parameters. Assuming Δf = 1, the noise intensity of Hospital A is λ A = 1, that of Hospital B is λ B = 1.25, and that of Hospital C is λ C = 0.83.

[0041] Step S330: After adding differential privacy noise to the first aggregation parameter group with the respective noise intensity information, according to the corresponding relationship between the noise intensity and the hospital side, send it back to each hospital side. Specifically, for each parameter of the first aggregation parameter group, add noise according to the corresponding noise intensity. For example, for the aggregation parameter M agg1 , the parameter after adding Laplace noise is M' agg1 = M agg1 + Laplace(λ), where λ is the noise intensity calculated according to the privacy budget. The central node sends the parameter group M' agg1 added with noise back to each hospital side according to the corresponding relationship between the noise intensity and the hospital side. This implementation method can generate noise parameters that meet the privacy requirements of each hospital side by receiving the privacy budget information of each hospital side and calculating the corresponding noise intensity according to the privacy budget, ensuring that the privacy needs of each hospital side are met during the model parameter aggregation and sending-back process.

[0042] In a possible implementation method, when performing differential privacy noise intensity matching based on the respective privacy budget information to generate respective noise intensity information, step S320 further includes step S321: Based on the Laplace mechanism, calculate the noise addition intensity according to the respective privacy budget information to generate respective Laplace noise scale parameters. Specifically, the Laplace mechanism protects privacy by adding an appropriate amount of noise to the query result. The noise follows a Laplace distribution, and its probability density function is: where θ is the location parameter, b is the scale parameter, x is the random variable of the Laplace distribution, representing the noise value to be added or the perturbed data value. In differential privacy, the scale parameter b is set to

[0043] After the central node receives the privacy budget ∈ information from each hospital end, it calculates the corresponding Laplace noise scale parameter b according to the privacy budget of each hospital. Continuing with the example in step S320, it can be known that b A = 1, b B = 1.25, b C = 0.83.

[0044] Step S322, generate the respective noise intensity information using the respective Laplace noise scale parameters. Specifically, according to the calculated Laplace noise scale parameter b, specific noise intensity information is generated, and the noise intensity information is used to guide the subsequent noise addition process. That is, λ A = b A＝ 1, λ B = b B = 1.25, λ C = b C＝ 0.83.

[0045] In a possible implementation manner, after the central node adds differential privacy noise to the first aggregation parameter group and then transmits it back to each hospital end, each hospital end continues to call the local patient data for incremental learning after homomorphic encryption based on the transmitted-back aggregation parameter, and so on, performing multiple rounds of training optimization until the model prediction accuracy meets the preset accuracy constraint to generate the target model parameter group. Step S300 further includes step S340, updating the initial models of each hospital end using the transmitted-back aggregation parameter to generate respective local updated models. Specifically, after each hospital end receives the aggregation parameter transmitted back by the central node, it uses these parameters to update the local initial model. The update process includes directly replacing the corresponding parameters of the local model with the aggregation parameter, or combining the aggregation parameter with the local model parameters through a certain fusion mechanism. For example, after hospital A receives the aggregation parameter M' agg1 , it uses it as the new model parameter to update the local model M localA , generating a new local updated model

[0046] Step S350, based on the respective local updated models, continue to call the local patient data for incremental learning after homomorphic encryption to generate respective second model parameter groups, and send them to the central node for aggregation and differential privacy noise addition and then transmit them back to each hospital end. Specifically, each hospital end uses the updated local model and local patient data for incremental learning, that is, on the basis of the local updated model, the model is updated and trained using the data. For example, hospital A uses the updated model and the locally encrypted patient data E(D A ) for incremental training to generate a new model parameter group M 2a。Each hospital end will generate the second model parameter group M 2a 、M 2b and M 2c and send them to the central node.

[0047] Step S360, and so on, perform multiple rounds of training optimization until the accuracy test results of each model after updating the model with the aggregated model parameter group back-transmitted to each hospital end all meet the preset accuracy constraint, then stop the training and generate the target model parameter group. Specifically, after the central node receives the second model parameter groups sent by each hospital end, it repeatedly performs the aggregation and differential privacy noise addition operations to generate a new aggregated parameter group and back-transmit it to each hospital end. Each hospital end continues to update the local model with the back-transmitted aggregated parameters and perform incremental learning to generate a new model parameter group and send it to the central node. After each round of training, each hospital end uses the local data to perform an accuracy test on the updated model. When the accuracy test results of all hospital ends' models all meet the preset accuracy constraint, stop the training. For example, the preset accuracy constraint is that the model accuracy rate is not less than 90%. If the model accuracy rates of hospitals A, B, and C are 92%, 91%, and 93% respectively, then the condition is met and the training is stopped. This implementation method can enable each hospital end to make full use of the local data resources to optimize the model performance until the preset accuracy requirements are met while protecting data privacy by continuously updating the local model and performing incremental learning.

[0048] In a possible implementation method, when performing multiple rounds of training optimization, step S360 further includes step S361. During the multiple rounds of training optimization, if in any round of optimization, the accuracy test result of the model after updating the model with the aggregated model parameter group by any hospital end meets the preset accuracy constraint, mark the any hospital end as an exiting party. Specifically, after each hospital end receives the aggregated model parameters back-transmitted by the central node, it uses the local data to perform an accuracy test on the updated model. The accuracy test includes indicators such as accuracy rate and recall rate. For example, after a certain round of training in hospital A, it uses the local data to test the updated model and finds that the accuracy rate reaches 92%, meeting the preset accuracy constraint (such as 90%). If the accuracy test result of a certain hospital end's model meets the preset accuracy constraint, then mark this hospital end as an exiting party. The exiting party will no longer participate in the subsequent training process.

[0049] Step S362: After extracting the encryption digest based on gradient statistics for the exiting party and uploading the encryption digest to the central node, delete the exiting party from each hospital end. Specifically, perform statistical analysis on the model gradients of the exiting party, extract key information (such as the mean and variance of the gradients), and use a homomorphic encryption algorithm to encrypt this information to generate an encryption digest. For example, use the Paillier homomorphic encryption algorithm to encrypt the gradient statistical information. Upload the encryption digest to the central node. The central node uses these encryption digests for subsequent global aggregation. Delete the exiting party from the list of hospital ends participating in the training, and the subsequent training will no longer involve this exiting party.

[0050] Step S363: The central node performs global aggregation based on the encryption digest with reference to the historical aggregation contribution of the exiting party. Specifically, when the central node performs global aggregation, it refers to the historical aggregation contribution of the exiting party. The historical contribution can be reflected by the encryption digest uploaded by the exiting party. The central node combines the encryption digests of all participating parties (including the exiting party) to perform global aggregation and generate new aggregated model parameters. For example, during multiple rounds of training, after the model accuracy of Hospital A reaches 92% in the 3rd round of training, which meets the preset accuracy constraint of 90%, it is marked as an exiting party. Hospital A extracts the statistical information (such as the mean and variance) of its model gradients, encrypts them using the Paillier homomorphic encryption algorithm, generates an encryption digest and uploads it to the central node. In subsequent global aggregations, the central node refers to the encryption digest of Hospital A and combines the parameters of other hospital ends to generate new aggregated model parameters. This implementation method allows hospital ends that meet the accuracy requirements to exit early during multiple rounds of training, reducing unnecessary training rounds, saving computing resources and time. At the same time, by extracting and uploading the encryption digest, the central node can make full use of the historical contribution of the exiting party without revealing privacy, ensuring the accuracy and reliability of the global model.

[0051] In a possible implementation, for extracting the encryption digest based on gradient statistics for the exiting party, step S362 further includes step S3621: Extract the historical model gradient sequence of the exiting party with a historical round limit. Specifically, set a historical round limit T, which means that when extracting the historical model gradients, only consider the training data of the most recent T rounds. This helps reduce the computational amount and storage requirements, and at the same time avoids the influence of outdated information on the current model. For example, set T = 5, which means only extract the model gradient information of the most recent 5 rounds. Extract the model gradient sequence of the most recent T rounds from the local record of the exiting party. These gradient sequences reflect the changes of the model in these rounds. For example, Hospital A meets the accuracy requirements and exits after the 3rd round of training, and extract its gradient sequence of the most recent 5 rounds (assuming the training has been carried out for more than 5 rounds).

[0052] Step S3622: Calculate the mean and variance of the gradients based on the historical model gradient sequence to form a statistical summary. Specifically, for the extracted historical model gradient sequence, calculate the mean and variance of each parameter. The mean and variance are used as the statistical summary of the gradients for subsequent aggregation calculations. For example, for the gradient sequence calculate the mean μ and variance σ of each parameter 2 : where i is an index variable representing the i-th gradient in the gradient sequence. Combine the calculated mean and variance into a statistical summary, which can be expressed as (μ, σ 2 ).

[0053] Step S3623: Encrypt and upload the statistical summary to the server through homomorphic encryption to generate the encrypted summary. Specifically, use a homomorphic encryption algorithm (such as the Paillier algorithm) to encrypt the statistical summary. For example, use the Paillier algorithm to encrypt the statistical summary (μ, σ 2 ) to generate the encrypted summary E(μ, σ 2 ). Upload the encrypted summary to the central node. The central node can use these encrypted summaries for subsequent global aggregation without decrypting the specific data, thereby protecting data privacy. This implementation method extracts the historical model gradient sequence and calculates the statistical summary. The central node can perform global aggregation with reference to the historical contributions of the exiting parties without revealing the specific data, which helps to achieve a balance between privacy protection and model performance and ultimately realizes efficient model training under privacy protection.

[0054] Step S400: Add differential privacy noise to the target model parameter group and then send it back to each hospital end for generating the target model.

[0055] Specifically, add differential privacy noise to the target model parameter group again and then send it back to each hospital end. Each hospital end generates the target model according to the final parameter group. For example, after multiple rounds of training and optimization, the target model parameter group M final is obtained. Again, adopt the differential privacy noise addition mechanism to add noise to M final to obtain M' final , and then send M' final back to each hospital end. Each hospital end generates the target model according to M' finalGenerate a final target model for subsequent tasks such as patient data prediction. In the embodiments of this application, each hospital end homomorphically encrypts patient data, trains an initial model with the encrypted data to generate a first model parameter group, sends the first model parameter group to the central node, aggregates it using a secure multi-party computation protocol to generate a first aggregated parameter group, the central node adds differential privacy noise to the aggregated parameter group and then sends it back to the hospital end, the hospital end continues to encrypt and train based on the returned aggregated parameters, iterates multiple rounds until the model accuracy meets the requirements, generates a target model parameter group, adds differential privacy noise to the target model parameter group and then sends it back to the hospital end to generate the target model and other technical means, achieving the technical effects of small computational overhead, low communication cost, privacy protection, and precise aggregation of model parameters through means such as homomorphic encryption, cloud computing aggregation, differential privacy, and secure multi-party computation.

[0056] In the above, reference is made to Figure 1 describe in detail the distributed privacy protection method based on federated learning according to the embodiments of the present invention. Next, reference will be made to Figure 2 describe the distributed privacy protection system based on federated learning according to the embodiments of the present invention.

[0057] The distributed privacy protection system based on federated learning according to the embodiments of the present invention is used to solve the technical problems existing in the existing privacy protection, such as large computational overhead, high communication cost, and difficulty in achieving precise aggregation of model parameters while ensuring privacy, achieving the technical effects of small computational overhead, low communication cost, privacy protection, and precise aggregation of model parameters through means such as homomorphic encryption, cloud computing aggregation, differential privacy, and secure multi-party computation. The distributed privacy protection system based on federated learning includes: a first model parameter group generation module 10, a first aggregated parameter group generation module 20, a target model parameter group generation module 30, and a target model generation module 40.

[0058] The first model parameter group generation module 10 is used to perform homomorphic encryption on the patient data of each hospital end respectively, and call the initial model for incremental training with the encrypted patient data to generate the respective first model parameter groups corresponding to the respective hospital ends; the first aggregation parameter group generation module 20 is used to send the respective first model parameter groups to the central node, and at the central node, aggregate the respective first model parameter groups using a secure multi-party computation protocol to generate a first aggregation parameter group; the target model parameter group generation module 30 is used to add differential privacy noise to the first aggregation parameter group at the central node and then send it back to the respective hospital ends, and the respective hospital ends continue to call the local patient data for incremental learning after homomorphic encryption based on the sent-back aggregation parameter, and so on, perform multiple rounds of training optimization until the model prediction accuracy meets the preset accuracy constraint to generate a target model parameter group; the target model generation module 40 is used to add differential privacy noise to the target model parameter group and then send it back to the respective hospital ends for generating a target model.

[0059] Next, the specific configuration of the first model parameter group generation module 10 will be described in detail. As described above, the first model parameter group generation module 10 may further include: an initial model construction unit, where the initial model is a model shared by the respective hospital ends, and the initial model is constructed by any one of the respective hospital ends and then the initial model structure and initial model parameters are shared with other hospital ends.

[0060] Next, the specific configuration of the first aggregation parameter group generation module 20 will be described in detail. As described above, the respective first model parameter groups are sent to the central node, and at the central node, the respective first model parameter groups are aggregated using a secure multi-party computation protocol to generate a first aggregation parameter group. The first aggregation parameter group generation module 20 may further include: a secure multi-party computation protocol configuration unit for configuring the secure multi-party computation protocol applicable to N-party computation, where N is an integer greater than 2; a weighted calculation unit for the respective hospital ends to send the encrypted respective first model parameter groups to the central node, and perform weighted calculation on the respective first model parameter groups in the encrypted state based on the secure multi-party computation protocol to generate the first aggregation parameter group.

[0061] Among them, the weighted calculation unit may further include: a model aggregation weight setting sub-unit for the respective hospital ends to send the encrypted respective first model parameter groups to the central node and at the same time send the local data scale information, and based on the local data scale information of the respective hospital ends, set the model aggregation weights of the respective hospital ends in a relationship where the weight is proportional to the data scale, and perform weighted calculation on the respective first model parameter groups in the encrypted state with the model aggregation weights.

[0062] Next, the specific configuration of the target model parameter group generation module 30 will be described in detail. As described above, after adding differential privacy noise to the first aggregated parameter group at the central node and then transmitting it back to each hospital end, the target model parameter group generation module 30 may further include: a privacy budget information receiving unit for receiving the privacy budget information of each hospital end; a differential privacy noise intensity matching unit for performing differential privacy noise intensity matching based on the privacy budget information of each hospital end to generate respective noise intensity information; a differential privacy noise adding unit for adding differential privacy noise to the first aggregated parameter group respectively with the respective noise intensity information, and then transmitting it back to each hospital end according to the correspondence between the noise intensity and the hospital end.

[0063] Among them, for performing differential privacy noise intensity matching based on the privacy budget information of each hospital end to generate respective noise intensity information, the differential privacy noise intensity matching unit may further include: a noise addition intensity calculation subunit for calculating the noise addition intensity based on the Laplace mechanism according to the privacy budget information of each hospital end to generate respective Laplace noise scale parameters; a noise intensity information generation subunit for generating the respective noise intensity information with the respective Laplace noise scale parameters.

[0064] Among them, after adding differential privacy noise to the first aggregated parameter group at the central node and then transmitting it back to each hospital end, each hospital end continues to call the local patient data for incremental learning after homomorphic encryption based on the transmitted aggregated parameter, and so on, performing multiple rounds of training optimization until the model prediction accuracy meets the preset accuracy constraint to generate the target model parameter group. The target model parameter group generation module 30 may further include: an initial model update unit for updating the initial models of each hospital end with the transmitted aggregated parameter to generate respective local updated models; a second model parameter group generation unit for continuing to call the local patient data for incremental learning after homomorphic encryption based on the respective local updated models to generate respective second model parameter groups, and sending them to the central node for aggregation and differential privacy noise addition and then transmitting them back to each hospital end; a multiple-round training optimization unit for performing multiple rounds of training optimization in this way until the respective model accuracy test results after updating the model with the aggregated model parameter group transmitted back to each hospital end all meet the preset accuracy constraint, stopping the training, and generating the target model parameter group.

[0065] Among them, multiple rounds of training optimization are performed. The multiple-round training optimization unit may further include: an exit party marking subunit for marking any hospital end as an exit party during the multiple-round training optimization process if the model accuracy test result after updating the model with the aggregated model parameter group by any hospital end in any round of the optimization process meets the preset accuracy constraint; an exit party deletion subunit for deleting the exit party from the various hospital ends after extracting the encrypted summary based on gradient statistics for the exit party and uploading the encrypted summary to the central node; and a global aggregation subunit for the central node to perform global aggregation based on the encrypted summary with reference to the historical aggregation contribution of the exit party.

[0066] Among them, for extracting the encrypted summary based on gradient statistics for the exit party, the exit party deletion subunit may further include: a historical model gradient sequence extraction component for extracting the historical model gradient sequence of the exit party with a historical round limit; a mean and variance calculation component for calculating the mean and variance of the gradient based on the historical model gradient sequence to form a statistical summary; and an encrypted summary generation component for encrypting and uploading the statistical summary to the server through homomorphic encryption to generate the encrypted summary.

[0067] The distributed privacy protection system based on federated learning provided by the embodiments of the present invention can execute the distributed privacy protection method based on federated learning provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0068] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The included respective units and modules are only divided according to the functional logic, but are not limited to the above division as long as the corresponding functions can be achieved; in addition, the specific names of the respective functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0069] The above specific embodiments do not constitute a limitation to the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present application shall be included within the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be executed in a different order from that in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A distributed privacy protection method based on federated learning, characterized in that Including: Each hospital end homomorphically encrypts its own patient data respectively, and uses the encrypted patient data to call the initial model for incremental training to generate each first model parameter group corresponding to each hospital end; Send each first model parameter group to the central node. At the central node, use the secure multi-party computation protocol to aggregate each first model parameter group to generate a first aggregated parameter group; Add differential privacy noise to the first aggregated parameter group at the central node and then send it back to each hospital end. Each hospital end continues to call the local patient data for incremental learning after homomorphic encryption based on the back-transmitted aggregated parameter, and so on, perform multiple rounds of training optimization until the model prediction accuracy meets the preset accuracy constraint to generate a target model parameter group; Add differential privacy noise to the target model parameter group and then send it back to each hospital end for the generation of the target model.

2. The distributed privacy protection method based on federated learning according to claim 1, characterized in that, The initial model is a model shared by each hospital end, and the initial model is constructed by any one of the hospital ends and then the initial model structure and initial model parameters are shared with other hospital ends.

3. The distributed privacy protection method based on federated learning according to claim 1, characterized in that Send each first model parameter group to the central node. At the central node, use the secure multi-party computation protocol to aggregate each first model parameter group to generate a first aggregated parameter group, including: Configure the secure multi-party computation protocol applicable to N-party computation, where N is an integer greater than 2; Each hospital end sends the encrypted first model parameter groups to the central node, and based on the secure multi-party computation protocol, performs weighted calculation on each first model parameter group in the encrypted state to generate the first aggregated parameter group.

4. The distributed privacy protection method based on federated learning according to claim 3, wherein When each hospital end sends the encrypted first model parameter groups to the central node, it also sends the local data scale information. Based on the local data scale information of each hospital end, set the model aggregation weights of each hospital end in a relationship where the weight is proportional to the data scale, and perform weighted calculation on each first model parameter group in the encrypted state with the model aggregation weights.

5. The distributed privacy protection method based on federated learning according to claim 1, characterized in that, Adding differential privacy noise to the first aggregated parameter group at the central node and then sending it back to each hospital end, including: Receive the privacy budget information of each hospital end; Perform differential privacy noise intensity matching based on the privacy budget information to generate each noise intensity information; After adding differential privacy noise to the first aggregated parameter group with each noise intensity information respectively, send it back to each hospital end according to the corresponding relationship between the noise intensity and the hospital end.

6. The distributed privacy protection method based on federated learning according to claim 5, characterized in that, Performing differential privacy noise intensity matching based on the privacy budget information to generate each noise intensity information, including: Based on the Laplace mechanism, calculate the noise addition intensity according to the privacy budget information to generate each Laplace noise scale parameter; Generate each noise intensity information with each Laplace noise scale parameter.

7. The distributed privacy protection method based on federated learning according to claim 1, wherein, After adding differential privacy noise to the first aggregation parameter group at the central node, it is sent back to each hospital end. Each hospital end continues to perform incremental learning after homomorphically encrypting the local patient data based on the sent-back aggregation parameters, and so on, performing multiple rounds of training optimization until the model prediction accuracy meets the preset accuracy constraint, generating a target model parameter group, including: Updating the initial models of each hospital end with the sent-back aggregation parameters to generate respective local updated models; Based on each local updated model, continue to perform incremental learning after homomorphically encrypting the local patient data to generate respective second model parameter groups, and send them to the central node for aggregation and adding differential privacy noise before sending them back to each hospital end; And so on, performing multiple rounds of training optimization until the model accuracy test results of each model after updating the model with the aggregation model parameter group sent back to each hospital end all meet the preset accuracy constraint, stop training, and generate a target model parameter group.

8. The distributed privacy protection method based on federated learning according to claim 7, wherein, Performing multiple rounds of training optimization further includes: During the multiple rounds of training optimization, if in any round of optimization, the model accuracy test result of any hospital end after updating the model with the aggregation model parameter group meets the preset accuracy constraint, mark the any hospital end as an exiting party; Extracting an encrypted summary based on gradient statistics for the exiting party, uploading the encrypted summary to the central node, and then deleting the exiting party from each hospital end; The central node performs global aggregation based on the encrypted summary with reference to the historical aggregation contribution of the exiting party.

9. The distributed privacy protection method based on federated learning according to claim 8, wherein Extracting an encrypted summary based on gradient statistics for the exiting party includes: Extracting the historical model gradient sequence of the exiting party with a historical round limit; Calculating the mean and variance of the gradients based on the historical model gradient sequence to form a statistical summary; Encrypting and uploading the statistical summary to the server through homomorphic encryption to generate the encrypted summary.

10. A distributed privacy protection system based on federated learning, characterized in that, The system is used to implement the distributed privacy protection method based on federated learning according to any one of claims 1-9. The system includes: A first model parameter group generation module, configured to perform homomorphic encryption on the respective patient data of each hospital end, and perform incremental training on the initial model with the encrypted patient data to generate respective first model parameter groups corresponding to each hospital end; A first aggregation parameter group generation module, configured to send the respective first model parameter groups to the central node, and at the central node, use a secure multi-party computation protocol to aggregate the respective first model parameter groups to generate a first aggregation parameter group; A target model parameter group generation module, configured to add differential privacy noise to the first aggregation parameter group at the central node and send it back to each hospital end. Each hospital end continues to perform incremental learning after homomorphically encrypting the local patient data based on the sent-back aggregation parameters, and so on, performing multiple rounds of training optimization until the model prediction accuracy meets the preset accuracy constraint, generating a target model parameter group; The target model generation module is used to add differential privacy noise to the target model parameter group and then send it back to each hospital end for the generation of the target model.

Images