Privacy security fine-grained federal aggregation method, device, equipment and medium
By determining the model hierarchy priority in federated learning, generating random masks and performing central aggregation calculations, combined with the aggregation strategy update, the high cost and insufficient security performance of federated learning framework are solved, and lower cost and higher security are achieved federated aggregation calculation.
Patent Information
- Application Number
- CN202411026878.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-30
AI Technical Summary
The existing federated learning framework has high computing and communication costs in practical applications, and its security performance is not ideal when it is attacked by Byzantine attacks and data poisoning, and cannot effectively solve the data silos in industrial applications.
By determining the priority of different levels of the local model, generating a random mask and adding it to the model parameters, uploading it to the server for central aggregation calculation, and comparing the similarity between the aggregation calculation results and the model parameters, updating the federated aggregation strategy, adjusting the aggregation interval to Improve safety performance.
It reduces communication costs, enhances the security of model parameters, effectively reduces the possibility of Byzantine attacks and data poisoning attacks, and improves the security performance of federated learning.
Smart Images

Figure CN118944935A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information security technology, and in particular to a privacy-safe fine-grained federated aggregation method, device, equipment and medium. Background Art
[0002] Federated learning is a distributed machine learning method in which model training is performed on multiple clients, while model updates are completed by aggregated calculations on a central server. Federated learning allows model training while keeping data decentralized, thus solving some challenges involving privacy and data security issues. The main advantage of federated learning is to protect the data privacy of participants, because the original data does not need to leave the local area, and only the model parameters are transmitted to the server, avoiding the risk of data leakage. Therefore, federated learning has broad application scenarios in scenarios such as healthcare and finance involving sensitive data.
[0003] However, the existing federated learning framework has high computing and communication costs in practical applications. Its security performance is not ideal when subjected to Byzantine attacks and data poisoning attacks, and it cannot effectively solve the problem of data islands in industrial applications.
[0004] There is currently no effective solution to the problems of poor security and vulnerability to attacks in existing federated learning frameworks. Summary of the invention
[0005] The present invention provides a privacy-safe fine-grained federated aggregation method, device, equipment and medium to solve the defects of poor security performance and vulnerability to attacks in the prior art.
[0006] In a first aspect, the present invention provides a privacy-safe fine-grained federated aggregation method, comprising: Determine priorities of different levels of local models, assign corresponding aggregation intervals according to the priorities of different levels, and train the local models according to the federated aggregation strategy; Generate a random mask, and add the random mask to the model parameters of the corresponding level of the local model; Uploading the parameter vector of the model parameter carrying the random mask to the server, and receiving the aggregate calculation result returned by the server; the aggregate calculation result is obtained by the server performing central aggregate calculation on the parameter vector; The similarity between the aggregation calculation result and the parameter vector of the model parameter is compared, and the federation aggregation strategy is updated according to the comparison result.
[0007] According to a privacy-safe fine-grained federated aggregation method provided by the present invention, priorities of different levels of local models are determined, corresponding aggregation intervals are allocated according to the priorities of different levels, and the local models are trained according to the federated aggregation strategy, including: Initializing a priority parameter of the level according to the level of the local model; the priority parameter is used to represent the priority of the corresponding level; Determining an aggregation interval in the federation aggregation strategy according to the initialized priority parameter; Acquire local data, train the local model according to the federated aggregation strategy, and update the local model.
[0008] According to a privacy-safe fine-grained federated aggregation method provided by the present invention, a random mask is generated and added to the model parameters of the corresponding level of the local model, including: Negotiate and determine a secure random number, and generate a key based on the secure random number; Based on the key, generating the random mask by a random mask generation function; Each client adds the random mask to the model parameters of the corresponding layer of the local model, and the sum of the random masks added by all the clients is 0.
[0009] According to a privacy-safe fine-grained federated aggregation method provided by the present invention, a secure random number is determined through negotiation, and a key is generated according to the secure random number, including: Determining a private key between two adjacent clients from a circular group; The client determines the public key according to the private key, and exchanges the public keys with each other; Each of the clients determines the secret key using a private key and a public key of another of the clients.
[0010] According to a privacy-safe fine-grained federated aggregation method provided by the present invention, the server performs central aggregation calculation on the parameter vector, including: The server sums up all parameter vectors with the random mask to obtain a sum result; An average calculation is performed based on the summation result to obtain the aggregate calculation result.
[0011] According to a privacy-safe fine-grained federated aggregation method provided by the present invention, the similarity between the aggregation calculation result and the parameter vector of the model parameter is compared, and the federated aggregation strategy is updated according to the comparison result, including: Determine the Pearson correlation coefficient of the corresponding level of the local model according to the aggregate calculation result; adjusting the aggregation interval in the federated aggregation strategy according to the Pearson correlation coefficient; The local model is updated according to the aggregate calculation result and the model parameters of the corresponding level of the local model.
[0012] According to a privacy-safe fine-grained federated aggregation method provided by the present invention, adjusting the aggregation interval in the federated aggregation strategy according to the Pearson correlation coefficient includes: Compare the Pearson correlation coefficient with a preset threshold interval: If the Pearson correlation coefficient is lower than the minimum value of the threshold interval, the aggregation interval is set to infinity; If the Pearson correlation coefficient exceeds the maximum value of the threshold interval, the aggregate calculation result of this round is discarded; If the Pearson correlation coefficient is within the threshold interval, the aggregation interval is re-determined according to the product of the Pearson correlation coefficient and the aggregation interval.
[0013] In a second aspect, the present invention further provides a privacy-safe fine-grained federated aggregation device, comprising: A determination module, used to determine the priorities of different levels of the local model, allocate corresponding aggregation intervals according to the priorities of the different levels, and train the local model according to the federated aggregation strategy; A generating module, used for generating a random mask and adding the random mask to the model parameters of the corresponding level of the local model; A processing module, used for uploading the parameter vector of the model parameter carrying the random mask to the server, and receiving the aggregate calculation result returned by the server; the aggregate calculation result is obtained by the server performing central aggregate calculation on the parameter vector; An updating module is used to compare the similarity between the aggregation calculation result and the parameter vector of the model parameter, and update the federation aggregation strategy according to the comparison result.
[0014] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the privacy-secure fine-grained federated aggregation method as described in the first aspect above is implemented.
[0015] In a fourth aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the privacy-secure fine-grained federated aggregation method as described in the first aspect above.
[0016] In a fifth aspect, the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the privacy-secure fine-grained federated aggregation method as described in the first aspect above.
[0017] Compared with the prior art, the present invention has the following beneficial effects: The privacy-safe fine-grained federated aggregation method provided by the present invention compares the similarity between the aggregation calculation results and the parameter vectors of the model parameters, and updates the federated aggregation strategy according to the comparison results, and adjusts the aggregation interval and the local model. Through the above operation, there is no need to set up a trusted center, the communication cost is lower, and before the aggregation calculation is performed, a random mask is assigned to the model parameters on the local model. Since the random mask has strong randomness, the model parameters cannot be easily obtained by malicious objects, which can effectively reduce the possibility of Byzantine attacks and data poisoning attacks, and solve the problems of poor security performance and vulnerability to attacks in the existing federated learning framework. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 is a flow chart of the privacy-safe fine-grained federated aggregation method provided by the present invention; Figure 2 It is a structural block diagram of the privacy-safe fine-grained federated aggregation device provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0021] The present invention provides a privacy-safe fine-grained federated aggregation method. Figure 1 is a flow chart of the privacy-safe fine-grained federated aggregation method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step S101, determine the priorities of different levels of the local model, allocate corresponding aggregation intervals according to the priorities of different levels, and train the local model according to the federated aggregation strategy.
[0022] Step S102, generating a random mask, and adding the random mask to the model parameters of the corresponding level of the local model.
[0023] Step S103, uploading the parameter vector of the model parameter carrying the random mask to the server, and receiving the aggregate calculation result returned by the server. The aggregate calculation result is obtained by the server performing central aggregate calculation on the parameter vector.
[0024] Step S104, comparing the similarity between the aggregation calculation result and the parameter vector of the model parameter, and updating the federation aggregation strategy according to the comparison result.
[0025] In this method, for the local model, there are certain differences in the hierarchical structure. Therefore, before performing aggregation training, the client needs to first determine the corresponding priority according to the different levels, and then assign the corresponding aggregation interval to each level according to the priority. Then train the local model according to the federal aggregation strategy, which includes the assigned aggregation interval. Then, generate a random mask and add the random mask to the model parameters of the corresponding level of the local model, so as to protect the privacy information of the local model and improve the security performance of the local model. Each client participant then uploads the parameter vector of the model parameters with the random mask to the server, which performs central aggregation calculation and returns the aggregation calculation results to the client. Finally, the client compares the similarity of the aggregation calculation results and the parameter vector of the model parameters, and updates the federal aggregation strategy according to the comparison results, and adjusts the aggregation interval and the local model. Through the above operations, there is no need to set up a trusted center, and the communication cost is lower. Before performing aggregate calculations, a random mask is assigned to the model parameters on the local model. Since the random mask has strong randomness, the model parameters cannot be easily obtained by malicious objects, which can effectively reduce the possibility of Byzantine attacks and data poisoning attacks, and solve the problems of poor security performance and vulnerability to attacks in the existing federated learning framework.
[0026] In some of the embodiments, step S101 determines the priorities of different levels of the local model, allocates corresponding aggregation intervals according to the priorities of different levels, and trains the local model according to the federal aggregation strategy, including: initializing the priority parameters of the level according to the level of the local model; the priority parameters are used to characterize the priority of the corresponding level; determining the aggregation interval in the federal aggregation strategy according to the initialized priority parameters; obtaining local data, training the local model according to the federal aggregation strategy, and updating the local model.
[0027] Exemplarily, the client first initializes the corresponding priority parameters according to the different levels in the local model , and according to the priority parameter Set the aggregation interval in the federation aggregation policy. The specific formula is as follows:
[0028] in, represents the aggregation interval, Indicates level priority parameters, The closer it is to 1, the higher the priority, and vice versa; It represents the minimum aggregation time interval in the federated aggregation learning process. Then, local data is obtained, and the local model is trained and updated through the local data. The specific formula is as follows:
[0029] in, Indicates i local models held by participants, represents the learning rate during local model training, For sample representation, is the sample data, is the sample label, is the model prediction function, and the output is the regularized secret vector; The cross entropy loss function calculates the cross entropy between the predicted label and the true label for backpropagation to adjust the local model.
[0030] In some of the embodiments, step S102, generating a random mask and adding the random mask to the model parameters of the corresponding level of the local model, includes: negotiating to determine a secure random number, and generating a key based on the secure random number; based on the key, generating a random mask through a random mask generation function; each client adds the random mask to the model parameters of the corresponding level of the local model, and the sum of the random masks added by all clients is 0.
[0031] In this embodiment, all clients jointly negotiate a secure random number and generate a key, and then use a random mask generation function to generate a share of 0 as a random mask. Each client then adds the generated random mask to the model parameters of the corresponding level of the local model, and ensures that the sum of the random masks added by all clients is 0, so as not to affect the correctness of the final aggregation result. The specific formula is as follows:
[0032] in, represents the parameter vector of the model parameters with random masks, Indicates The local model held by the participants parameter vector of layer model parameters, is the random mask generation function, Indicates the participants With the participants The negotiated key is or ; n Indicates the number of federated aggregate learning participants.
[0033] Specifically, the secure random number is determined through negotiation, and a key is generated based on the secure random number, including: two adjacent clients determine a private key from a cyclic group; the client determines a public key based on the private key and exchanges the public key with each other; each client determines the key using the private key and the public key of the other client.
[0034] In this embodiment, the client only negotiates with two adjacent clients to generate a key using a random mask through the Diffile-Hellman key agreement protocol. and participants From the cyclic group Randomly select your own private key, which are and ,in or . Then, each party calculates its own public key and exchanges the public key with each other. The calculation formula is as follows:
[0035]
[0036] in, Represents the client party The public key of Represents the client party The public key of Cyclic group The generators in is the order of the prime cyclic group. In this embodiment, is a large prime number, set to 2 127 -1. Participants Utilize the participants The public key and your own private key are used to calculate the key. The calculation formula is as follows:
[0037] in, Indicates the participants With the participants The generated key is determined through negotiation. represents the order of the prime cyclic group, Represents the client party The public key of Indicates the participants The private key of is the order of the cyclic group. Similarly, the participants also use the participants The public key and your own private key are used to calculate the key. The calculation formula is as follows:
[0038] in, Indicates the participants With the participants The generated key is determined through negotiation. represents the order of the prime cyclic group, Represents the client party The public key of Indicates the participants The private key of is the order of the cyclic group. From this, we can observe that:
[0039] In some of the embodiments, the server performs central aggregation calculation on the parameter vectors, including: the server sums all parameter vectors with random masks to obtain a sum result; and performs mean calculation based on the sum result to obtain an aggregate calculation result.
[0040] Exemplarily, after receiving the parameter vector uploaded by the client, the server sums all parameter vectors with random masks. The specific formula is as follows:
[0041] in, represents the summation result, represents the parameter vector of the model parameters with random masks, Indicates The local model held by the participants parameter vector of layer model parameters, Represents the set of participants. Then, the summation results are averaged to obtain the final aggregate calculation result of the local model. The specific formula is as follows:
[0042] in, Represents the result of aggregate calculation. represents the summation result, represents the set of participants, n Indicates the number of federated aggregate learning participants.
[0043] In some of the embodiments, step S104 uses federated distillation calculation to compare the similarity between the aggregation calculation results and the parameter vectors of the model parameters, and updates the federated aggregation strategy based on the comparison results, including: determining the Pearson correlation coefficient of the corresponding level of the local model based on the aggregation calculation results; adjusting the aggregation interval in the federated aggregation strategy based on the Pearson correlation coefficient; and updating the local model based on the aggregation calculation results and the model parameters of the corresponding level of the local model.
[0044] Pearson correlation coefficient It is a statistical method to measure the strength of the linear relationship between two variables. It is usually used to measure the correlation between two continuous variables, and its value range is between -1 and 1. Specifically, the Pearson correlation coefficient can measure the degree of linear relationship between two variables. It means that the two vectors are completely positively correlated, showing a completely linear positive relationship; Indicates a perfect negative correlation, that is, the two variables show a complete linear negative relationship; It means that the two vectors are linearly independent.
[0045] Exemplarily, the client determines the Pearson correlation coefficient of the corresponding level of the local model based on the aggregate calculation result of the local model. The specific formula is as follows:
[0046] in, r represents the Pearson correlation coefficient, Indicates i The local model of the client l Layer parameters, Indicates t The global model parameters obtained by the round aggregation process are l Layer parameters, t Indicates the first t Wheel polymerization process.
[0047] Specifically, the aggregation interval in the federal aggregation strategy is adjusted according to the Pearson correlation coefficient, including: comparing the Pearson correlation coefficient and the preset threshold interval: if the Pearson correlation coefficient is lower than the minimum value of the threshold interval, the aggregation interval is set to infinity; if the Pearson correlation coefficient exceeds the maximum value of the threshold interval, the aggregation calculation result of this round is discarded; if the Pearson correlation coefficient is in the threshold interval, the aggregation interval is redetermined according to the product of the Pearson correlation coefficient and the aggregation interval.
[0048] For example, if the Pearson correlation coefficient If the value is lower than the minimum value of the threshold interval, it means that the aggregation result is close to the local model and no aggregation calculation is required. Therefore, let the aggregation interval ; If the Pearson correlation coefficient If the value is higher than the maximum value of the threshold interval, it means that the aggregation result deviates too much from the local model, and there may be malicious nodes and Byzantine nodes in the system. Therefore, the aggregation calculation result of this round is discarded; if the Pearson correlation coefficient is Within the threshold range, according to the Pearson correlation coefficient To adjust the aggregation interval, the specific formula is as follows:
[0049] in, represents the aggregation interval, r represents the Pearson correlation coefficient.
[0050] The client updates the local model based on the parameter vector of the local model and the aggregate calculation results. The specific formula is as follows:
[0051] in, represents the local model parameters in the t+1th round of the federated learning process, t represents the t-th round aggregation process in the federated learning process, Represents the weight parameter of the aggregate calculation result in the local model update process, Represents the result of an aggregate calculation.
[0052] The present invention also provides a privacy-security fine-grained federal aggregation device. The privacy-security fine-grained federal aggregation device provided by the present invention is described below. The privacy-security fine-grained federal aggregation device described below and the privacy-security fine-grained federal aggregation method described above can be referenced to each other. Figure 2 is a structural block diagram of the privacy-safe fine-grained federated aggregation device provided by the present invention, such as Figure 2 As shown, the device comprises: A determination module 201 is used to determine the priorities of different levels of the local model, allocate corresponding aggregation intervals according to the priorities of different levels, and train the local model according to the federated aggregation strategy; A generating module 202, used to generate a random mask and add the random mask to the model parameters of the corresponding level of the local model; The processing module 203 is used to upload the parameter vector of the model parameter carrying the random mask to the server, and receive the aggregate calculation result returned by the server; the aggregate calculation result is obtained by the server performing central aggregate calculation on the parameter vector; The updating module 204 is used to compare the similarity between the aggregation calculation result and the parameter vector of the model parameter, and update the federation aggregation strategy according to the comparison result.
[0053] When the device is in use, for the local model, there are certain differences in the hierarchical structure. Therefore, before performing aggregation training, the determination module 201 needs to first determine the corresponding priority according to the different hierarchies, and then assign the corresponding aggregation interval to each hierarchical level according to the priority. Then, the local model is trained according to the federal aggregation strategy, in which the allocated aggregation interval is included. Then, the generation module 202 generates a random mask and adds the random mask to the model parameters of the corresponding level of the local model, thereby protecting the privacy information of the local model and improving the security performance of the local model. Each processing module 203 then uploads the parameter vector of the model parameters with the random mask to the server, and the server performs central aggregation calculation and returns the aggregation calculation results to the client. Finally, the update module 204 performs a similarity comparison between the aggregation calculation results and the parameter vector of the model parameters, and updates the federal aggregation strategy according to the comparison results, and adjusts the aggregation interval and the local model. Through the above process, there is no need to set up a trusted center, the communication cost is lower, and before performing aggregate calculations, a random mask is assigned to the model parameters on the local model. Since the random mask has strong randomness, the model parameters cannot be easily obtained by malicious objects, which can effectively reduce the possibility of Byzantine attacks and data poisoning attacks, and solve the problems of poor security performance and vulnerability to attacks in the existing federated learning framework.
[0054] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302 and the memory 303 communicate with each other through the communication bus 304. The processor 301 may call the logic instructions in the memory 303 to execute the privacy-safe fine-grained federated aggregation method, which includes: Determine the priorities of different levels of the local model, assign corresponding aggregation intervals according to the priorities of different levels, and train the local model according to the federated aggregation strategy; Generate a random mask and add the random mask to the model parameters of the corresponding layer of the local model; Upload the parameter vector of the model parameters carrying the random mask to the server, and receive the aggregate calculation result returned by the server; the aggregate calculation result is obtained by the server performing central aggregate calculation on the parameter vector; Compare the similarity between the aggregation calculation results and the parameter vector of the model parameters, and update the federation aggregation strategy based on the comparison results.
[0055] In addition, the logic instructions in the above-mentioned memory 303 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0056] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the privacy-safe fine-grained federated aggregation method provided by the above methods, and the method includes: Determine the priorities of different levels of the local model, assign corresponding aggregation intervals according to the priorities of different levels, and train the local model according to the federated aggregation strategy; Generate a random mask and add the random mask to the model parameters of the corresponding layer of the local model; Upload the parameter vector of the model parameters carrying the random mask to the server, and receive the aggregate calculation result returned by the server; the aggregate calculation result is obtained by the server performing central aggregate calculation on the parameter vector; Compare the similarity between the aggregation calculation results and the parameter vector of the model parameters, and update the federation aggregation strategy based on the comparison results.
[0057] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the privacy-safe fine-grained federated aggregation method provided by the above methods, the method comprising: Determine the priorities of different levels of the local model, assign corresponding aggregation intervals according to the priorities of different levels, and train the local model according to the federated aggregation strategy; Generate a random mask and add the random mask to the model parameters of the corresponding layer of the local model; Upload the parameter vector of the model parameters carrying the random mask to the server, and receive the aggregate calculation result returned by the server; the aggregate calculation result is obtained by the server performing central aggregate calculation on the parameter vector; Compare the similarity between the aggregation calculation results and the parameter vector of the model parameters, and update the federation aggregation strategy based on the comparison results.
[0058] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0059] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A privacy-safe fine-grained federated aggregation method, characterized in that: include: Determine priorities of different levels of local models, assign corresponding aggregation intervals according to the priorities of different levels, and train the local models according to the federated aggregation strategy; Generate a random mask, and add the random mask to the model parameters of the corresponding level of the local model; Uploading the parameter vector of the model parameter carrying the random mask to the server, and receiving the aggregate calculation result returned by the server; the aggregate calculation result is obtained by the server performing central aggregate calculation on the parameter vector; The similarity between the aggregation calculation result and the parameter vector of the model parameter is compared, and the federation aggregation strategy is updated according to the comparison result.
2. The privacy-safe fine-grained federated aggregation method according to claim 1, characterized in that: Determine the priorities of different levels of the local model, assign corresponding aggregation intervals according to the priorities of the different levels, and train the local model according to the federated aggregation strategy, including: Initializing a priority parameter of the level according to the level of the local model; the priority parameter is used to represent the priority of the corresponding level; Determining an aggregation interval in the federation aggregation strategy according to the initialized priority parameter; Acquire local data, train the local model according to the federated aggregation strategy, and update the local model.
3. The privacy-safe fine-grained federated aggregation method according to claim 1, characterized in that: Generating a random mask and adding the random mask to the model parameters of the corresponding level of the local model, including: Negotiate and determine a secure random number, and generate a key based on the secure random number; Based on the key, generating the random mask by a random mask generation function; Each client adds the random mask to the model parameters of the corresponding layer of the local model, and the sum of the random masks added by all the clients is 0.
4. The privacy-safe fine-grained federated aggregation method according to claim 3, characterized in that: Negotiating and determining a secure random number, and generating a key based on the secure random number, including: Determining a private key between two adjacent clients from a circular group; The client determines a public key according to the private key, and exchanges the public keys with each other; Each of the clients determines the secret key using a private key and a public key of another of the clients.
5. The privacy-safe fine-grained federated aggregation method according to claim 1, characterized in that: The server performs central aggregation calculation on the parameter vector, including: The server sums up all parameter vectors with the random mask to obtain a sum result; An average calculation is performed based on the summation result to obtain the aggregate calculation result.
6. The privacy-safe fine-grained federated aggregation method according to claim 1, characterized in that: Comparing the similarity between the aggregation calculation result and the parameter vector of the model parameter, and updating the federation aggregation strategy according to the comparison result, including: Determine the Pearson correlation coefficient of the corresponding level of the local model according to the aggregate calculation result; adjusting the aggregation interval in the federated aggregation strategy according to the Pearson correlation coefficient; The local model is updated according to the aggregate calculation result and the model parameters of the corresponding level of the local model.
7. The privacy-safe fine-grained federated aggregation method according to claim 6, characterized in that: Adjusting the aggregation interval in the federation aggregation strategy according to the Pearson correlation coefficient includes: Compare the Pearson correlation coefficient with a preset threshold interval: If the Pearson correlation coefficient is lower than the minimum value of the threshold interval, the aggregation interval is set to infinity; If the Pearson correlation coefficient exceeds the maximum value of the threshold interval, the aggregate calculation result of this round is discarded; If the Pearson correlation coefficient is within the threshold interval, the aggregation interval is re-determined according to the product of the Pearson correlation coefficient and the aggregation interval.
8. A privacy-safe fine-grained federated aggregation device, characterized in that: include: A determination module, used to determine the priorities of different levels of the local model, allocate corresponding aggregation intervals according to the priorities of the different levels, and train the local model according to the federated aggregation strategy; A generating module, used for generating a random mask and adding the random mask to the model parameters of the corresponding level of the local model; A processing module, used to upload the parameter vector of the model parameters carrying the random mask to a server, and receive the aggregate calculation result returned by the server; The aggregate calculation result is obtained by the server performing a central aggregate calculation on the parameter vector; An updating module is used to compare the similarity between the aggregation calculation result and the parameter vector of the model parameter, and update the federation aggregation strategy according to the comparison result.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the privacy-safe fine-grained federated aggregation method as described in any one of claims 1 to 7 is implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the privacy-safe fine-grained federated aggregation method as described in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Federal learning local model parameter aggregation method
CN115021905A
Federal learning privacy protection method and system, medium, equipment and terminal
CN115883076A
Dynamic model training method, system and equipment based on annular knowledge distillation and meta-federal learning
CN117010534A
Adaptive federated learning method and system based on heterogeneous edge-end equipment
CN118194976A
Single-party real-time prediction algorithm based on federated learning
WO2024060409A1
