Efficient federal learning method and system

Abstract

The invention discloses an efficient federal learning method and system. According to the method, firstly, through model privacy measurement and sensitivity analysis, privacy risk assessment and grading are carried out on model parameters, homomorphic encryption is only carried out on high-sensitivity parameters, light disturbance is applied to medium-sensitivity parameters, and plaintext transmission is kept on low-sensitivity parameters, so that a'key encryption and non-key plaintext 'hybrid protection mechanism is realized. A server generates a global encryption mask according to sensitivity information uploaded by a client, and performs security aggregation in a ciphertext domain and a plaintext domain respectively. And the client carries out decryption and adaptive updating after receiving the global model, dynamically adjusts an encryption strategy, and realizes continuous balance between privacy protection strength and model precision. According to the invention, the communication and calculation overhead is effectively reduced, the security and training efficiency of the federal learning system are improved, and the method is suitable for privacy collaborative learning of multiple scenes such as medical treatment, industry and Internet of Things.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and privacy computing technology, and in particular to an efficient federated learning method and system. Background Technology

[0002] Federated learning is a novel distributed machine learning framework designed to enable collaborative training of data from multiple parties while protecting user privacy. Each client retains its original data locally, uploading only the model parameters to a central server for aggregation and updates. This avoids centralized storage and transmission of original data, meeting privacy compliance requirements in fields such as finance, healthcare, and the Internet of Things.

[0003] However, although federated learning frameworks avoid the leakage of raw data, the exchange of model gradients or parameters can still lead to the leakage of private information. Studies have shown that attackers can reconstruct the original training samples from plaintext parameters through techniques such as member inference attacks or gradient inversion attacks, seriously threatening data privacy and security.

[0004] Existing privacy protection mechanisms mainly include three categories: differential privacy, secure aggregation, and homomorphic encryption. Differential privacy methods achieve privacy by adding noise to model parameters, but this leads to a decrease in model accuracy. Secure aggregation requires clients to negotiate masks pairwise, resulting in high communication overhead and extremely high network requirements, making it unsuitable for unstable network environments. Homomorphic encryption theoretically achieves "computable and invisible" security, but its full-parameter encryption incurs extremely high computational and communication costs, severely impacting system performance.

[0005] Existing technologies significantly improve communication efficiency through homomorphic encryption of some parameters. However, due to the heterogeneity of model and client data, the encryption ratio needs to be manually set for each training session to ensure the security of federated learning, making dynamic adjustment of the encryption ratio impossible. The encryption ratio is a critical factor; if it is set too high, the communication efficiency of federated learning will be very low; if it is set too low, there is a risk of privacy leakage. Therefore, how to improve communication efficiency while ensuring a high level of privacy protection has become a key challenge in promoting the practical application of federated learning. Summary of the Invention

[0006] The purpose of this invention is to achieve dynamic adjustment of the encryption ratio of some parameters while ensuring a high level of privacy protection, and to propose an efficient federated learning method and system.

[0007] This invention designs a model privacy metric function to perform sensitivity analysis and dynamic classification of model parameters. Homomorphic encryption is applied only to high-sensitivity parameters, perturbations are added to medium-sensitivity parameters, and low-sensitivity parameters are transmitted in plaintext. This enables dynamic adjustment of the encryption ratio of some parameters, thereby significantly reducing communication and computational overhead.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: an efficient federated learning method, comprising the following steps:

[0009] Step 1: Calculation of Comprehensive Privacy Risk: The server first simulates the behavior of potential attackers, performing privacy attack simulations on the global model, such as member inference or gradient inversion, to preliminarily assess the model's privacy leakage risk. Then, the server uses sample data to calculate the degree of influence of each parameter on the model's output, and calculates the comprehensive privacy risk through the model's privacy metric function.

[0010] Step 2: Client-Side Sensitive Parameter Analysis: Each client calculates the sensitivity of its model parameters based on its local training data. Then, all parameter sensitivity information is normalized. Finally, the client uploads the parameter sensitivity information to the server.

[0011] Step 3: Encryption Mask Generation and Dynamic Threshold Adjustment: The server performs security aggregation on the parameter sensitivity information uploaded by each client and conducts a unified analysis of the overall distribution of all parameters. Based on the analysis results, the server dynamically determines the global sensitivity threshold and generates a global encryption mask. This mask is used to mark which parameters require strong encryption, which only require light protection, and which can be transmitted in plaintext. Finally, the server distributes the encryption mask to each client.

[0012] Step 4: Hierarchical Encryption of Some Parameters: After receiving the encryption mask from the server, the client classifies the model parameters according to the mask. Specifically, the model parameters are divided into a high-sensitivity encrypted area, a medium-sensitivity mixed area, and a low-sensitivity plaintext area. Homomorphic encryption is performed on the high-sensitivity parameters to ensure that they remain in ciphertext during transmission and aggregation; slight perturbations or random noise are applied to the medium-sensitivity parameters; and the low-sensitivity parameters are kept in plaintext form to avoid unnecessary encryption calculations. Finally, the client uploads all parameters to the server.

[0013] Step 5, Secure Aggregation: After receiving the parameters uploaded by each client, the server performs secure aggregation in both the ciphertext and plaintext domains. Finally, the server distributes the aggregated global model parameters.

[0014] Step 6: Model Decryption and Comprehensive Privacy Risk Update: After receiving the aggregated global model parameters from the server, the client uses its local private key to decrypt the encrypted parts of the parameters and recombine them with the plaintext parameters to form the complete model parameters. Subsequently, the local sensitivity is updated to dynamically adjust the global sensitivity threshold.

[0015] This invention also proposes an efficient federated learning system, wherein the system applies an efficient federated learning method as described above, and the system includes: a comprehensive privacy risk calculation module, a client sensitive parameter analysis module, an encryption mask generation and threshold dynamic adjustment module, a partial parameter hierarchical encryption module, a security aggregation module, and a model decryption and comprehensive privacy risk update module.

[0016] The modules deployed on the server include: a comprehensive privacy risk calculation module, an encryption mask generation and threshold dynamic adjustment module, a security aggregation module, and a model decryption and comprehensive privacy risk update module;

[0017] The modules deployed on the client include: a client-side sensitive parameter analysis module, a partial parameter hierarchical encryption module, and a model decryption and comprehensive privacy risk update module.

[0018] The comprehensive privacy risk calculation module is used to: calculate the comprehensive privacy risk using the model privacy metric function when federated learning begins;

[0019] The client-side sensitive parameter analysis module is used to: after the server has completed the privacy risk calculation, each client performs sensitivity calculations on the model parameters based on its local training data;

[0020] The encryption mask generation and threshold dynamic adjustment module is used for: the server to generate an encryption mask and dynamically determine the global sensitivity threshold;

[0021] The partial parameter hierarchical encryption module is used to: divide model parameters into high-sensitivity encryption areas, medium-sensitivity mixed areas, and low-sensitivity plaintext areas, and encrypt them hierarchically.

[0022] The secure aggregation module is used by the server to perform secure aggregation in both the ciphertext and plaintext domains.

[0023] The model decryption and comprehensive privacy risk update module is used for: client-side model decryption for training and updating of comprehensive privacy risks.

[0024] Preferably, in step 1, a model privacy measurement method specifically includes the following steps:

[0025] Step 1.1: Calculation of Empirical Privacy Guarantee. First, the server simulates the behavior of a potential attacker, attacking the existing model. The false positive rate (FPR) and false negative rate (FNR) are calculated based on the attack success rate. Then, the empirical privacy guarantee is calculated using these two metrics.

[0026]

[0027] in, n is the number of clients, and privacy is guaranteed. It is used to measure the strength of privacy guarantees, generally speaking A smaller value indicates a higher level of privacy protection, while a larger value indicates a lower level of privacy protection.

[0028] Step 1.2, Model Complexity Calculation. The specific calculation of the model complexity MC is as follows:

[0029] MC = α·P + β·FLOPs + γ·Com

[0030] Where P is the number of parameters in the model; FLOPs is the computational complexity of the model, i.e., the number of floating-point operations performed; Com is the communication complexity of the model, i.e., the amount of data transmitted between the client and the server during federated learning. α, β, and γ are weighting coefficients used to balance the proportions of the number of parameters, computational complexity, and communication complexity in the total complexity. The default values ​​are α = 1, β = 1, and γ = 1.

[0031] Step 1.3, Attacker Capability Calculation. The attacker's capability in federated learning can be uniformly quantified as attack gain G, expressed as:

[0032] G=(1-μ)·MIA_Acc+μ·Rec_Err

[0033] Where MIA_Acc represents the F1 score of a member inference attack, measuring the attacker's ability to determine whether a record belongs to the training set; Re_Err represents the minimum inverse loss of a data reconstruction attack, i.e., the reconstruction error when the attacker restores the gradient / model update to the original input, the smaller the error, the more successful the attack; μ∈[0,1] represents the attacker's preference weights, μ→1 emphasizes reconstruction, and μ→0 emphasizes inference. When G→1, it means that the attacker has both high inference accuracy and low reconstruction error rate; G→0 means that the attack capability is effectively weakened.

[0034] Step 1.4, Calculation of Comprehensive Privacy Risk. Calculate the comprehensive privacy risk ρ using the following formula:

[0035]

[0036] Among them, w1, w2, and w3 are the weights of each part, used to control the actual range of ρ. The default values ​​are w1 = 0.6, w2 = 0.2, and w3 = 0.2.

[0037] Preferably, for the calculation of empirical privacy guarantees in step 1.1, there is a more efficient calculation method, as follows:

[0038] Step 1.1.1: Calculation of local sensitivity of parameters based on gradient energy. The server first simulates various inference attacks using a small number of proxy samples, including membership inference attacks and gradient inversion attacks, and performs one forward and backward propagation to obtain the gradient matrix of the model parameters.

[0039] To avoid the high costs associated with repeated attacks, this step uses the gradient squared energy method to calculate local sensitivity:

[0040]

[0041] Where s i This represents the local sensitivity of the i-th parameter. This metric measures the direct impact of a parameter on changes in the model's loss function and can be calculated within a single propagation. i This represents the gradient of the i-th parameter.

[0042] Step 1.1.2: Gradient Distribution Skewness Correction and Comprehensive Privacy Risk Modeling. Since parameter sensitivity in neural networks typically exhibits a long-tailed distribution, directly using the mean or variance as the screening criterion can lead to "local over-encryption." Therefore, this invention introduces a gradient distribution skewness correction mechanism.

[0043] The server statistical local sensitivity sample set S = {s1, s2, ..., s} is S = {s1, s2, ..., s}. θ} and calculate skewness κ and kurtosis η:

[0044]

[0045] When κ > 0 and η > 3, it indicates the existence of a highly sensitive parameter group. E[·] represents the mathematical expectation operator. It is the average of the local sensitivity sample set S, where σ represents the standard deviation, i.e., the sensitivity s. i The degree of fluctuation.

[0046] Step 1.1.3, Empirical Privacy Guarantee Calculation. The empirical privacy guarantee can be calculated using the following formula based on skewness κ and kurtosis η.

[0047]

[0048] Here, a and b are adjustment factors used to control the degree of influence of κ and η on the value of e. The default values ​​are a=1 and b=1.

[0049] Preferably, in step 2, a sensitive parameter analysis method is employed. After completing the global privacy measurement, each client, based on its own data distribution, performs local sensitive parameter filtering. Each client is required to use the sensitive parameter analysis module for analysis. Specific steps include:

[0050] Step 2.1: Calculation of the second derivative tensor. Calculate the second derivative tensor J. m (y k The specific calculation formula is as follows:

[0051]

[0052] Among them, w m Let m represent the m-th parameter, m∈{0,1,2,…,θ}, where θ is the total number of parameters. X and y represent the input matrix and label vector of the given model W and K samples, respectively, and l(·) represents the loss function.

[0053] Step 2.2, Parameter Sensitivity Calculation: To characterize the sensitivity τ of the gradient of the m-th parameter to the true output. m Take J m (y k Its average absolute value over the sample dimension is calculated as follows:

[0054]

[0055] After each client i calculates the sensitivity of each parameter in turn, a sensitivity vector is obtained. After the calculation is complete, each client will send the sensitivity vector. Uploaded to the server for subsequent global aggregation of encryption masks.

[0056] Preferably, in step 3, the encryption mask generation and threshold dynamic adjustment module is deployed on the server. This module starts running after receiving the sensitivity vectors from all clients, and its specific steps include:

[0057] Step 3.1: Client-side Sensitivity Aggregation and Consistency Correction. The server receives the sensitivity vector sets uploaded by all clients. Aggregation calculations can be performed within the ciphertext domain using the properties of homomorphic addition.

[0058]

[0059] Where w i The client-side weights can be adaptively set based on the data size, reputation, or training epochs. The default is... n is the number of clients.

[0060] The server decrypts to obtain the global sensitivity vector. And achieve uniformity of sensitivity distribution among multiple clients through Z-score standardization:

[0061]

[0062] Where μ g For Mglobal The average value, σ g For M global The variance of ∈ is a minimum value to prevent the denominator from being 0. The default ∈ = 0.1. This correction process eliminates the bias caused by data heterogeneity between clients, making the mask generation based on a uniform statistical scale.

[0063] Step 3.2: Determining the Global Dynamic Threshold Based on Skewed Distribution. In traditional federated learning encryption, sensitive parameters are often selected using fixed proportions or manually set, lacking data-driven approaches. Based on the server's perception of the global sensitivity distribution M... norm Calculate skewness κ g and kurtosis η g Automatically determine the dynamic threshold Ω max and Ω min :

[0064]

[0065] Where λ1 and λ2 are balance coefficients used to adjust the encryption strength and plaintext ratio, with the default values ​​of λ1 = 0.5 and λ2 = 0.5, and skewness κ. g and kurtosis η g The calculation formula can be found in step 1; For M norm The average value.

[0066] Step 3.3, Encryption Mask Generation and Sparsity Optimization. The server generates an encryption mask based on the filtering results. The definition is as follows:

[0067]

[0068] Where, p m ∈(0,1) is a randomization flag used for minor disturbance protection of medium-sensitivity parameters, thereby improving system robustness.

[0069] Preferably, in step 4, a partial parameter hierarchical encryption module needs to be deployed on each client. After deployment, according to... Hierarchical encryption is implemented by encrypting and transmitting local model parameters. The specific steps include:

[0070] Step 4.1: Parameter classification and encryption strategy matching. The client obtains the encryption mask. Then, the local parameters are divided into three sets:

[0071]

[0072] Among them, the high-sensitivity encryption zone Implement strong homomorphic encryption protection to prevent member inference and gradient inversion attacks; Medium-sensitivity hybrid zone Semi-encryption protection is achieved using light perturbations (such as multiplicative noise and random permutations); low-sensitivity plaintext area. Maintain plaintext transmission to maximize communication and computation efficiency.

[0073] Step 4.2, Homomorphic Encryption Execution and Key Management Mechanism. The client uses the public key pk to perform encryption operations and generate a ciphertext parameter vector:

[0074]

[0075] Among them, Enc pk (w m ) represents the parameters encrypted using the public key pk, Perturb(w m The parameters are those with random noise added. Meanwhile, the client locally retains the private key `sk` for subsequent decryption. After processing, all parameters are combined into a single set. And upload it to the server.

[0076] Preferably, in step 5, the security aggregation module needs to be deployed on the server. The specific steps are as follows:

[0077] Step 5.1, Ciphertext Field Aggregation. This involves a set of parameters for the highly sensitive encrypted areas uploaded by each client. The server leverages the homomorphic addition property to perform weighted aggregation operations without decryption:

[0078]

[0079] Among them, v i Weighting coefficients for the client, by default n is the number of clients; W HE This represents the set of parameters for the aggregated, highly sensitive encrypted region; the aggregation process is completed in the ciphertext domain, ensuring that the server cannot access the plaintext model of any individual client throughout the entire computation process.

[0080] Step 5.2, Plaintext Region Aggregation. This involves a set of parameters identified as medium-sensitivity mixed regions and low-sensitivity plaintext regions. The server performs lightweight aggregation calculations within the plaintext domain:

[0081]

[0082] Among them, W plain This represents the set of parameters for the aggregated medium-sensitivity mixed region and the low-sensitivity plaintext region.

[0083] Finally, W HE and W plain Combined into a global model W agg It is then distributed to each client, specifically:

[0084] W agg =W HE ∪W plain

[0085] Preferably, in step 6, the model decryption and comprehensive privacy risk update modules need to be deployed on the client and server respectively. The client performs decryption locally, and the server performs comprehensive privacy risk update in the cloud, so that the next round of training can better adapt to the local data distribution and the global objective. The specific steps are as follows:

[0086] Step 6.1, Decryption and Reconstruction. The client receives the global model W from the server. agg And use its own private key sk to decrypt the encrypted part:

[0087] W dec =Dec sk (W HE )

[0088] W dec This represents the set of parameters for the highly sensitive encrypted area after decryption. Then, the decryption result is recombined with the plaintext portion.

[0089] W global =W dec ∪W piain

[0090] Form a complete trainable global model W global .

[0091] Step 6.2, Adaptive Feedback and Sensitivity Update. To maintain the continuity of dynamic privacy protection, the client re-evaluates the parameter sensitivity distribution after each round of global model update, forming a feedback loop. Local sensitivity is recalculated based on the latest gradient distribution:

[0092]

[0093] Where ρ represents the overall privacy risk, as given in step 1.

[0094] Finally, the client uploads the updated local sensitivity and overall privacy risk to the server for use in the next round of encryption mask generation.

[0095] Step 6.3, Convergence Monitoring. The server monitors whether the training has converged using the following criteria:

[0096]

[0097] Where ε is a local minimum value, with the default ε = 0.1. For the global model trained in round t, when the above conditions are met or If convergence fails, the system considers the global model to have reached its optimal equilibrium state, stops training, and outputs the final global model. Otherwise, it proceeds to the next iteration.

[0098] Compared with the prior art, the present invention has the following beneficial effects:

[0099] (1) By determining the global dynamic threshold, only highly sensitive parameters are encrypted, significantly reducing communication and computing overhead;

[0100] (2) A dynamic balance between privacy protection and efficiency can be achieved through parameter grading method;

[0101] (3) A hybrid mechanism of homomorphic encryption and plaintext transmission is adopted to enhance privacy protection while maintaining model accuracy;

[0102] (4) The system has good scalability and is suitable for large-scale neural networks and cross-domain federated learning scenarios. Attached Figure Description

[0103] Figure 1 A flowchart illustrating an efficient federated learning method.

[0104] Figure 2 A schematic diagram of a model for an efficient federated learning method and system.

[0105] Figure 3 A graph comparing the communication volume of an efficient federated learning method with other methods.

[0106] Figure 4 System architecture diagram of an efficient federated learning system Detailed Implementation

[0107] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0108] It should be noted that, in order to avoid unnecessary details from obscuring the important parts of the present invention, the accompanying drawings only show structures or processing steps closely related to the present invention, while omitting other details that are not closely related to the present invention.

[0109] Example 1, as Figure 1 The flowchart of the defense method shown is provided below. To facilitate understanding of the technical solutions of the embodiments of this application, a simple execution flow of the embodiments of this application is first given:

[0110] Step 1, Model Privacy Measurement: The server simulates attacker behavior, performs inference attacks on the client model, calculates the risk of model privacy leakage, and determines the model privacy risk value by combining the number of model parameters, computational complexity, and communication complexity.

[0111] Step 2, Sensitive Parameter Analysis: Based on the comprehensive privacy risks provided by the server, the client calculates the sensitivity of local model parameters and evaluates the degree of influence of parameters on the output by using the second derivative of the gradient.

[0112] Step 3: Encryption Mask Generation: The server aggregates the sensitivity vectors uploaded by each client, selects a set of highly sensitive parameters based on parameter sensitivity, and generates an encryption mask.

[0113] Step 4: Partial parameter homomorphic encryption: The client performs homomorphic encryption on the highly sensitive parameters marked "1" according to the encryption mask, while keeping the other parameters in plaintext and uploading them to the server;

[0114] Step 5, Secure Aggregation: The server performs aggregation operations in the ciphertext and plaintext fields respectively, and completes the weighted summation without decryption by leveraging homomorphic encryption.

[0115] Step 6, Model Update: The server sends the aggregated model parameters to the client. The client decrypts the encrypted part and reconstructs the complete global model, then enters the next round of training.

[0116] Figure 2 This presents a possible scenario: a general federated learning system consists of three steps: (1) In the model privacy measurement phase, the server simulates an attacker to perform inference attacks on each client and calculates the model privacy leakage risk based on the attack results and the model itself; (2) In the encrypted parameter calculation phase, the client uses the privacy leakage risk measured by the server to calculate the local model encryption mask, and then performs encryption consensus through the server to form the final encryption mask; (3) In the encrypted federated learning phase, the client uses the encryption mask to implement homomorphic encryption of some parameters to protect the model parameters and updates. The server can only aggregate mixed parameters (parameters with both plaintext and ciphertext), but cannot access the sensitive local model.

[0117] This embodiment first considers a scenario where inference attacks are encountered in federated learning. The specific steps include:

[0118] Step 1: Model privacy measurement and inference attack initialization: The server simulates attacker behavior, performs inference attacks on the client model, calculates the risk of model privacy leakage, and determines the model privacy risk value by combining the number of model parameters, computational complexity, and communication complexity; at the same time, the GRNN inference attack program is uploaded to the server and the GRNN attack program is started.

[0119] Step 2, Sensitive Parameter Analysis: Based on the comprehensive privacy risks provided by the server, the client calculates the sensitivity of local model parameters and evaluates the degree of influence of parameters on the output by using the second derivative of the gradient.

[0120] Step 3: Encryption Mask Generation: The server aggregates the sensitivity vectors uploaded by each client, selects the set of most sensitive parameters based on parameter sensitivity, and generates an encryption mask.

[0121] Step 4: Partial parameter homomorphic encryption: The client performs homomorphic encryption on the highly sensitive parameters marked "1" according to the encryption mask, while keeping the other parameters in plaintext and uploading them to the server;

[0122] Step 5, Secure Aggregation and Inference Attack: Server 1 performs aggregation operations in the ciphertext and plaintext domains respectively, and completes the weighted summation without decryption by using homomorphic encryption; at the same time, the GRNN attack program steals the gradient information uploaded by the client and recovers the client's private training data through the program.

[0123] Step 6, Model Update: The server sends the aggregated model parameters to the client. The client decrypts the encrypted part and reconstructs the complete global model, then enters the next round of training.

[0124] Thus, an efficient federated learning method and system can be realized to defend against some federated learning inference attacks.

[0125] To demonstrate the effectiveness of the proposed defense method, this invention employs a novel inference attack method, GRNN, to attack the defense framework in this embodiment, and the results are shown in the table.

[0126] Table 1 Attack Results

[0127]

[0128]

[0129] A negative SSIM value indicates that the difference between the images is even greater than that between random images.

[0130] Secondly, this embodiment also compares the communication volume with some other fully homomorphic encryption schemes, and the specific results are as follows: Figure 3 As shown in the figure, our method (DSPHS) is compared with CKKS and BFV in terms of communication overhead. Experiments were conducted on three widely used datasets: MNIST, CIFAR-10, and CIFAR-100. The results show that by encrypting only high-sensitivity parameters while transmitting low-sensitivity parameters in plaintext, the communication cost is significantly reduced. In most cases, its communication overhead is less than 20% of that of the full-parameter encryption method, fully demonstrating the efficiency of the dynamic sensitivity parameter grading strategy.

[0131] This embodiment also provides an efficient federated learning system, such as Figure 4As shown, the system employs the efficient federated learning method described above, and includes: a comprehensive privacy risk calculation module, a client-side sensitive parameter analysis module, an encryption mask generation and threshold dynamic adjustment module, a partial parameter hierarchical encryption module, a security aggregation module, and a model decryption and comprehensive privacy risk update module.

[0132] The modules deployed on the server include: a comprehensive privacy risk calculation module, an encryption mask generation and threshold dynamic adjustment module, a security aggregation module, and a model decryption and comprehensive privacy risk update module;

[0133] The modules deployed on the client include: a client-side sensitive parameter analysis module, a partial parameter hierarchical encryption module, and a model decryption and comprehensive privacy risk update module.

[0134] The comprehensive privacy risk calculation module is used to: calculate the comprehensive privacy risk using the model privacy metric function when federated learning begins;

[0135] The client-side sensitive parameter analysis module is used to: after the server has completed the privacy risk calculation, each client performs sensitivity calculations on the model parameters based on its local training data;

[0136] The encryption mask generation and threshold dynamic adjustment module is used for: the server to generate an encryption mask and dynamically determine the global sensitivity threshold;

[0137] The partial parameter hierarchical encryption module is used to: divide model parameters into high-sensitivity encryption areas, medium-sensitivity mixed areas, and low-sensitivity plaintext areas, and encrypt them hierarchically.

[0138] The secure aggregation module is used by the server to perform secure aggregation in both the ciphertext and plaintext domains.

[0139] The model decryption and comprehensive privacy risk update module is used for: client-side model decryption for training and updating of comprehensive privacy risks.

[0140] Example 2: This example considers a federated learning application scenario suitable for collaborative maintenance of industrial robots across factories. A large industrial group has deployed multiple automated manufacturing plants globally, each equipped with a variety of robot types, including welding robots, handling robots, painting robots, and assembly robots. These robots generate heterogeneous sensor data from multiple sources during long-term operation, resulting in large data volumes, complex structures, and significant differences in operating conditions. To avoid the direct sharing of raw sensor waveforms between factories, which could lead to privacy and trade secret leaks, the group aims to use federated learning to train a fault prediction model that is cross-model, cross-operating condition, and possesses strong generalization capabilities. Combined with the dynamic grading strategy for sensitive parameters proposed in this invention, the above application can be implemented as follows:

[0141] Step 1: Model Privacy Risk Assessment and Attack Simulation. The server first simulates an attack on the global model using a small subset of proxy data, based on a privacy metric. Attack methods include member inference attacks (determining whether a specific robot appears in the training set) and gradient inversion attacks (attempting to reconstruct the original vibration signal or current waveform from uploaded gradients).

[0142] Simulations revealed that the LSTM-based temporal feature extraction layer in the model achieved a high success rate in attacks on several factories, indicating that the local sensitivity of the parameters satisfies the following:

[0143]

[0144] These parameters fall into the high-end region of the long tail of sensitivity, which is considered a high-risk area. Based on this, the server marks a portion of the LSTM time step weights as the initial high-sensitivity encryption zone.

[0145] Step 2: Local sensitivity analysis of client-side data based on sliding window data. Each factory client constructs time-series training batches using a 10-minute sliding window and calculates local gradient information:

[0146]

[0147] The results show that a small number of parameters in the weight matrix from the hidden layer to the output layer of the LSTM have a significant amplification effect on the logits change of the fault category. Their sensitivity exceeds that of the local distribution high quantile function. These parameters are automatically classified as high-sensitivity set by the client, while the parameters in the middle part remain in the medium-sensitivity mixed region, and the small gradient part at the tail is classified as low-sensitivity plaintext region.

[0148] Step 3: Server-side aggregation of sensitivity distribution and generation of encryption mask. The server performs homomorphic aggregation on the encryption sensitivity vectors uploaded by all factories and statistically models the skewness κ and kurtosis η of the global sensitivity. It is found that it exhibits a significant long-tail distribution, satisfying:

[0149] K >> 0, η >> 0

[0150] The server uses the dynamic threshold formula of this invention to determine the encryption range:

[0151]

[0152] Based on this, an encryption mask is generated, and homomorphic encryption is performed only on a small number of parameters that fall into the upper segment of the interval, thereby significantly reducing the overall encryption ratio.

[0153] Step 4: The client performs hierarchical encryption and hybrid upload. Upon receiving the encryption mask, each factory edge gateway performs different processing methods according to the hierarchical strategy: the LSTM parameters in the high-sensitivity encryption area are homomorphically encrypted using the CKKS encryption scheme; lightweight Gaussian noise is injected into the parameters in the medium-sensitivity hybrid area (e.g., deep convolutional kernels of CNN); and the parameters in the low-sensitivity plaintext area are uploaded in plaintext. The hybrid parameters sent by the client ensure that the high-sensitivity parts are invisible while reducing unnecessary encryption overhead.

[0154] Step 5: The server performs cross-domain weighted aggregation. The server performs a weighted average on the highly sensitive parameters in the encrypted domain:

[0155]

[0156] The server performs a weighted average of the remaining parameters in the plaintext domain:

[0157]

[0158] Step 6: Client decryption. The client performs private key decryption on the ciphertext result:

[0159]

[0160] Example 3: This example considers constructing a federated learning system for classifying benign and malignant pulmonary nodules across hospitals using the method of this invention. Each hospital within a regional medical alliance possesses a large amount of chest CT scan data. However, due to the highly sensitive medical privacy involved and the restrictions imposed by regional laws and regulations, hospitals are prohibited from sharing original patient images; therefore, federated learning must be used. Within this framework, to resist gradient inversion attacks and model inference leakage, the sensitivity dynamic grading mechanism of this invention can be implemented according to the following steps:

[0161] Step 1: Simulate an attack on the server side. The server uses 3D DenseNet as the attack model to attempt to reconstruct gradients from various hospital areas. When a significantly high SSIM (structural similarity index) is observed (i.e., the gradient contains obvious lesion texture information), the server maps the relevant parameters to high-risk areas and uses this as a reference indicator for subsequent classification.

[0162] Step 2: The client locally calculates the second-order gradient sensitivity and performs personalized grading. Each campus calculates the second-order gradient of the parameters using its local validation set:

[0163]

[0164] Since the transition layers of DenseNet involve feature compression operations, their parameter changes are more sensitive to the probability of malignancy (logits), thus naturally placing them in the high-end region of the sensitivity ranking.

[0165] Step 3: The server aggregates sensitivity and performs distribution pattern analysis. After receiving the encryption sensitivity data, the server aggregates the ciphertext and models the skewness κ and kurtosis η of the global distribution. It is found that the distribution exhibits hyperpeak characteristics, i.e., a large number of low-sensitivity parameters and a small number of extremely high-sensitivity parameters. Based on the bimodal clustering strategy proposed in this invention, the server automatically divides the parameters into three regions: high, medium, and low, and automatically generates a hierarchical strategy according to the distribution, without the need for manual setting of critical points.

[0166] Step 4: Implement a tiered encryption strategy. Each hospital processes the parameters according to the encryption mask issued by the server. Homomorphic encryption is used for the 3DDenseNet transition layer parameters in the high-sensitivity encryption region; multiplicative random noise θ′ is injected into the compression factor γ in the medium-sensitivity mixed region. i =θ i •δ, δ∈[1-ρ,1+ρ]; Low-sensitivity plaintext areas are uploaded in plaintext. This hybrid upload method effectively reduces encryption computation and transmission overhead, especially when the number of 3D model parameters is large.

[0167] Step 5: Server-side hierarchical encryption. The server performs a weighted average of the parameters in the high-sensitivity area within the ciphertext domain:

[0168]

[0169] Among them, w c The settings can be adaptively adjusted according to the number of nodule samples in each hospital area to ensure the statistical fairness of the results.

[0170] For the plaintext portions of the medium-sensitivity mixed area and the low-sensitivity plaintext area, the server employs a structured weighted fusion mechanism:

[0171]

[0172] Among them, the high-sensitivity encryption zone λ l The value should be close to 1 to ensure that subtle, malignant features are not revealed; λ in the intermediate-sensitivity mixed region l The size should be appropriately reduced to balance model stability and privacy.

[0173] Step 6: Client-side decryption. After decrypting the encrypted portion, the client performs Gaussian smoothing correction on the parameters, a process commonly used in medical imaging: θ = G. σ *θ is used to reduce local gradient mutations caused by encryption. Subsequently, the client analyzes the diameter distribution of lung nodules detected in the new round. If the average diameter changes significantly compared to the previous round, it is considered that the lesion morphology may have evolved. To increase the encryption strength in sensitive areas, the client automatically lowers the global threshold.

[0174]

[0175] By following the steps above, more robust privacy protection can be achieved, preventing malicious features from being leaked in the gradient.

[0176] This invention addresses the challenge of balancing privacy protection and communication efficiency in practical applications of federated learning by proposing an efficient federated learning method and system. This method constructs a privacy metric function to classify the sensitivity of model parameters and generates a uniform encryption mask globally. This allows for homomorphic encryption only on high-sensitivity parameters, perturbation on medium-sensitivity parameters, and plaintext transmission for low-sensitivity parameters. Theoretically, this mechanism effectively defends against attacks such as gradient inversion and member inference while significantly reducing communication and computational overhead. Experimental results show that, on various datasets and models, it can reduce communication overhead to one-fifth to one-seventh of traditional full-parameter encryption methods while maintaining model accuracy, and exhibits good scalability on large-scale models (such as ResNet-50).

[0177] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An efficient federated learning method and system, characterized in that, Includes the following steps: Step 1, Comprehensive Privacy Risk Calculation Module: The server first simulates the behavior of potential attackers and performs privacy attack simulations on the global model, such as member inference or gradient inversion, to preliminarily assess the privacy leakage risk of the model; Then the server uses the sample data to calculate the degree of influence of each parameter on the model output, and calculates the overall privacy risk through the model privacy metric function. This module needs to be deployed on a server; Step 2, Client-side Sensitive Parameter Analysis Module: Each client calculates the sensitivity of model parameters based on local training data; then, normalization is performed on all parameter sensitivity information; finally, the client uploads the parameter sensitivity information to the server. Step 3, Encryption Mask Generation and Dynamic Threshold Adjustment: The server performs security aggregation on the parameter sensitivity information uploaded by each client and conducts a unified analysis on the overall distribution of all parameters; based on the analysis results, the server dynamically determines the global sensitivity threshold and generates a global encryption mask; This mask is used to mark which parameters require strong encryption, which only require light protection, and which can be transmitted in plaintext; finally, the server sends the encryption mask to each client. Step 4, Partial Parameter Hierarchical Encryption Module: After receiving the encryption mask sent by the server, the client classifies the model parameters according to the encryption mask. Specifically, the model parameters are divided into high-sensitivity encryption area, medium-sensitivity mixed area, and low-sensitivity plaintext area. Homomorphic encryption is applied to highly sensitive parameters to ensure that they remain in ciphertext state throughout transmission and aggregation. Apply slight perturbations or random noise to medium-sensitivity parameters; keep low-sensitivity parameters in plaintext to avoid unnecessary encrypted calculations; finally, the client uploads all parameters to the server. Step 5, Secure Aggregation Module: After receiving the parameters uploaded by each client, the server performs secure aggregation in both the ciphertext and plaintext domains; finally, the server issues the aggregated global model parameters. Step 6, Model Decryption and Comprehensive Privacy Risk Update Module: After receiving the aggregated global model parameters from the server, the client uses its local private key to decrypt the encrypted parts of the parameters and recombine them with the plaintext parameters to form complete model parameters; subsequently, the local sensitivity is updated to achieve dynamic adjustment of the global sensitivity threshold.

2. The method and system according to claim 1, characterized in that: Deploy the model privacy metric module on the server, specifically: Step 1.1: Calculation of Empirical Privacy Guarantee; First, the server simulates the behavior of a potential attacker to attack the existing model. The false positive rate (FPR) and false negative rate (FNR) are calculated based on the attack success rate. Then, the empirical privacy guarantee is calculated using these two metrics. in, n is the number of clients, and privacy guarantees are provided. It is used to measure the strength of privacy guarantees, generally speaking A smaller value indicates a higher level of privacy protection, while a larger value indicates a lower level of privacy protection. Step 1.2, Model Complexity Calculation; The specific calculation of the model complexity MC is as follows: MC = α·P + β·FLOPs + γ·Com Where P is the number of parameters in the model; FLOPs is the computational complexity of the model, i.e. the number of floating-point operations in the model; Com is the communication complexity of the model, i.e. the amount of data transmitted between the client and the server during the federated learning process; α, β, γ are weighting coefficients used to balance the proportions of the number of parameters, computational complexity, and communication complexity in the total complexity, with the default values ​​of α=1, β=1, and γ=1. Step 1.3, Attacker Capability Calculation; The attacker's capability in federated learning can be uniformly quantified as attack gain G, the expression of which is: G=(1-μ)·MIA_Acc+μ·Rec_Err Where MIA_Acc represents the F1 score of the member inference attack, which measures the attacker's ability to determine whether a record belongs to the training set; Rec_Err represents the minimum inverse loss of the data reconstruction attack, which is the reconstruction error when the attacker restores the gradient / model update to the original input. The smaller the error, the more successful the attack; μ∈[0,1] represents the attacker's preference weights, μ→1 emphasizes reconstruction, and μ→0 emphasizes inference; when G→1, it means that the attacker has both high inference accuracy and low reconstruction error rate; G→0 means that the attack capability is effectively weakened. Step 1.4, Calculation of Comprehensive Privacy Risk; Calculate the comprehensive privacy risk ρ using the following formula: Where w1, w2, and w3 are the weights of each part, used to control the actual range of ρ. The default values ​​are w1 = 0.6, w2 = 0.2, and w3 = 0.

2. Preferably, for the calculation of empirical privacy guarantees in step 1.1, there is a more efficient calculation method, as follows: Step 1.1.1: Preliminary Sensitivity Estimation of Parameters Based on Gradient Energy; The server first simulates various inference attacks using a small number of proxy samples, including membership inference attacks and gradient inversion attacks, and performs one forward and backward propagation to obtain the gradient matrix of the model parameters. To avoid the high costs associated with repeated attacks, this step uses the gradient squared energy method to calculate local sensitivity: Where s f This represents the local sensitivity of the i-th parameter; this metric measures the direct impact of the parameter on changes in the model's loss function and can be calculated within a single propagation. i This represents the gradient of the i-th parameter; Step 1.1.2, Gradient Distribution Skewness Correction and Comprehensive Privacy Risk Modeling; Since the parameter sensitivity in neural networks is usually distributed in a long tail, directly using the mean or variance as the screening criterion will cause "local over-encryption"; Therefore, this invention introduces a gradient distribution skewness correction mechanism. The server statistical local sensitivity sample set S = {s1, s2, ..., s} is S = {s1, s2, ..., s}. θ } and calculate skewness κ and kurtosis η: When κ > 0 and η > 3, it indicates the existence of a highly sensitive parameter group; E[·] represents the mathematical expectation operator. That is, the sensitivity samples s of all parameters i The average is calculated, where σ represents the standard deviation, i.e., the sensitivity value s. i The degree of fluctuation; Step 1.1.3, Empirical Privacy Guarantee Calculation: The empirical privacy guarantee can be calculated using the following formula based on skewness κ and kurtosis η. Here, a and b are adjustment factors used to control the degree of influence of κ and η on the value of e. The default values ​​are a=1 and b=1.

3. The method and system according to claim 1, characterized in that: Deploy the sensitive parameter analysis module on each client; after completing the global privacy measurement, each client, based on its own data distribution, performs local sensitive parameter filtering, requiring each client to use the sensitive parameter analysis module for analysis. Specific steps include: Step 2.1: Calculation of the second derivative tensor; To avoid the enormous time consumption caused by repeatedly performing inference attacks and evaluating their success rate in actual training, parameter sensitivity is used as a proxy indicator for parameter leakage risks; Specifically, the second derivative tensor J is first calculated. m (y k ): Among them, w m Let m represent the m-th parameter, m∈{0,1,2,…,θ}, where θ is the total number of parameters, X and y represent the input matrix and label vector of the given model W and K samples, respectively, and l(·) represents the loss function; Step 2.2, Parameter Sensitivity Calculation: To characterize the sensitivity τ of the gradient of the m-th parameter to the true output. m Take J m (y k Its average absolute value over the sample dimension is calculated as follows: After each client i calculates the sensitivity of each parameter in turn, a sensitivity vector is obtained. After the calculation is complete, each client will send the sensitivity vector. Uploaded to the server for subsequent global aggregation of encryption masks.

4. The method and system according to claim 1, characterized in that: The encryption mask generation and threshold dynamic adjustment module is deployed on the server. The server aggregates the encryption sensitivity vectors of each client using homomorphic addition and achieves sensitivity consistency correction among multiple clients through Z-score normalization. It automatically determines the dynamic threshold based on the global distribution skewness, generating a binary encryption mask with global consistency and dynamic adaptability. Specifically: Step 3.1: Client Sensitivity Aggregation and Consistency Correction; The server receives the set of encryption sensitivity vectors uploaded by all clients. Aggregation calculations can be performed within the ciphertext domain using the properties of homomorphic addition. Where w i The client-side weights can be adaptively set based on the data size, reputation, or training epochs. The default is... n is the number of clients; The server decrypts to obtain the global sensitivity vector. And achieve uniformity of sensitivity distribution among multiple clients through Z-score standardization: Where μ g For M global The average value, σ g For M global The variance of ∈ is a minimum value to prevent the denominator from being 0. The default ∈ = 0.

1. This correction process eliminates the bias caused by data heterogeneity between clients, making the mask generation based on a uniform statistical scale. Step 3.2: Determining the global dynamic threshold based on skewed distribution; In traditional federated learning encryption, sensitive parameters are often selected using fixed proportions or manually set, lacking data-driven approaches; Based on the server's perception of the global sensitivity distribution M... norm Calculate skewness κ g and kurtosis η g Automatically determine the dynamic threshold Ω max and Ω min : Where λ1 and λ2 are balance coefficients used to adjust the encryption strength and plaintext ratio, with the default values ​​of λ1 = 0.5 and λ2 = 0.5, and skewness κ. g and kurtosis η g The calculation formula can be found in step 1; For M norm The average value; Step 3.3: Global encryption mask generation and sparsity optimization; the server generates a binary encryption mask based on the filtering results. The definition is as follows: Where, p m ∈(0,1) is a randomization flag used for minor disturbance protection of medium-sensitivity parameters, thereby improving system robustness.

5. The method and system according to claim 1, characterized in that: A partial parameter hierarchical encryption module was adopted to obtain the global encryption mask. Subsequently, each client performs hierarchical encryption and hybrid transmission of its local model parameters based on the mask, achieving minimization of homomorphic encryption computation and optimization of communication efficiency. Its specific steps include: Step 4.1: Matching parameter hierarchy and encryption strategy; the client obtains the global mask. Then, the local parameters are divided into three sets: Among them, the high-sensitivity encryption zone Implement strong homomorphic encryption protection to prevent member inference and gradient inversion attacks; Medium-sensitivity hybrid zone Semi-encryption protection is achieved using light perturbations (such as multiplicative noise and random permutations); low-sensitivity plaintext area. To maximize communication and computation efficiency, plaintext transmission is maintained. Step 4.2, Homomorphic Encryption Execution and Key Management Mechanism: The client uses the shared public key pk to perform encryption operations and generate a ciphertext parameter vector: Among them, Enc pk (w m ) represents the parameters encrypted using the public key pk, Perturb(w m The parameters are those with random noise added. Meanwhile, the client locally retains the private key `sk` for subsequent decryption. After processing, all parameters are combined into a set. And upload it to the server.

6. An efficient federated learning system, wherein, The system employs the efficient federated learning method described above, and includes: a comprehensive privacy risk calculation module, a client sensitive parameter analysis module, an encryption mask generation and threshold dynamic adjustment module, a partial parameter hierarchical encryption module, a security aggregation module, and a model decryption and comprehensive privacy risk update module. The modules deployed on the server include: a comprehensive privacy risk calculation module, an encryption mask generation and threshold dynamic adjustment module, a security aggregation module, and a model decryption and comprehensive privacy risk update module; The modules deployed on the client include: a client-side sensitive parameter analysis module, a partial parameter hierarchical encryption module, and a model decryption and comprehensive privacy risk update module. The comprehensive privacy risk calculation module is used to: calculate the comprehensive privacy risk using the model privacy metric function when federated learning begins; The client-side sensitive parameter analysis module is used to: after the server has completed the privacy risk calculation, each client performs sensitivity calculations on the model parameters based on its local training data; The encryption mask generation and threshold dynamic adjustment module is used for: the server to generate an encryption mask and dynamically determine the global sensitivity threshold; The partial parameter hierarchical encryption module is used to: divide model parameters into high-sensitivity encryption areas, medium-sensitivity mixed areas, and low-sensitivity plaintext areas, and encrypt them hierarchically; The secure aggregation module is used by the server to perform secure aggregation in both the ciphertext and plaintext domains. The model decryption and comprehensive privacy risk update module is used for: client-side model decryption for training and updating of comprehensive privacy risks.

Images