Byzantine attack-resistant clustering federated learning framework and secure aggregation method and system
By introducing verifiable orthogonal matrix obfuscated aggregation and secure ReLU function calculations in cluster federated learning framework, combining gradient segmentation and hierarchical aggregation, the problem of Byzantine attack and efficiency in cluster federated learning is solved, and efficient privacy protection is achieved and system security.
Patent Information
- Application Number
- CN202411030322.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-22
AI Technical Summary
The existing technology has not effectively resisted Byzantine attacks under the distributed computing framework, and the secure aggregation scheme of traditional federated learning is difficult to balance efficiency and robustness, especially in clustered federated learning scenarios.
A cluster federated learning framework was designed to achieve resistance to Byzantine attacks through the collaboration of clients, servers and trusted institutions using verifiable orthogonal matrix obfuscation aggregation and secure ReLU function calculations, combining gradient encoding and key management. , and reduce computing and communication overhead through gradient segmentation and hierarchical aggregation.
Effectively defend against Byzantine attacks, ensure system security and stability, while improving privacy protection and efficiency, and is suitable for a variety of distributed computing scenarios.
Smart Images

Figure CN119011136A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of network security, and in particular relates to a clustering federated learning framework and a security aggregation method and system for resisting Byzantine attacks. Background Art
[0002] Machine learning under the distributed computing framework has been widely studied in various scenarios. Federated learning, as one of the modern distributed learning frameworks, aims to make full use of the data and computing power on edge devices. Although federated learning does not require uploading client local data, avoiding privacy leakage caused by direct server access to client data, this distributed training method still has some security risks. For example, the server can perform inference attacks through the model or gradient uploaded by the client to restore the client local data. In addition, Byzantine adversaries can hinder the training process by uploading poisonous gradients.
[0003] In the field of efficient and secure aggregation, Bonawitz et al. proposed a lightweight protocol that allows servers to securely aggregate model updates from multiple clients without accessing local gradients. Choi et al. proposed a low-complexity scheme using sparse random graphs to protect data privacy through the topological structure of secret sharing nodes. Kadhe et al. proposed the FastSecAgg framework, which used fast Fourier transform (FFT) to create a multi-secret sharing scheme FastShare to achieve efficient computing and communication. Jahani-Nezhad et al. proposed SwiftAgg+, which significantly reduced communication overhead and achieved privacy protection and efficient communication. However, these solutions have failed to solve the problem of Byzantine attacks.
[0004] In the research field of resisting Byzantine attacks, Liu et al. proposed the PEFL framework, which uses homomorphic encryption for privacy protection and punishes poisonous gradients by extracting the logarithmic function of the gradient, solving the problems of privacy protection and poisoning attacks. Zhang proposed the SAFELearning technology, which supports secure aggregation and detects backdoor attacks through inadvertent random grouping (ORG) and partial parameter leakage (PPD), effectively preventing malicious attacks and reducing the success rate of backdoor attacks. The solution proposed by Wan uses an adaptive client selection strategy to deal with Byzantine attacks, but does not encrypt or obfuscate gradients.
[0005] At present, research work mainly focuses on the study of secure aggregation and Byzantine robustness algorithms for traditional federated learning, but does not consider the compatibility of the scheme with cluster federated learning, and few research works can comprehensively consider efficiency, security and robustness. Therefore, designing an efficient and secure aggregation framework that is compatible with cluster federated learning and has Byzantine robustness is a research topic with broad application prospects and is extremely challenging. (1) First, traditional secure aggregation schemes are mainly implemented through homomorphic encryption or multi-party secure computing, which has high computational and communication overheads and is difficult to balance security and system performance at the same time; (2) Secondly, Byzantine robustness is an important attribute of federated learning systems, but most advanced Byzantine robustness algorithms require complex calculations and are difficult to implement under the framework of secure aggregation. On the other hand, for homomorphic encryption and multi-party secure computing, complex operations will greatly increase the computational and communication overhead of the overall system; (3) In addition, the privacy protection scheme for federated learning based on differential privacy faces a trade-off between security and system performance. Systems based on differential privacy add noise information to meet security requirements, and the training of federated learning systems is difficult to converge, which ultimately leads to reduced model performance. At the same time, in order to improve system performance and reduce noise, differential privacy cannot meet the constraints of privacy protection; (4) Finally, cluster federated learning requires training multiple models according to different clusters, making most solutions designed for traditional federated learning no longer applicable, and the complexity of cluster federated learning makes the design of solutions more difficult. Therefore, there is a lack of efficient privacy protection algorithms that are compatible with cluster federated learning, and a secure aggregation method and system that can resist Byzantine adversaries to ensure the overall security of the federated learning system. Summary of the invention
[0006] In view of the problems existing in the prior art, the purpose of the present invention is to provide a cluster federated learning framework and a secure aggregation method and system that can resist Byzantine attacks. This method and system innovatively propose a cluster federated learning secure aggregation framework and can effectively resist Byzantine attacks on malicious clients. By taking into account both privacy security and system performance, it can be widely adapted to a variety of distributed computing application scenarios, which is specifically implemented through the following technical solutions:
[0007] A clustering federated learning framework and security aggregation system that resists Byzantine attacks. The system includes the following three participants:
[0008] 1) Client: responsible for training local data using the encoding key, encoding the obtained gradients, and uploading the encoded gradients to the server;
[0009] 2) Server: responsible for broadcasting the global model, verifying the gradients uploaded by the client, and performing weighted clustering aggregation;
[0010] 3) Trusted organization: responsible for generating and distributing keys based on security parameters, as well as managing keys.
[0011] A clustering federated learning framework and secure aggregation method that resists Byzantine attacks. The method includes the following six components:
[0012] (1) A clustering federated learning system with Byzantine robustness consisting of a clustering model update algorithm and a robust clustering federated learning algorithm;
[0013] (2) Verifiable orthogonal matrix obfuscation aggregation: It has the function that a single piece of data cannot be decrypted and can only be decrypted after all the data are aggregated. It also verifies the gradient encoding submitted by the client to prevent the client from not executing the protocol according to the rules and bypassing the robustness algorithm check. Specifically:
[0014] For 1≤i≤n, the i-th client encodes the secret x as follows i :
[0015] χ i =x i M i +R i M i+n ;
[0016] in is a mutually orthogonal matrix group; construct the verification key VK as The verification function is as follows:
[0017]
[0018] in is with R i The random matrix with the same dimension, the decoding key DK is constructed as Satisfies for 1≤i≤n:
[0019] χ i ·Dk T =x i +R i ,
[0020]
[0021] in, represents the ciphertext tampered by the adversary,
[0022] satisfy:
[0023]
[0024] in λ represents the probability of the pseudo-random number being destroyed, l represents x i size;
[0025] (3) Secure ReLU function calculation: In order to eliminate the negative gradient of cosine similarity, it is necessary to calculate the ReLU function output of cosine similarity. Secure ReLU function calculation can securely calculate the ReLU function of multi-dimensional input values and obtain encrypted output. The value obtained after decryption is equivalent to calculating the ReLU function output for each dimension of the multi-dimensional input value and summing all the output values. In addition, the encrypted output can be inner-producted with other multi-dimensional data, and the inner-product result can be obtained by decryption;
[0026] (4) Aggregation scheme: Based on the above clustering federated learning framework with Byzantine robustness, a secure aggregation scheme is designed. In each round of training, the number of participating clients is represented by n, the number of cluster identities is represented by m, and the dimension of the gradient is represented by l. It specifically includes the following five steps:
[0027] 4-1) Initialization: TA pre-generates multiple sets of mutually orthogonal matrices and The server will update the gradient Sent to TA, who then generates β, τ1, τ2, and τ3, as well as the key, for secure ReLU function calculation VK and DK, as described below:
[0028] (M) i represents the i-th row of the matrix M, Where 1≤i≤n, 1≤j≤m, ||R″ ij ||=1, And ||μ i ||=1;
[0029] For 1≤i≤2lmn:
[0030]
[0031] in is a random matrix generated by TA;
[0032] 4-2) Broadcast phase: In the first round, TA will EK i and Sent to the i-th client; in subsequent rounds, TA updates R ij ″ to update And the updated To send to the i-th client, the server first classifies the current model using the public dataset and then sends the global model to the selected client;
[0033] 4-3) Training phase: The client training is implemented through the clustering model update algorithm and uploaded to the server. Assuming that the cluster of the i-th client is j, the gradient encoding is
[0034] 4-4) Aggregation stage: TA will β, τ1, τ2, DK and VK are sent to the server, which verifies each client according to the verification function in the verifiable orthogonal matrix confusion aggregation and checks whether 1≤i≤n satisfies ‖δ i ‖ 2 =3; if the verification function returns 1, it can be verified that the client has not tampered with the ciphertext; if ‖δ i ‖ 2 =3, it can be verified that the client has performed the normalization operation If the verification fails, the server will request the failed client to resend its gradient code or restart the algorithm execution process; for 1≤i≤n, the server calculates α as follows i :
[0035] (α i ) j =δ i α j '
[0036] The server calculates the function according to the steps in the secure ReLU function calculation and
[0037] Finally, the server computes the aggregate gradient g as follows:
[0038]
[0039] 4-5) Update phase: For 1≤j≤m, the server updates as follows:
[0040]
[0041] (5) Security key conversion: for 1≤i≤n, 1≤j≤m, and the inverse function Construct the transmission key TK i As For χ i The key conversion and transmission function are as follows:
[0042]
[0043] Where c is a constant, Denotes other mutually orthogonal matrices, χ i =x i M i+R i M i+n ,as well as Satisfies the following equation:
[0044]
[0045] in
[0046] (6) Compression scheme: Reduce the gradient dimension of a single aggregation through gradient splitting and minimize the number of clients involved in each group aggregation through hierarchical aggregation.
[0047] Furthermore, the specific steps of the clustering model updating algorithm in step (1) are:
[0048] a) Input: Initialization parameters Training data set D, batch size b, learning rate η, number of iterations T;
[0049] b) Set the initial parameters to
[0050] c) Calculate and select the model with the smallest loss among the initial models
[0051] d) Perform T iterations: Randomly extract a batch D from the training dataset D b , update the parameters
[0052] e) Returns the selected model index j and the gradient
[0053] Furthermore, the robust clustering federated learning algorithm in step (2) is specifically:
[0054] The server sets up m clusters and initializes m models. The server collects a small and clean public dataset and trains to obtain m server updates, denoted as The server broadcasts all global models, the client receives the model and runs the clustering model update algorithm to train the local model, and uploads the updated gradient and cluster identity; the server first classifies each client according to its corresponding cluster identity, and then calculates the cosine similarity between the client gradient and the server update gradient, and removes the toxic gradient with negative similarity:
[0055]
[0056] Where 1≤k≤m, 1≤i, j≤n,
[0057] Furthermore, in step (3), the secure ReLU function is calculated as follows:
[0058] Defining functions To obtain the square of each element in the matrix while preserving the original sign of each element, define The inverse function of , which calculates the square root of the absolute value of each element while preserving the original sign; the function and is defined as follows:
[0059]
[0060] in, For 1≤i≤n, input Define the encoding of x as:
[0061]
[0062] structure as follows:
[0063]
[0064] in, represents mutually orthogonal matrices;
[0065] Construct β, τ1, τ2 and τ3 as follows:
[0066]
[0067] in represents the Hadamard product, R i ′ is a reversible matrix,
[0068] For 1≤j≤lm, 1≤k≤2lmn, and is a random number, and satisfy:
[0069] If (M i ) jk ≤0,
[0070]
[0071] in If (M i ) jk >0,
[0072]
[0073] in
[0074] It can be calculated as
[0075]
[0076] and It can be calculated as
[0077]
[0078] Furthermore, in step (6), the specific process of the compression scheme is:
[0079] 1) Gradient segmentation
[0080] a) The i-th client flattens the local gradient by row and concatenates it layer by layer to form a vector g i , let vector g i is divided into s segments, represented as Each segment has a length of t, satisfying s·t≥l. For 1≤i≤n, 1≤j≤m, and 1≤k≤s, R″ and μ satisfy:
[0081]
[0082] b) For 1≤i≤n, represents the i-th client, and the corresponding inner product is calculated as follows:
[0083]
[0084] c) For 1≤i≤3mnt, 1≤j≤n, and 1≤k≤s,
[0085]
[0086] Aggregate all the fragments and then connect the fragments back to the original gradient;
[0087] 2) Hierarchical aggregation: The total number of clients is denoted as n, and the clients are divided into groups, with the maximum size of each group being
[0088] TA generates EK, VK and α for each group i ′, then TA generates the conversion key for k groups and generates the remaining keys for all groups after conversion; in addition, ζ, μ, R″ and Shared among all groups;
[0089] The specific steps of aggregation are as follows:
[0090] a) First, in the aggregation phase, the server calculates However, the subsequent steps are not performed because each group cannot eliminate the random matrix and has no corresponding DK before key conversion;
[0091] b) Secondly, the server performs key conversion in each group and uses the converted result as input for a new round of conversion until all key conversions are completed;
[0092] c) Finally, the server performs the update phase;
[0093] The server can group all participants and recursively divide each group into smaller groups. The conversion process can be regarded as the aggregation of a group. The server uses the conversion key to aggregate the small groups into large groups and finally aggregate all participants. The number of conversion rounds can be expressed as x, and the total size of the conversion key is It can be expressed as follows:
[0094]
[0095] in satisfy
[0096] For 1≤i≤n, 1≤j≤x, the key transformation is as follows:
[0097]
[0098] where v = i-(i mod ξ j ), Compared with the prior art, the present invention has the following beneficial effects:
[0099] 1) Combining the advanced clustering federated learning framework with the robustness algorithm, a clustering federated learning framework that resists Byzantine attacks is designed. This invention can effectively defend against attacks from Byzantine clients while achieving efficient clustering training, thus ensuring the overall security and stability of the system;
[0100] 2) A verifiable orthogonal matrix confusion aggregation algorithm is designed, which uses mutually orthogonal matrix groups to efficiently encode client gradients, ensuring that the encoded gradients are decoded to zero in non-corresponding clusters, thereby effectively masking sensitive information in the gradients. This invention protects the privacy of client gradients without affecting the aggregation effect of the final model. In addition, combined with the verification mechanism, the invention can effectively ensure that the client strictly abides by the protocol, further improving the security and reliability of the solution;
[0101] 3) A secure ReLU function calculation mechanism is proposed, which can securely calculate the sum of multi-input encoded ReLU values to prevent the server from accessing or tampering with individual ReLU values. The non-interactive secure calculation of the ReLU function is innovatively implemented. After receiving all necessary parameters, the server does not need to interact with any entity and can independently run the security protocol to complete the ReLU function calculation. While greatly reducing communication overhead, the security and efficiency of the scheme are significantly improved. As a key component of the robustness algorithm, this mechanism provides strong support for robust aggregation in federated learning and can effectively deal with attacks from Byzantine clients;
[0102] 4) By integrating advanced federated learning technology and matrix technology, a high-performance privacy protection solution is designed. This solution has high availability and can be widely applied in practical application scenarios. It can achieve efficient distributed learning while ensuring data privacy;
[0103] 5) Based on an in-depth analysis of the characteristics of the matrix and combined with the specific design process of the algorithm, an efficient compression scheme was designed. Using gradient segmentation and hierarchical aggregation, the dimension of the key matrix was effectively reduced without affecting the functionality and security of the overall scheme, thereby greatly improving communication and computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0104] Figure 1 This is the system architecture diagram;
[0105] Figure 2 It is a block diagram and data flow diagram of the robust clustering federated learning algorithm;
[0106] Figure 3 This is the algorithm block diagram and data flow diagram of the aggregation scheme. DETAILED DESCRIPTION
[0107] The present invention is further described below in conjunction with the accompanying drawings in order to better understand the technical solution.
[0108] This invention is based on the typical application scenario of cluster federated learning system. The overall architecture of the system is as follows: Figure 1 As shown, it includes the following three parties:
[0109] 1) Client: Responsible for training local data using the encoding key, encoding the obtained gradients, and uploading the encoded gradients to the server. Malicious clients will try to attack the global model by uploading poisonous gradients or running the gradient encoding algorithm in violation of the rules to bypass the robustness algorithm;
[0110] 2) Server: Responsible for broadcasting the global model, verifying the gradients uploaded by the client, and performing weighted clustering aggregation. A malicious server can attempt multiple aggregations, select some gradients for aggregation to obtain the plaintext gradients of a specific client, or execute the Byzantine robustness algorithm not according to the rules to exclude the target gradient and access its plaintext information;
[0111] 3) Trusted Authority (TA): Responsible for generating and distributing keys based on security parameters, as well as managing keys.
[0112] The overall solution of the present invention includes the following 6 components, the algorithm block diagram and data flow diagram are as follows: Figure 2 and Figure 3 As shown:
[0113] (1) Clustered Federated Learning System with Byzantine Robustness
[0114] The algorithm framework and data flow diagram are as follows Figure 2 As shown, the system consists of the following two algorithms:
[0115] 1) Clustering model update algorithm:
[0116] a) Input: Initialization parameters Training data set D, batch size b, learning rate η, number of iterations T;
[0117] b) Set the initial parameters to
[0118] c) Calculate and select the model with the smallest loss among the initial models
[0119] d) Perform T iterations: Randomly extract a batch D from the training dataset D b , update the parameters
[0120] e) Returns the selected model index j and the gradient
[0121] 2) Robust clustering federated learning algorithm: The server sets up m clusters and initializes m models. The server collects a small and clean public dataset and trains to obtain m server updates, expressed as The server broadcasts all global models, and the client receives the model and runs the clustering model update algorithm to train the local model, and uploads the updated gradient and cluster identity. The server first classifies each client according to its corresponding cluster identity, and then calculates the cosine similarity between the client gradient and the server update gradient, and removes the toxic gradient with negative similarity:
[0122]
[0123] Where 1≤k≤m, 1≤i, j≤n,
[0124] (2) Verifiable orthogonal matrix obfuscation aggregation: Verifiable orthogonal matrix obfuscation aggregation is the key privacy-preserving aggregation technology of the present invention. Single data cannot be decrypted and can only be decrypted after all data are aggregated. In addition, the present invention also verifies the gradient encoding submitted by the client to prevent the client from not executing the protocol according to the rules and bypassing the robustness algorithm check.
[0125] For 1≤i≤n, the i-th client encodes the secret x as follows i :
[0126] χ i =x i M i +R i M i+n
[0127] in is a mutually orthogonal matrix group. Construct the verification key VK as The verification function is as follows:
[0128]
[0129] in is with R i The decoding key DK is constructed as Satisfies for 1≤i≤n:
[0130] χ i ·Dk T =x i +R i ,
[0131]
[0132] in Indicates ciphertext that has been tampered with by an adversary.
[0133] satisfy:
[0134]
[0135] in λ represents the probability of the pseudo-random number being destroyed, l represents x i size;
[0136] (3) Secure ReLU function calculation: In order to remove the negative gradient of cosine similarity, it is necessary to calculate the ReLU function output of cosine similarity. Secure ReLU function calculation can securely calculate the ReLU function of multi-dimensional input values and obtain encrypted output. The value obtained after decryption is equivalent to calculating the ReLU function output for each dimension of the multi-dimensional input value and summing all the output values. In addition, the encrypted output can be inner-producted with other multi-dimensional data and decrypted to obtain the inner-product result.
[0137] Defining functions to obtain the square of each element in the matrix while preserving the original sign of each element. The inverse function of , which computes the square root of the absolute value of each element while preserving the original sign. and is defined as follows:
[0138]
[0139] in For 1≤i≤n, input Define the encoding of x as:
[0140]
[0141] structure as follows:
[0142]
[0143] in Represents mutually orthogonal matrices.
[0144] Construct β, τ1, τ2 and τ3 as follows:
[0145]
[0146] in represents the Hadamard product, R i ′ is a reversible matrix,
[0147] For 1≤j≤lm, 1≤k≤2lmn, and is a random number, and satisfy:
[0148] If (M i ) jk ≤0,
[0149]
[0150] in If (M i ) jk >0,
[0151]
[0152] in
[0153] It can be calculated as:
[0154]
[0155] and It can be calculated as
[0156]
[0157] (4) Aggregation scheme: Based on the cluster federated learning framework with Byzantine robustness designed by the present invention, a secure aggregation scheme is designed. In each round of training, the number of participating clients is represented by n, the number of cluster identities is represented by m, the dimension of the gradient is represented by l, the algorithm block diagram of the aggregation scheme, and the data flow diagram are shown in Fig. Figure 3 As shown, it includes the following 5 steps:
[0158] 1) Initialization: TA pre-generates multiple sets of mutually orthogonal matrices and The server will update the gradient Sent to TA, who then generates β, τ1, τ2, and τ3, as well as the key, for secure ReLU function calculation VK and DK, as described below:
[0159] (M) i represents the i-th row of the matrix M, Where 1≤i≤n, 1≤j≤m, ||R″ ij ||=1, And ||μ i ||=1.
[0160] For 1≤i≤2lmn:
[0161]
[0162] in is a random matrix generated by TA;
[0163] 2) Broadcast phase: In the first round, TA will broadcast EK i and Sent to the i-th client. In subsequent rounds, TA updates R ij ″ to update And the updated The server first classifies the current model using the public dataset, and then sends the global model to the selected client;
[0164] 3) Training phase: The client training is implemented through the clustering model update algorithm and uploaded to the server. Assume that the cluster of the i-th client is j, and the gradient encoding is
[0165] 4) Aggregation stage: TA will β, τ1, τ2, DK and VK are sent to the server. The server verifies each client according to the verification function in the verifiable orthogonal matrix confusion aggregation and checks whether 1≤i≤n satisfies ‖δ i ‖ 2 =3. If the verification function returns 1, it can be verified that the client has not tampered with the ciphertext. i ‖ 2 =3, it can be verified that the client has performed the normalization operation If the verification fails, the server will request the failed client to resend its gradient encoding or restart the algorithm execution process. For 1≤i≤n, the server calculates α as follows i :
[0166] (α i ) j =δ i α j '
[0167] The server calculates the function according to the steps in the secure ReLU function calculation and
[0168] Finally, the server computes the aggregate gradient g as follows:
[0169]
[0170]
[0171] 5) Update phase: For 1≤j≤m, the server updates as follows:
[0172]
[0173] (5) Security key conversion: for 1≤i≤n, 1≤j≤m, and the inverse function Construct the transmission key TK i As For χ i The key conversion and transmission function are as follows:
[0174]
[0175] Where c is a constant, Denotes other mutually orthogonal matrices, χ i =x i M i +R i M i+n ,as well as Satisfies the following equation:
[0176]
[0177] in
[0178] (6) Compression scheme: The compression scheme reduces the gradient dimension of a single aggregation by gradient segmentation and minimizes the number of clients involved in each group aggregation by hierarchical aggregation. The specific process is as follows:
[0179] 1) Gradient segmentation
[0180] a) The i-th client flattens the local gradient by row and concatenates it layer by layer to form a vector g i . Let vector g i is divided into s segments, represented as Each segment has a length of t, satisfying s·t≥l. For 1≤i≤n, 1≤j≤m, and 1≤k≤s, R″ and μ satisfy:
[0181]
[0182] b) For 1≤i≤n, represents the i-th client, and the corresponding inner product is calculated as follows:
[0183]
[0184] c) For 1≤i≤3mnt, 1≤j≤n, and 1≤k≤s,
[0185]
[0186] Aggregate all the fragments and then connect the fragments back to the original gradient;
[0187] 2) Hierarchical aggregation
[0188] The total number of clients is denoted as n, and the clients are divided into groups ξ, with the maximum size of each group being
[0189] TA generates EK, VK and α for each group i ′. Then, TA generates the conversion key for k groups and generates the remaining key for all groups after conversion. In addition, ζ, μ, R″ and Shared among all groups.
[0190] The specific steps of aggregation are as follows:
[0191] a) First, in the aggregation phase, the server calculates However, the subsequent steps are not performed because each group cannot eliminate the random matrix and has no corresponding DK before key conversion;
[0192] b) Secondly, the server performs key conversion in each group and uses the converted result as input for a new round of conversion until all key conversions are completed;
[0193] c) Finally, the server performs the update phase;
[0194] The server can group all participants and recursively divide each group into smaller groups. The conversion process can be regarded as the aggregation of a group. The server uses the conversion key to aggregate the small groups into large groups and finally aggregate all participants. The number of conversion rounds can be represented as x. The total size of the conversion key is It can be expressed as follows:
[0195]
[0196] in satisfy
[0197] For 1≤i≤n, 1≤j≤x, the key transformation is as follows:
[0198]
[0199] where v = i-(i mod ξ j ),
Claims
1. A clustering federated learning framework and security aggregation system that resists Byzantine attacks, characterized by: The system includes the following three parties: 1) Client: responsible for training local data using the encoding key, encoding the obtained gradients, and uploading the encoded gradients to the server; 2) Server: responsible for broadcasting the global model, verifying the gradients uploaded by the client, and performing weighted clustering aggregation; 3) Trusted organization: responsible for generating and distributing keys based on security parameters, as well as managing keys.
2. A clustering federated learning framework and secure aggregation method that resists Byzantine attacks, characterized in that: The approach consists of the following six components: (1) A clustering federated learning system with Byzantine robustness consisting of a clustering model update algorithm and a robust clustering federated learning algorithm; (2) Verifiable orthogonal matrix obfuscation aggregation: It has the function that a single piece of data cannot be decrypted and can only be decrypted after all the data are aggregated. It also verifies the gradient encoding submitted by the client to prevent the client from not executing the protocol according to the rules and bypassing the robustness algorithm check. Specifically: For 1≤i≤n, the i-th client encodes the secret x as follows i : x i =x i M i +R i M i+n ; in is a mutually orthogonal matrix group; construct the verification key VK as The verification function is as follows: in is with R i The random matrix with the same dimension, the decoding key DK is constructed as Satisfies for 1≤i≤n: x i ·Dk T =x i +R i , in, represents the ciphertext tampered by the adversary, satisfy: in λ represents the probability of the pseudo-random number being destroyed, l represents x i size; (3) Secure ReLU function calculation: In order to eliminate the negative gradient of cosine similarity, it is necessary to calculate the ReLU function output of cosine similarity. Secure ReLU function calculation can securely calculate the ReLU function of multi-dimensional input values and obtain encrypted output. The value obtained after decryption is equivalent to calculating the ReLU function output for each dimension of the multi-dimensional input value and summing all the output values. In addition, the encrypted output can be inner-producted with other multi-dimensional data, and the inner-product result can be obtained by decryption; (4) Aggregation scheme: Based on the above clustering federated learning framework with Byzantine robustness, a secure aggregation scheme is designed. In each round of training, the number of participating clients is represented by n, the number of cluster identities is represented by m, and the dimension of the gradient is represented by l. It specifically includes the following five steps: 4-1) Initialization: TA pre-generates multiple sets of mutually orthogonal matrices and The server will update the gradient Sent to TA, who then generates β, τ1, τ2, and τ3, as well as the key, for secure ReLU function calculation VK and DK, as described below: (M) i Represents the i-th row of matrix M: Where 1≤i≤n, 1≤j≤m, ||R″ ij ||=1, And ||μ i ||=1; For 1≤i≤2lmn: in is a random matrix generated by TA; 4-2) Broadcast phase: In the first round, TA will EK i and Sent to the i-th client; in subsequent rounds, TA updates R ij ″ to update And the updated To send to the i-th client, the server first classifies the current model using the public dataset and then sends the global model to the selected client; 4-3) Training phase: The client training is implemented through the clustering model update algorithm and uploaded to the server. Assuming that the cluster of the i-th client is j, the gradient encoding is 4-4) Aggregation stage: TA will β, τ1, τ2, DK and VK are sent to the server, which verifies each client according to the verification function in the verifiable orthogonal matrix confusion aggregation and checks whether 1≤i≤n satisfies ‖δ i ‖ 2 =3; if the verification function returns 1, it can be verified that the client has not tampered with the ciphertext; if ‖δ i ‖ 2 =3, it can be verified that the client has performed the normalization operation If the verification fails, the server will request the failed client to resend its gradient code or restart the algorithm execution process; for 1≤i≤n, the server calculates α as follows i : (a i ) j =d i ·a j ′ The server calculates the function according to the steps in the secure ReLU function calculation and Finally, the server computes the aggregate gradient g as follows: 4-5) Update phase: For 1≤j≤m, the server updates as follows: (5) Security key conversion: for 1≤i≤n, 1≤j≤m, and the inverse function Construct the transmission key TK i As For X i The key conversion and transmission function are as follows: Where c is a constant, Denotes other mutually orthogonal matrices, χ i =x i M i +R i M i+n ,as well as Satisfies the following equation: in (6) Compression scheme: Reduce the gradient dimension of a single aggregation through gradient splitting and minimize the number of clients involved in each group aggregation through hierarchical aggregation.
3. A clustering federated learning framework and security aggregation method for resisting Byzantine attacks as described in claim 2, characterized in that: The specific steps of the clustering model update algorithm in step (1) are: a) Input: Initialization parameters Training data set D, batch size b, learning rate η, number of iterations T; b) Set the initial parameters to c) Calculate and select the model with the smallest loss among the initial models d) Perform T iterations: Randomly extract a batch D from the training dataset D b , update the parameters e) Returns the selected model index j and the gradient 4. A clustering federated learning framework and security aggregation method for resisting Byzantine attacks as described in claim 2, characterized in that: The robust clustering federated learning algorithm in step (2) is specifically: The server sets up m clusters and initializes m models. The server collects a small and clean public dataset and trains to obtain m server updates, denoted as The server broadcasts all global models, the client receives the model and runs the clustering model update algorithm to train the local model, and uploads the updated gradient and cluster identity; the server first classifies each client according to its corresponding cluster identity, and then calculates the cosine similarity between the client gradient and the server update gradient, and removes the toxic gradient with negative similarity: Where 1≤k≤m, 1≤i, j≤n, 5. A clustering federated learning framework and security aggregation method for resisting Byzantine attacks as described in claim 2, characterized in that: In step (3), the secure ReLU function is calculated as follows: Defining functions To obtain the square of each element in the matrix while preserving the original sign of each element, define The inverse function of , which calculates the square root of the absolute value of each element while preserving the original sign; the function and is defined as follows: in, For 1≤i≤n, input Define the encoding of x as: structure as follows: in, represents mutually orthogonal matrices; Construct β, τ1, τ2 and τ3 as follows: in represents the Hadamard product, R i ′ is a reversible matrix, For 1≤j≤lm, 1≤k≤2lmn, and is a random number, and satisfy: If (M i ) jk ≤0, in If (M i ) jk >0, in It can be calculated as and It can be calculated as 6. A clustering federated learning framework and security aggregation method for resisting Byzantine attacks as described in claim 2, characterized in that: In step (6), the specific process of the compression scheme is: 1) Gradient segmentation a) The i-th client flattens the local gradient by row and concatenates it layer by layer to form a vector g i , let vector g i is divided into s segments, represented as Each segment has a length of t, satisfying s·t≥l. For 1≤i≤n, 1≤j≤m, and 1≤k≤s, R″ and μ satisfy: b) For 1≤i≤n, represents the i-th client, and the corresponding inner product is calculated as follows: c) For 1≤i≤3mnt, 1≤j≤n, and 1≤k≤s, Aggregate all the fragments and then connect the fragments back to the original gradient; 2) Hierarchical aggregation: The total number of clients is denoted as n, and the clients are divided into groups, with the maximum size of each group being TA generates EK, VK and α for each group i ′, then TA generates the conversion key for k groups and generates the remaining keys for all groups after conversion; in addition, ζ, μ, R″ and Shared among all groups; The specific steps of aggregation are as follows: a) First, in the aggregation phase, the server calculates However, the subsequent steps are not performed because each group cannot eliminate the random matrix and has no corresponding DK before key conversion; b) Secondly, the server performs key conversion in each group and uses the converted result as input for a new round of conversion until all key conversions are completed; c) Finally, the server performs the update phase; The server can group all participants and recursively divide each group into smaller groups. The conversion process can be regarded as the aggregation of a group. The server uses the conversion key to aggregate the small groups into large groups and finally aggregate all participants. The number of conversion rounds can be expressed as x, and the total size of the conversion key is It can be expressed as follows: in satisfy For 1≤i≤n, 1≤j≤x, the key transformation is as follows: where v = i-(i mod ξ j ),
Citation Information
Cited By
Asynchronous distributed key generation method and system with optimal time complexity
CN119382887A
Multi-source trusted federal learning-oriented privacy enhancement aggregation method and system
CN120579220A
Privacy protection elastic distributed solver design method and system for resisting hybrid attacks in resource configuration and storage medium
CN121644224A
