Efficient communication privacy protection federated learning method and system based on commutative permutation

Abstract

The application discloses a high-efficiency communication privacy protection federated learning method and system based on commutative permutation, and relates to the technical field of intelligent security calculation. The method comprises the following steps: based on a federated learning local training mechanism, performing privacy protection gradient processing according to a client data set and current global model parameters to obtain a sub-permutation and a gradient sharing share; based on a preset mask offset constraint condition, performing mask preprocessing according to the sub-permutation and the gradient sharing share to obtain an aligned intermediate representation and a server mask; and based on the aligned intermediate representation and the server mask, performing parameter updating using a collaborative aggregation calculation method according to the current global model parameters to obtain updated global model parameters. The application is a communication privacy protection federated learning method based on a secure aggregation mechanism, which is high in efficiency and strong in robustness.

Description

Technical Field

[0001] This invention relates to the field of intelligent secure computing technology, and in particular to an efficient communication privacy-preserving federated learning method and system based on interchangeable permutations. Background Technology

[0002] With the rapid development of artificial intelligence technology, Federated Learning (FL), as an important distributed machine learning method, can achieve multi-party collaborative modeling without sharing the original data, and is therefore widely used in data privacy protection scenarios. In the federated learning process, each client trains its model based on local data and periodically uploads model update information, such as gradients or parameters, to the server. The server then aggregates the model update information to obtain the global model parameters.

[0003] However, in practical applications, as model size increases, the dimensionality of gradient vectors uploaded by the client increases significantly, making communication overhead a major bottleneck restricting the performance of federated learning systems. To reduce the communication burden, various gradient compression methods have been proposed. By transmitting only some important gradient information, compressing gradient representations, or reducing data redundancy, the amount of communication data can be significantly reduced while maintaining model accuracy. Among these, the gradient importance-based sparsification method has become a commonly used communication optimization technique due to its simplicity and high compression efficiency. Compared to transmitting the entire original gradient vector, gradient sparsification significantly reduces communication complexity, thereby improving the training efficiency and scalability of federated learning systems.

[0004] While gradient sparsification offers significant advantages in communication efficiency, it still presents key challenges in privacy protection. During sparsification, in addition to the gradient values, the corresponding gradient position indices must also be transmitted. The set of gradient indices essentially reflects the importance distribution of model parameters, and its statistical characteristics are strongly correlated with the distribution of local client data. Even if the gradient values ​​are encrypted or secretly shared, attackers can still infer client data characteristics by analyzing the distribution patterns of the indices, leading to privacy breaches.

[0005] To address the aforementioned issues, existing technologies are gradually incorporating privacy protection mechanisms into federated learning. These mechanisms employ methods such as encrypted computation, secure multi-party computation, and secret sharing to protect gradient information and enhance system security. Building upon this foundation, a new approach is proposed that simultaneously protects both gradient values ​​and their position index information to achieve more comprehensive privacy protection. For example, a secure aggregation scheme based on Replicated Secret Sharing (RSS) can distribute data shares across multiple servers, completing computation without restoring the original data, thereby achieving joint protection of gradient values ​​and index information.

[0006] The methods described above still have significant shortcomings in terms of system robustness. Existing solutions typically rely on multiple servers executing computational processes in a predetermined order, and their correctness depends on the complete participation of each participating node. Once a server node goes offline or communication is interrupted, the original computational link will be disrupted, causing the aggregation process to be unable to continue. This type of method has a structural dependency on computational order, meaning that the computational processes between different participants are not interchangeable, making it difficult to maintain computational correctness in the event of node absence.

[0007] In the existing technology, there is a lack of an efficient and robust federated learning method for communication privacy protection based on a secure aggregation mechanism. Summary of the Invention

[0008] To address the shortcomings of existing technologies in index privacy protection, which makes them unsuitable for high-security applications; the difficulty in simultaneously achieving communication efficiency and privacy protection in sparse gradient compression scenarios; the reliance on multiple participating nodes executing computations in a fixed order, resulting in insufficient system robustness; and the lack of structural fault-tolerance mechanisms due to the non-commutative nature of computational operations between different participating nodes, this invention provides an efficient communication-privacy-preserving federated learning method and system based on commutative permutations. The technical solution is as follows: On the one hand, an efficient communication privacy-preserving federated learning method based on commutative permutations is provided. This method is implemented by an efficient communication privacy-preserving federated learning device and includes: Retrieve the client dataset and the current global model parameters; Based on the federated learning local training mechanism, privacy-preserving gradient processing is performed according to the client dataset and the current global model parameters to obtain sub-permutations and gradient sharing shares. Based on the preset mask cancellation constraint, the mask preprocessing is performed according to the sub-permutation and gradient sharing share to obtain the aligned intermediate representation and the server mask. Based on the aligned intermediate representation and the server mask, the parameters are updated using a collaborative aggregation calculation method according to the current global model parameters, thus obtaining the updated global model parameters.

[0009] On the other hand, an efficient communication privacy-preserving federated learning system based on commutative permutations is provided. This system is applied to an efficient communication privacy-preserving federated learning method based on commutative permutations. The system includes a client and a server. The client is used to obtain the client dataset and the current global model parameters; based on the federated learning local training mechanism, it performs privacy-preserving gradient processing according to the client dataset and the current global model parameters to obtain sub-permutations and gradient sharing shares; The server is used to perform mask preprocessing based on a preset mask cancellation constraint, according to sub-permutation and gradient sharing share, to obtain an aligned intermediate representation and a server mask; based on the aligned intermediate representation and the server mask, it uses a collaborative aggregation calculation method to update the parameters according to the current global model parameters, to obtain updated global model parameters.

[0010] On the other hand, an efficient communication privacy-preserving federated learning device is provided, the efficient communication privacy-preserving federated learning device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the methods described above for efficient communication privacy-preserving federated learning methods based on interchangeable permutations is implemented.

[0011] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described efficient communication privacy-preserving federated learning methods based on interchangeable permutations.

[0012] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention proposes an efficient communication privacy-preserving federated learning method based on commutative permutations. By protecting gradient information through secret sharing, and through index preprocessing and commutative permutation construction, the position information corresponding to the gradient cannot be directly mapped to the position of the original model parameters during transmission and calculation, thereby effectively avoiding the privacy risks caused by index leakage and achieving dual protection of gradient values ​​and index information. By sparsifying the gradients, only the important gradient components and their corresponding information are transmitted, significantly reducing the amount of communication data, thereby reducing the communication overhead in the federated learning process and improving the overall system efficiency. Based on the interchangeable permutation construction, the order of different permutation combinations does not affect the final calculation result during multi-server collaborative computing; when one server node goes offline, the remaining server nodes can still complete the correct aggregation calculation, thereby improving the robustness and fault tolerance of the system. By restructuring the structure and aligning the unified index space, the data of each server node is processed under a consistent structure. Combined with a random mask generation mechanism, the correctness of the final aggregation result is ensured while hiding the intermediate calculation results, thus avoiding information leakage and calculation deviation. This invention reduces the possibility of a single server or some nodes recovering the original gradient information through the collaborative design of multiple mechanisms such as secret sharing, interchangeable permutation and random masking, thereby enhancing the overall security of the system and making it suitable for federated learning application scenarios with high privacy protection requirements. While achieving strong privacy protection, this invention features a clear structure for each computational step, controllable communication and computational overhead, and maintains high operating efficiency while ensuring system security, demonstrating significant engineering application value. This invention is a highly efficient and robust federated learning method for communication privacy protection based on a secure aggregation mechanism. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of an efficient communication privacy-preserving federated learning method based on commutative permutation provided in an embodiment of the present invention; Figure 2 This is a block diagram of an efficient communication privacy-preserving federated learning system based on interchangeable permutations provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a high-efficiency communication privacy-preserving federated learning device provided in an embodiment of the present invention. Detailed Implementation

[0015] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0016] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0017] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0018] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0019] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0020] This invention provides an efficient communication privacy-preserving federated learning method based on commutative permutations. This method can be implemented by an efficient communication privacy-preserving federated learning device, which can be a terminal or a server. Figure 1 The flowchart shown is for an efficient communication privacy-preserving federated learning method based on commutative permutations. The processing flow of this method may include the following steps: S1. Obtain the client dataset and current global model parameters; In one feasible implementation, the client dataset in this invention consists of private training data held locally by each participant, including but not limited to image data, text data, structured data, or multimodal data. The client dataset does not participate in cross-node transmission. The current global model parameters are the global model state variables broadcast to each client after the previous round of server aggregation processing, and their dimensions correspond to the model structure.

[0021] S2. Based on the federated learning local training mechanism, privacy-preserving gradient processing is performed according to the client dataset and the current global model parameters to obtain sub-permutations and gradient sharing shares. Optionally, based on the federated learning local training mechanism, privacy-preserving gradient processing is performed according to the client dataset and the current global model parameters to obtain sub-permutations and gradient sharing shares, including: Based on the current global model parameters, local model training is performed using the client dataset to obtain the client gradient vector; The client gradient vector is sparsified to obtain a set of position indices, a set of candidate indices, and a sparse gradient representation. Based on the fixed-point elimination method and the cyclic generator construction method, a commutative permutation is constructed according to the position index set and the candidate index set to obtain the sub-permutation; Based on the replication secret sharing mechanism, the share splitting process is performed according to the sparse gradient representation to obtain the gradient sharing share received by the server.

[0022] In one feasible implementation, the client initializes the local model based on the current global model parameters, performs forward propagation calculation on the client dataset using a preset loss function, and calculates the client gradient vector. The dimension of the client gradient vector is consistent with the dimension of the current global model parameters.

[0023] Client In the Wheel based on local data The gradient is calculated as follows: (1) (1); in, The preset loss function, These are the pre-global model parameters. It is a client Local data, It is the gradient operator, used to differentiate the model parameters.

[0024] Top-K sparsification is performed on the client gradient vector, and the K gradient components with the largest absolute values ​​are selected to obtain the set of position indices, the set of candidate indices, and the sparse gradient representation; fixed points in the set of position indices are detected and rearranged or expanded.

[0025] Fixed points can cause permutations to degenerate into identity mappings, thereby reducing the index obfuscation effect and increasing the risk of location leakage. The mathematical expression for detecting whether an index has a fixed point, i.e., the index position being the same as its mapping result, is as follows (2). (2); in, For location index mapping; This is the mapping result.

[0026] Indexes with fixed points can be rearranged or expanded. For example, when multiple fixed points exist, a new loop structure can be constructed to map the original fixed points to different positions; when the number of fixed points is insufficient to form a loop, auxiliary indexes can be introduced for expansion, so that the processed index does not correspond to the original position. That is, a new mapping is constructed. This satisfies the following mathematical expression (3): (3); Output processed index mapping , used for subsequent substitution construction.

[0027] The processed index map is decomposed into disjoint cyclic structures and extended to prime length. Based on the extended cyclic structures, cyclic generators are constructed, and multiple sub-permutations that satisfy the commutative law are generated through exponentiation operations and distributed to each server node.

[0028] The sparse gradient representation is split using a replicated secret sharing mechanism, resulting in gradient sharing shares sent to each server node. The secret sharing of sparse gradient information employs a replicated secret sharing (RSS) mechanism. And sent to three servers respectively. , , This makes it impossible for any single server node to directly recover complete gradient information.

[0029] This invention reduces the amount of communication data through sparsification (e.g., Top-K sparsification) and combines a secret sharing mechanism to split gradient information into multiple shared shares, thereby reducing communication overhead while protecting the privacy of gradient values.

[0030] Optionally, the client gradient vector is sparsified to obtain a set of position indices, a set of candidate indices, and a sparse gradient representation, including: Based on the preset sparsity rules, the gradient components of the client gradient vector are filtered to obtain the filtered gradient components and the unfiltered gradient components. The indexes are extracted based on the filtered and unfiltered gradient components, and then sorted to obtain a set of position indices and a set of candidate indices. Based on the location index set, gradient values ​​are extracted from the client gradient vector, and a sparse gradient representation is constructed.

[0031] In one feasible implementation, the gradient information is sparsified, and sorted or equivalent selection is performed according to the absolute value of the gradient components. The position index corresponding to the gradient component that satisfies the preset sparsification rule is selected. This step is shown in the following formula (4). (4); The selected indices are sorted to form an ordered index set; the corresponding gradient values ​​are extracted from the original gradients based on the index set to construct a sparse gradient representation; the index information and corresponding gradient values ​​are recorded for subsequent processing; and a complete index set is generated for subsequent permutation and expansion processing.

[0032] The corresponding gradient value is extracted as shown in equation (5): (5); in, d represents the dimension of the gradient vector, and K represents the number of gradient components to be retained.

[0033] Optionally, based on the fixed-point elimination method and the cyclic generator construction method, commutative permutations are constructed according to the position index set and the candidate index set to obtain sub-permutations, including: Based on the inconsistency constraint of index mapping relationship, fixed point elimination processing is performed on the location index set to obtain the processed index set; Based on the processed index set, the structure is decomposed using the loop decomposition method to obtain non-overlapping loop structures; Based on the preset loop length condition, the loop structure is lengthened according to the candidate index set to obtain an extended loop structure; Based on the cyclic generator construction method, exponentiation is performed according to the extended cyclic structure to obtain sub-permutations that satisfy the commutative law.

[0034] In one feasible implementation, the preprocessed index information is replaced and constructed.

[0035] Loop decomposition represents the index as several non-overlapping loop structures. The mathematical expression for this process is as follows (6). (6); in, This represents a loop structure consisting of index mappings; Indicates the index of the element in the loop; This indicates the length of the loop structure.

[0036] When the length of the loop structure does not meet the preset length condition, auxiliary index elements are selected from the candidate index set and added to the corresponding loop structure so that the length of the expanded loop meets the preset length condition; wherein, the preset length condition is preferably a prime number length condition; Based on the extended loop structure, a loop generator is constructed, and multiple sub-permutations are generated based on the loop generator. This process is as follows: (7) (7); in, These represent the three sub-permutations generated; Let represent the generation parameters of the corresponding sub-permutation, and satisfy . .in, Represents a set of integers. The exponentiation of a loop structure represents the number of iterations within the loop structure. As a loop generator, a loop shift operation is applied continuously a corresponding number of times.

[0037] By using parameter constraints and construction methods, any two sub-permutations can satisfy the commutative law, so that the order of different permutation combinations does not affect the final result. This process is as follows (8). (8); in, This indicates a permutation composition operation. and This indicates the number corresponding to different sub-permutations.

[0038] Exchangeability enables subsequent collaborative computations between different server nodes to be independent of a fixed execution order. Multiple sub-permutations are assigned to different server nodes for subsequent order-independent collaborative aggregation computations.

[0039] This invention not only protects the gradient values, but also protects the privacy of the index information by preprocessing and permuting the indexes corresponding to the gradients. This prevents the indexes from directly corresponding to the positions of the original model parameters during transmission and calculation.

[0040] S3. Based on the preset mask cancellation constraint, perform mask preprocessing according to the sub-permutation and gradient sharing share to obtain the aligned intermediate representation and the server mask. Optionally, based on preset mask cancellation constraints, mask preprocessing is performed according to sub-permutations and gradient sharing shares to obtain aligned intermediate representations and server masks, including: Based on a preset dimension, the dimension is expanded according to the gradient sharing share to obtain the expanded gradient sharing share. Based on sub-permutation, index space alignment is performed according to the extended gradient shared share to obtain an aligned intermediate representation; Based on the preset mask cancellation constraint, a random mask is generated according to the aligned intermediate representation to obtain the server mask.

[0041] In one feasible implementation, in this invention, three server nodes respectively receive the corresponding shared shares and fill the shared shares to expand the low-dimensional sparse representation into a vector representation of a predetermined dimension, so as to form a data structure of a unified format for subsequent calculation. This process is as follows: (9) (9); in, This indicates a padding operation, used to expand a low-dimensional sparse representation into a vector representation of a preset dimension based on the index position.

[0042] The three server nodes perform structural reconstruction on the padded data based on the exchangeable permutation to make it meet the structural requirements of the subsequent permutation process. The mathematical expression is as follows (10). (10); Under commutative permutation, the constructed sub-permutations satisfy the commutative property, ensuring consistency in the data reconstruction process across different server nodes within a unified index space, independent of the specific permutation execution order. The final output is an intermediate representation with a unified structure constructed based on the aligned data. .

[0043] Each of the three server nodes generates a random mask based on its own intermediate representation. The random mask satisfies a preset constraint relationship, which enables the random masks generated by multiple server nodes to cancel each other out during collaborative computation, thereby hiding intermediate computation results without affecting the correctness of the final aggregation result.

[0044] The masked intermediate representation is used for subsequent commutative permutation processing, enabling different server nodes to complete collaborative aggregation calculations under different permutation combination orders and maintain consistent aggregation results. The mathematical expressions that satisfy the conditions are as follows: (11) and (12): (11); (12); in, This represents the finite field used in the permutation construction and mask calculation. The meaning is defined on the finite field. 3D vector space. Formula (12) indicates that the random mask satisfies the cancellation constraint relationship, that is, the random masks generated by multiple server nodes cancel each other out during the aggregation process, so as to hide the intermediate calculation results without affecting the final aggregation result.

[0045] S4. Based on the aligned intermediate representation and the server mask, update the parameters using the collaborative aggregation calculation method according to the current global model parameters to obtain the updated global model parameters.

[0046] Optionally, based on the aligned intermediate representation and the server mask, the parameters are updated using a collaborative aggregation calculation method according to the current global model parameters to obtain the updated global model parameters, including: A partial shared representation is obtained by performing masking and obfuscation based on the aligned intermediate representation and the server mask. Based on the local shared representation, a collaborative aggregation computation is performed to obtain the intermediate gradient aggregation representation; Based on the intermediate gradient aggregation representation, the current global model parameters are updated to obtain the updated global model parameters.

[0047] In one feasible implementation, the three server nodes each assign a random mask. Add to the middle of the aligned representation Above, generate obfuscated locally shared results. And as input for subsequent collaborative aggregation calculations, the process is as follows (13): (13); because Alignment based on commutative permutations has been completed, so locally shared results can participate in subsequent collaborative aggregation calculations in an order-independent manner across different server nodes.

[0048] The three server nodes share the results. Perform a collaborative summation to obtain the intermediate gradient aggregation result. The process is as follows (14): (14); Since the random mask satisfies the cancellation constraint, it satisfies the following condition (15): (15); Because the intermediate representation has been aligned based on commutative permutations, different server nodes can perform collaborative aggregation computations in a sequence-independent manner. When one server node goes offline, the remaining server nodes can still perform correct aggregation computations based on commutative permutations and maintain consistent aggregation results.

[0049] The global model parameters are updated based on the aggregation results, and the updated model is broadcast to each client node. This process is as follows: (16) and (17): (16); (17); in, This indicates the learning rate used during the model parameter update process.

[0050] This invention proposes an efficient communication privacy-preserving federated learning method based on commutative permutations. By protecting gradient information through secret sharing, and through index preprocessing and commutative permutation construction, the position information corresponding to the gradient cannot be directly mapped to the position of the original model parameters during transmission and calculation, thereby effectively avoiding the privacy risks caused by index leakage and achieving dual protection of gradient values ​​and index information. By sparsifying the gradients, only the important gradient components and their corresponding information are transmitted, significantly reducing the amount of communication data, thereby reducing the communication overhead in the federated learning process and improving the overall system efficiency. Based on the interchangeable permutation construction, the order of different permutation combinations does not affect the final calculation result during multi-server collaborative computing; when one server node goes offline, the remaining server nodes can still complete the correct aggregation calculation, thereby improving the robustness and fault tolerance of the system. By restructuring the structure and aligning the unified index space, the data of each server node is processed under a consistent structure. Combined with a random mask generation mechanism, the correctness of the final aggregation result is ensured while hiding the intermediate calculation results, thus avoiding information leakage and calculation deviation. This invention reduces the possibility of a single server or some nodes recovering the original gradient information through the collaborative design of multiple mechanisms such as secret sharing, interchangeable permutation and random masking, thereby enhancing the overall security of the system and making it suitable for federated learning application scenarios with high privacy protection requirements. While achieving strong privacy protection, this invention features a clear structure for each computational step, controllable communication and computational overhead, and maintains high operating efficiency while ensuring system security, demonstrating significant engineering application value. This invention is a highly efficient and robust federated learning method for communication privacy protection based on a secure aggregation mechanism.

[0051] Figure 2 This is a block diagram of an efficient communication privacy-preserving federated learning system based on commutative permutations, provided by an embodiment of the present invention. This system is used for an efficient communication privacy-preserving federated learning method based on commutative permutations. (Refer to...) Figure 2 The system includes a client 210 and a server 220. Among them: Client 210 is used to obtain the client dataset and the current global model parameters; based on the federated learning local training mechanism, it performs privacy-preserving gradient processing according to the client dataset and the current global model parameters to obtain sub-permutations and gradient sharing shares. Server 220 is used to perform mask preprocessing based on sub-permutations and gradient sharing shares according to preset mask cancellation constraints to obtain aligned intermediate representations and server masks; based on aligned intermediate representations and server masks, parameters are updated using a collaborative aggregation calculation method according to the current global model parameters to obtain updated global model parameters.

[0052] Optionally, client 210 is further used for: Based on the current global model parameters, local model training is performed using the client dataset to obtain the client gradient vector; The client gradient vector is sparsified to obtain a set of position indices, a set of candidate indices, and a sparse gradient representation. Based on the fixed-point elimination method and the cyclic generator construction method, a commutative permutation is constructed according to the position index set and the candidate index set to obtain the sub-permutation; Based on the replication secret sharing mechanism, the share splitting process is performed according to the sparse gradient representation to obtain the gradient sharing share received by the server.

[0053] Optionally, client 210 is further used for: Based on the preset sparsity rules, the gradient components of the client gradient vector are filtered to obtain the filtered gradient components and the unfiltered gradient components. The indexes are extracted based on the filtered and unfiltered gradient components, and then sorted to obtain a set of position indices and a set of candidate indices. Based on the location index set, gradient values ​​are extracted from the client gradient vector, and a sparse gradient representation is constructed.

[0054] Optionally, client 210 is further used for: Based on the inconsistency constraint of index mapping relationship, fixed point elimination processing is performed on the location index set to obtain the processed index set; Based on the processed index set, the structure is decomposed using the loop decomposition method to obtain non-overlapping loop structures; Based on the preset loop length condition, the loop structure is lengthened according to the candidate index set to obtain an extended loop structure; Based on the cyclic generator construction method, exponentiation is performed according to the extended cyclic structure to obtain sub-permutations that satisfy the commutative law.

[0055] Optionally, server 220 is further used for: Based on a preset dimension, the dimension is expanded according to the gradient sharing share to obtain the expanded gradient sharing share. Based on sub-permutation, index space alignment is performed according to the extended gradient shared share to obtain an aligned intermediate representation; Based on the preset mask cancellation constraint, a random mask is generated according to the aligned intermediate representation to obtain the server mask.

[0056] Optionally, server 220 is further used for: A partial shared representation is obtained by performing masking and obfuscation based on the aligned intermediate representation and the server mask. Based on the local shared representation, a collaborative aggregation computation is performed to obtain the intermediate gradient aggregation representation; Based on the intermediate gradient aggregation representation, the current global model parameters are updated to obtain the updated global model parameters.

[0057] This invention proposes an efficient communication privacy-preserving federated learning method based on commutative permutations. By protecting gradient information through secret sharing, and through index preprocessing and commutative permutation construction, the position information corresponding to the gradient cannot be directly mapped to the position of the original model parameters during transmission and calculation, thereby effectively avoiding the privacy risks caused by index leakage and achieving dual protection of gradient values ​​and index information. By sparsifying the gradients, only the important gradient components and their corresponding information are transmitted, significantly reducing the amount of communication data, thereby reducing the communication overhead in the federated learning process and improving the overall system efficiency. Based on the interchangeable permutation construction, the order of different permutation combinations does not affect the final calculation result during multi-server collaborative computing; when one server node goes offline, the remaining server nodes can still complete the correct aggregation calculation, thereby improving the robustness and fault tolerance of the system. By restructuring the structure and aligning the unified index space, the data of each server node is processed under a consistent structure. Combined with a random mask generation mechanism, the correctness of the final aggregation result is ensured while hiding the intermediate calculation results, thus avoiding information leakage and calculation deviation. This invention reduces the possibility of a single server or some nodes recovering the original gradient information through the collaborative design of multiple mechanisms such as secret sharing, interchangeable permutation and random masking, thereby enhancing the overall security of the system and making it suitable for federated learning application scenarios with high privacy protection requirements. While achieving strong privacy protection, this invention features a clear structure for each computational step, controllable communication and computational overhead, and maintains high operating efficiency while ensuring system security, demonstrating significant engineering application value. This invention is a highly efficient and robust federated learning method for communication privacy protection based on a secure aggregation mechanism.

[0058] Figure 3 This is a schematic diagram of the structure of a high-efficiency communication privacy-preserving federated learning device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, a high-efficiency communication privacy-preserving federated learning device may include the above-mentioned Figure 2 The illustrated efficient communication privacy-preserving federated learning system is based on interchangeable permutations. Optionally, the efficient communication privacy-preserving federated learning device 310 may include a first processor 2001.

[0059] Optionally, the efficient communication privacy-preserving federal learning device 310 may also include a memory 2002 and a transceiver 2003.

[0060] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0061] The following is combined Figure 3 A detailed introduction to the various components of the Efficient Communication Privacy-Preserving Federated Learning Device 310: The first processor 2001 is the control center of the high-efficiency communication privacy-preserving federated learning device 310. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0062] Optionally, the first processor 2001 can perform various functions of the efficient communication privacy-preserving federated learning device 310 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0063] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0064] In a specific implementation, as one example, the efficient communication privacy-preserving federated learning device 310 may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0065] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0066] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected to the interface circuit of the efficient communication privacy-protected federated learning device 310. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0067] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0068] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0069] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and its interface circuitry protects the privacy of the federated learning device 310 through efficient communication. Figure 3 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0070] It should be noted that, Figure 3 The structure of the high-efficiency communication privacy-preserving federated learning device 310 shown in the diagram does not constitute a limitation on the router. Actual high-efficiency communication privacy-preserving federated learning devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0071] Furthermore, the technical effects of the efficient communication privacy-preserving federated learning device 310 can be referred to the technical effects of the efficient communication privacy-preserving federated learning method based on interchangeable permutations described in the above method embodiments, and will not be repeated here.

[0072] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0073] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0074] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0075] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0076] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0077] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0078] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0079] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, systems, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0080] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0081] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0082] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0083] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A highly efficient communication privacy-preserving federated learning method based on commutative permutations, characterized in that, The method includes: Retrieve the client dataset and the current global model parameters; Based on the federated learning local training mechanism, privacy-preserving gradient processing is performed according to the client dataset and the current global model parameters to obtain sub-permutations and gradient sharing shares. Based on the preset mask cancellation constraint, the mask preprocessing is performed according to the sub-permutation and gradient sharing share to obtain the aligned intermediate representation and the server mask. Based on the aligned intermediate representation and the server mask, the parameters are updated using a collaborative aggregation calculation method according to the current global model parameters, thus obtaining the updated global model parameters.

2. The efficient communication privacy-preserving federated learning method based on commutative permutation according to claim 1, characterized in that, The federated learning-based local training mechanism performs privacy-preserving gradient processing based on the client dataset and current global model parameters to obtain sub-permutations and gradient sharing shares, including: Based on the current global model parameters, local model training is performed using the client dataset to obtain the client gradient vector; The client gradient vector is sparsified to obtain a set of position indices, a set of candidate indices, and a sparse gradient representation. Based on the fixed-point elimination method and the cyclic generator construction method, a commutative permutation is constructed according to the position index set and the candidate index set to obtain the sub-permutation; Based on the replication secret sharing mechanism, the share splitting process is performed according to the sparse gradient representation to obtain the gradient sharing share received by the server.

3. The efficient communication privacy-preserving federated learning method based on commutative permutation according to claim 2, characterized in that, The process of sparsifying the client gradient vector to obtain a set of position indices, a set of candidate indices, and a sparse gradient representation includes: Based on the preset sparsity rules, the gradient components of the client gradient vector are filtered to obtain the filtered gradient components and the unfiltered gradient components. The indexes are extracted based on the filtered and unfiltered gradient components, and then sorted to obtain a set of position indices and a set of candidate indices. Based on the location index set, gradient values ​​are extracted from the client gradient vector, and a sparse gradient representation is constructed.

4. The efficient communication privacy-preserving federated learning method based on commutative permutation according to claim 2, characterized in that, The method based on fixed-point elimination and cyclic generator construction constructs sub-permutations by constructing commutative permutations based on the set of position indices and the set of candidate indices, including: Based on the inconsistency constraint of index mapping relationship, fixed point elimination processing is performed on the location index set to obtain the processed index set; Based on the processed index set, the structure is decomposed using the loop decomposition method to obtain non-overlapping loop structures; Based on the preset loop length condition, the loop structure is lengthened according to the candidate index set to obtain an extended loop structure; Based on the cyclic generator construction method, exponentiation is performed according to the extended cyclic structure to obtain sub-permutations that satisfy the commutative law.

5. The efficient communication privacy-preserving federated learning method based on commutative permutation according to claim 1, characterized in that, The pre-processing of the mask based on the preset mask cancellation constraint, according to the sub-permutation and gradient sharing share, yields the aligned intermediate representation and the server mask, including: Based on a preset dimension, the dimension is expanded according to the gradient sharing share to obtain the expanded gradient sharing share. Based on sub-permutation, index space alignment is performed according to the extended gradient shared share to obtain an aligned intermediate representation; Based on the preset mask cancellation constraint, a random mask is generated according to the aligned intermediate representation to obtain the server mask.

6. The efficient communication privacy-preserving federated learning method based on commutative permutation according to claim 1, characterized in that, The step of updating global model parameters based on aligned intermediate representations and server masks, using a collaborative aggregation calculation method, to obtain updated global model parameters includes: A partial shared representation is obtained by performing masking and obfuscation based on the aligned intermediate representation and the server mask. Based on the local shared representation, a collaborative aggregation computation is performed to obtain the intermediate gradient aggregation representation; Based on the intermediate gradient aggregation representation, the current global model parameters are updated to obtain the updated global model parameters.

7. A high-efficiency communication privacy-preserving federated learning system based on commutative permutations, wherein the high-efficiency communication privacy-preserving federated learning system based on commutative permutations is used to implement the high-efficiency communication privacy-preserving federated learning method based on commutative permutations as described in any one of claims 1-6, characterized in that, The system includes a client and a server: The client is used to obtain the client dataset and the current global model parameters; based on the federated learning local training mechanism, it performs privacy-preserving gradient processing according to the client dataset and the current global model parameters to obtain sub-permutations and gradient sharing shares; The server is used to perform mask preprocessing based on a preset mask cancellation constraint, according to sub-permutation and gradient sharing share, to obtain an aligned intermediate representation and a server mask. Based on the aligned intermediate representation and the server mask, the parameters are updated using a collaborative aggregation calculation method according to the current global model parameters, thus obtaining the updated global model parameters.

8. The efficient communication privacy-preserving federated learning system based on commutative permutation according to claim 7, characterized in that, The client is further used for: Based on the current global model parameters, local model training is performed using the client dataset to obtain the client gradient vector; The client gradient vector is sparsified to obtain a set of position indices, a set of candidate indices, and a sparse gradient representation. Based on the fixed-point elimination method and the cyclic generator construction method, a commutative permutation is constructed according to the position index set and the candidate index set to obtain the sub-permutation; Based on the replication secret sharing mechanism, the share splitting process is performed according to the sparse gradient representation to obtain the gradient sharing share received by the server.

9. A high-efficiency communication privacy-preserving federated learning device, characterized in that, The high-efficiency communication privacy-preserving federated learning device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images