A method and system for federated learning multi-party gradient aggregation in the coding domain
By using pre-trained encoders and decoders to process gradient information in federated learning, the problems of computational and communication burdens are solved, achieving efficient gradient aggregation and privacy protection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PENG CHENG LAB
- Filing Date
- 2022-08-19
- Publication Date
- 2026-05-12
AI Technical Summary
The existing federated learning process has high computational and communication costs, and existing cryptographic protection schemes such as multi-party secure computation and homomorphic encryption technology lead to a decline in system performance.
A multi-directional gradient encoding domain aggregation method is adopted, which transforms the local dataset into quantized gradients through pre-trained encoders and decoders. The server aggregates the encoded gradients and updates the model, reducing the communication and computation burden.
This reduces the computational and communication burden while protecting gradient information from leakage, thus enabling an efficient federated learning process.
Smart Images
Figure CN115422562B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning technology, and in particular to a method and system for encoding domain aggregation of federated learning multi-directional gradients. Background Technology
[0002] Federated learning is a novel distributed deep learning paradigm that unites data holders across a wide area network to collaboratively train a deep learning model. Simultaneously, the privacy data of each data holder remains local, thus protecting the privacy of all participants. Existing research indicates that directly transmitting model parameters or gradient information still leads to privacy data leakage, thus requiring security technologies to protect gradient information during the federated learning process. Existing cryptographic protection schemes, such as multi-party secure computation (MPC) and homomorphic encryption (HE), impose significant computational and communication burdens on federated learning systems. Homomorphic encryption, in particular, enables specific operations to be performed on data within a ciphertext space, guaranteeing that the decrypted result is equal to the direct result of the original data. This process (taking additive homomorphism as an example) can be described as: E(x+y) = E(x) + E(y), and x+y = D[E(x) + E(y)]. In practical applications, this process introduces additional computational costs and communication burdens. Existing practical studies have shown that homomorphic encryption can increase the iteration time of a federated learning system by 135 times and increase the communication volume from 85.89MB to 13.1GB.
[0003] Therefore, existing technologies have shortcomings and need to be improved and developed. Summary of the Invention
[0004] The technical problem to be solved by this invention is to provide a method and system for encoding domain aggregation of multi-party gradients in federated learning, which addresses the above-mentioned deficiencies of the prior art and aims to solve the problem of high computational cost and communication burden introduced by the federated learning process in the prior art.
[0005] The technical solution adopted by this invention to solve the technical problem is as follows:
[0006] A federated learning method for encoding domain aggregation using multi-directional gradients, comprising:
[0007] Multiple federated participating terminals each convert their local datasets into quantized gradients, and input the quantized gradients into a pre-trained encoder to obtain the encoded gradients;
[0008] The server obtains the encoding gradient corresponding to each federated participating terminal, aggregates all encoding gradients to obtain the aggregation result, and inputs the aggregation result into the pre-trained decoder to obtain the decoding gradient;
[0009] The server updates the federated learning model based on the decoding gradient and sends the updated federated learning model to each federated participating terminal.
[0010] In one implementation, the plurality of federated participating terminals each convert their respective local datasets into quantized gradients, and input the quantized gradients into a pre-trained encoder to obtain encoded gradients, including:
[0011] Multiple federated participating terminals use their respective local datasets to compute the original gradient based on a neural network;
[0012] The original gradient is quantized to obtain the quantized gradient, which is then mapped to the integer field.
[0013] The quantized gradient is input into the pre-trained encoder to obtain the encoded gradient.
[0014] In one implementation, the pre-training steps of the pre-trained encoder and pre-trained decoder include:
[0015] The server initializes the encoder, decoder, and quantization parameters, and randomly generates a training dataset based on the quantization parameters.
[0016] Vectors are sampled from the training dataset, and the encoder and decoder are trained and updated based on a preset optimization objective.
[0017] When the encoder and decoder reach the preset optimization target and the encoding noise satisfies the standard Gaussian distribution, the pre-trained encoder and pre-trained decoder are output.
[0018] In one implementation, the preset optimization objective is:
[0019] Wherein, the D c For the decoder function, the E c Let m be the encoder function, l be the number of participating terminals in a federated round, l be the data samples in the training dataset, and L be the loss function.
[0020] In one implementation, the encoded noise is
[0021] Wherein, the D c For the decoder function, the E c For encoder function, m is the number of participating terminals in a federated round, and l is the data sample in the training dataset.
[0022] In one implementation, before the plurality of federated participating terminals convert their respective local datasets into quantized gradients and input the quantized gradients into a pre-trained encoder to obtain the encoded gradients, the method further includes:
[0023] The server sends the pre-trained encoder to multiple federated participating terminals.
[0024] In one implementation, the server initializes the encoder, decoder, and quantization parameters, and randomly generates a training dataset based on the quantization parameters, including:
[0025] The server defines the encoder, decoder, quantization space size, and input vector size;
[0026] A training dataset is randomly generated based on the quantization space size and the input vector size, wherein each data sample in the training dataset is an integer vector.
[0027] In one implementation, sampling vectors from the training dataset and training and updating the encoder and decoder based on a preset optimization objective includes:
[0028] Select m data samples from the training dataset and train the encoder and decoder based on a preset optimization objective;
[0029] The loss function is calculated based on the training results, the gradient is calculated by back-deriving the loss function, and the encoder and decoder are updated based on the gradient.
[0030] In one implementation, after sampling vectors from the training dataset and training and updating the encoder and decoder based on a preset optimization objective, the method further includes:
[0031] When the encoder and decoder fail to achieve the preset optimization target, and / or the encoded noise does not satisfy the standard Gaussian distribution, m data samples are selected from the training dataset, and the encoder and decoder are trained and updated based on the preset optimization target.
[0032] The present invention also discloses a federated learning multi-party gradient coding domain aggregation system, the federated learning multi-party gradient coding domain aggregation system comprising: multiple federated participating terminals and a server, wherein the multiple federated participating terminals are all connected to the server via a network;
[0033] The federated participating terminals are used to convert their respective local datasets into quantized gradients and input the quantized gradients into a pre-trained encoder to obtain encoded gradients.
[0034] The server is used to obtain the encoding gradient corresponding to each federated participating terminal, aggregate all encoding gradients to obtain the aggregation result, input the aggregation result into the pre-trained decoder to obtain the decoding gradient, update the federated learning model according to the decoding gradient, and send the updated federated learning model to each federated participating terminal.
[0035] The present invention also discloses a computer-readable storage medium storing a computer program that can be executed to implement the steps of the federated learning multi-directional gradient encoding domain aggregation method as described above.
[0036] This invention provides a method and system for encoding domain aggregation of multi-party gradients in federated learning. The method includes: multiple federated participating terminals converting their respective local datasets into quantized gradients and inputting the quantized gradients into a pre-trained encoder to obtain encoded gradients; a server acquiring the encoded gradients corresponding to each federated participating terminal, aggregating all encoded gradients to obtain an aggregation result, and inputting the aggregation result into a pre-trained decoder to obtain decoded gradients; the server updating the federated learning model based on the decoded gradients and sending the updated federated learning model to each federated participating terminal. This invention utilizes a pre-trained encoder across multiple federated participating terminals, allowing each terminal to calculate its own encoded gradient. The server then directly sums all encoded gradients, enabling communication to focus solely on the encoded gradients. The server directly sums the encoded gradients and aggregates the sum using the pre-trained decoder, avoiding additional communication costs and consuming minimal computational resources, thus reducing computational cost and communication burden. Attached Figure Description
[0037] Figure 1 This is a flowchart of a preferred embodiment of the federated learning multi-gradient coding domain aggregation method in this invention.
[0038] Figure 2 This is a flowchart of step S100 in a preferred embodiment of the federated learning multi-gradient coding domain aggregation method of the present invention.
[0039] Figure 3 This is a flowchart of the pre-training steps of the pre-trained encoder and pre-trained decoder in a preferred embodiment of the federated learning multi-gradient coding domain aggregation method of the present invention.
[0040] Figure 4 This is a flowchart of step S10 in a preferred embodiment of the federated learning multi-gradient coding domain aggregation method of the present invention.
[0041] Figure 5This is a flowchart of step S20 in a preferred embodiment of the federated learning multi-gradient coding domain aggregation method of the present invention.
[0042] Figure 6 This is a flowchart of the autoencoder pre-training process in a preferred embodiment of the federated learning multi-gradient encoding domain aggregation method of the present invention.
[0043] Figure 7 This is a functional principle diagram of a preferred embodiment of the federated learning multi-gradient coding domain aggregation system in this invention. Detailed Implementation
[0044] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0045] This invention reduces the burden that security and privacy protection methods place on federated learning systems, while protecting gradient content during the federated learning process from affecting gradient aggregation. This invention does not introduce additional communication costs and consumes only minimal computational resources. Based on theoretical proof, it protects gradients from being stolen by network attackers and malicious servers, ensuring the effectiveness of federated learning.
[0046] Please see Figure 1 , Figure 1 This is a flowchart of the encoding domain aggregation method for federated learning multi-directional gradients in this invention. For example... Figure 1 As shown in the embodiment of the present invention, the federated learning multi-directional gradient encoding domain aggregation method includes the following steps:
[0047] In step S100, multiple federated participating terminals convert their respective local datasets into quantized gradients and input the quantized gradients into a pre-trained encoder to obtain the encoded gradients.
[0048] Specifically, when a user's terminal and a server form a federated system, the user's terminal is a federated participating terminal.
[0049] For further implementation details, please refer to [link / reference]. Figure 2 Step S100 specifically includes:
[0050] Step S110: Multiple federated participating terminals use their respective local datasets to calculate the original gradient based on the neural network;
[0051] Step S120: Quantize the original gradient to obtain the quantized gradient and map it to the integer field;
[0052] Step S130: Input the quantized gradient into the pre-trained encoder to obtain the encoded gradient.
[0053] Specifically, multiple federated participating terminals use local privacy datasets to compute gradients based on neural networks, which are the original gradients; the original gradients are quantized and mapped to the integer domain; the quantized gradients can be encoded into the encoding space by a pre-trained encoder to obtain the corresponding encoded gradients, which are then transmitted to the server via a computer network.
[0054] Following step S100 is step S200: The server obtains the encoding gradient corresponding to each federated participating terminal, aggregates all encoding gradients to obtain the aggregation result, and inputs the aggregation result into the pre-trained decoder to obtain the decoding gradient.
[0055] Specifically, the server acquires all encoded gradients and aggregates them directly; then, the aggregated result within the encoded domain is input into the pre-trained decoder to calculate the decoded gradient. In other words, after pre-training, the encoder of this invention is used by multiple federated participating terminals. These terminals calculate their respective encoded gradients, and the server directly sums all the encoded gradients. This means that communication only requires communicating the encoded gradients; the server directly adds the encoded results together and aggregates the sum using the pre-trained decoder. Therefore, this invention does not introduce additional communication costs and consumes only small computational resources, resulting in low computational cost and communication burden. Furthermore, although this invention introduces computational load (for encoded gradients) to the federated participating terminals, the pre-trained encoder of this invention can be very small (tens of MB), thus reducing the computational load.
[0056] Following step S200 is step S300: The server updates the federated learning model according to the decoding gradient and sends the updated federated learning model to each federated participating terminal.
[0057] Specifically, the decoded gradient can be used to update the global federated learning model, synchronize the updated federated learning model to all federated participating terminals, and start the next round of federated learning process.
[0058] In one implementation, the present invention also utilizes an autoencoder pre-training technique. An autoencoder is a commonly used technique in deep learning to learn data representations and can reconstruct the original information from the learned representations. For details, please refer to... Figure 3 The pre-training steps of the pre-trained encoder and pre-trained decoder in this invention include:
[0059] Step S10: The server initializes the encoder, decoder, and quantization parameters, and randomly generates a training dataset based on the quantization parameters;
[0060] Step S20: Sample vectors from the training dataset and train and update the encoder and the decoder based on a preset optimization target;
[0061] Step S30a: When the encoder and decoder reach the preset optimization target and the encoding noise satisfies the standard Gaussian distribution, output the pre-trained encoder and pre-trained decoder.
[0062] Because this invention uses a unified pre-trained autoencoder on the server, and multiple federated participating terminals use a unified encoder for encoding, the encoded gradients obtained by the server can be directly summed. Since the process of restoring the original information involves information loss, the information restored by the autoencoder often contains noise. Therefore, this invention trains a novel additive homomorphic encryption approximation autoencoder on the server, which includes pairs of encoders and decoders. This invention normalizes the autoencoder noise to Gaussian noise and further constructs a Gaussian difference privacy mechanism to protect the federated gradient information, thus protecting user data privacy.
[0063] Specifically, the preset optimization target is Wherein, the D c For the decoder function, the E c Let m be the encoder function, l be the number of participating terminals in a federated round, l be the data samples in the training dataset, and L be the loss function. The loss function L can be either l1-norm or l2-norm loss.
[0064] In actual training, due to encoding noise, the preset optimization target cannot be optimized to zero. When the error conforms to a Gaussian distribution, the encoding and decoding formula of this invention conforms to the differential privacy mechanism. That is to say, this invention only needs to train an autoencoder with Gaussian noise, and the error distribution of the optimized encoder conforms to a Gaussian distribution, thus achieving the final goal.
[0065] In one implementation, the encoded noise is When the encoding noise satisfies When the standard Gaussian distribution is used, the autoencoder of this invention forms a differential privacy method based on a Gaussian mechanism. By normalizing the noise distribution of the autoencoder, the original lossy noise is constructed into effective noise that satisfies the differential privacy method, thereby further improving privacy security. Here, I represents a unit vector, that is, the Gaussian distribution ν is a vector.
[0066] In one implementation, before step S100, the method further includes: the server sending the pre-trained encoder to multiple federated participating terminals. In application, the pre-trained autoencoder is split into a pre-trained encoder and a pre-trained decoder. The encoder is sent to the federated participating terminals to encode privacy gradients, while the decoder is retained on the federated server to decode user gradients aggregated in the encoding domain. That is, the federated participating terminals obtain the trained encoder from the server for local gradient encoding.
[0067] For further implementation details, please refer to [link / reference]. Figure 4 Step S10 specifically includes:
[0068] Step S11: The server defines the encoder, decoder, quantization space size, and input vector size;
[0069] Step S12: Randomly generate a training dataset based on the quantization space size and the input vector size, wherein each data sample in the training dataset is an integer vector.
[0070] The encoder receives an integer vector as input and outputs a corresponding representation vector; the decoder receives an input of the same size as the representation vector and outputs a decoding result of the same size as the integer vector. Specifically, an autoencoder network E is defined. c D c Let quantization space size be s and input vector size be b. Encoder network E c The decoder network D accepts an integer vector as input, where each element x in the vector has a value in the range [0, s], and outputs the corresponding representation vector. c The encoder accepts an input of the same size as the representation vector and outputs a decoded result of the same size as the integer vector. The decoded result is a floating-point vector, meaning the size of the encoder's output vector is the same as the decoder's input. A training dataset D is randomly generated, where each data sample l is an integer vector of length b. In other words, a dataset is sampled from the quantization space for training, ensuring that the samples satisfy the constraints of the quantization parameters b and s.
[0071] This invention employs numerical processing methods of quantization and dequantization. Based on given quantization parameters, any floating-point number can be mapped to a specific integer domain. The invention uses quantization to standardize the input distribution of the encoder, constraining the input and output of the encoder and decoder. This allows a simple, small perceptron network to quickly fit a preset target optimization task, thus enabling a smaller encoder-decoder network.
[0072] In one implementation, please refer to Figure 5 Step S20 specifically includes:
[0073] Step S21: Select m data samples from the training dataset and train the encoder and decoder based on the preset optimization objective;
[0074] Step S22: Calculate the loss function based on the training results, calculate the gradient by back-derivative of the loss function, and update the encoder and decoder based on the gradient.
[0075] Specifically, after randomly generating a training dataset D, a standard neural network training process is performed. Based on the generated random dataset D and the predefined autoencoder network structure, the autoencoder network E is trained end-to-end. c D c In this process, m random vectors are randomly selected from the training dataset D each time. The loss is calculated using a preset optimization objective. Based on the loss, the gradient is calculated by backpropagation, and the autoencoder network parameters are updated. This process is repeated until the autoencoder network E... c D c The training results met the requirements, and the errors followed a Gaussian distribution.
[0076] In a further implementation, step S20 is followed by: S30b, when the encoder and decoder fail to achieve the preset optimization objective, and / or the encoded noise does not satisfy a standard Gaussian distribution, m data samples are selected from the training dataset, and the encoder and decoder are trained and updated based on the preset optimization objective. Updating the autoencoder network involves minimizing the loss function L on the training dataset. Each network update evaluates the encoder-decoder performance. By training the encoder-decoder network, the encoding / decoding result approximates the original, unencoded result. In actual training, the training results can be used to generate a test dataset, and the best-performing result is selected.
[0077] This invention is an approximate method of additive homomorphic encryption that does not affect the performance of federated learning models. By standardizing the noise distribution of the autoencoder, the original lossy noise is constructed into effective noise that satisfies the differential privacy method, thereby further improving privacy and security. Furthermore, this invention utilizes a general construction algorithm that can adjust the scale of the encoder-decoder network according to the actual application scenario, thereby reducing computational complexity (increasing the corresponding noise scale to obtain higher security) or increasing computational complexity (obtaining higher accuracy). In practical applications, compared with the original horizontal federated learning, it does not generate additional communication burden.
[0078] The following are specific examples for illustration.
[0079] Example 1:
[0080] Please see Figure 6The autoencoder pre-training process is as follows:
[0081] Step A1: Initialize encoder network E C Decoder Network D C and the quantization parameter s;
[0082] Step A2: Randomly generate the training dataset D based on the quantization parameter s;
[0083] Step A3: Sample vectors from the training dataset D;
[0084] Step A4: Train and update the encoder network E based on the optimization objective. C and decoder network D C ;
[0085] Step A5: Evaluate the performance of the encoder network and decoder network;
[0086] Step A6: Determine whether the performance of the encoder and decoder meets the optimization target and whether the error follows a Gaussian distribution; if yes, proceed to step A7; if no, return to step A3.
[0087] Step A7: Output the pre-trained encoder network E C and decoder network D C .
[0088] Furthermore, based on the above-mentioned federated learning multi-party gradient coding domain aggregation method, the present invention also provides a federated learning multi-party gradient coding domain aggregation system, the federated learning multi-party gradient coding domain aggregation system comprising: multiple federated participating terminals and a server, wherein the multiple federated participating terminals are all connected to the server via a network;
[0089] The federated participating terminals are used to convert their respective local datasets into quantized gradients and input the quantized gradients into a pre-trained encoder to obtain encoded gradients.
[0090] The server is used to obtain the encoding gradient corresponding to each federated participating terminal, aggregate all encoding gradients to obtain the aggregation result, input the aggregation result into the pre-trained decoder to obtain the decoding gradient, update the federated learning model according to the decoding gradient, and send the updated federated learning model to each federated participating terminal.
[0091] like Figure 7As shown, participants a, b, and c represent federated learning terminals within the network, and the server is a federated server. The federated learning terminals corresponding to participants a, b, and c use their local privacy datasets to compute the original gradients based on the neural network. These original gradients are quantized and mapped to the integer domain. The quantized gradients are then encoded into the encoding space using a pre-trained encoder. The encoded result is the encoded gradient, which is transmitted to the federated server via the computer network. The federated server acquires all encoded gradients and aggregates them directly. The aggregated result in the encoding domain is then input into a pre-trained decoder to compute the decoded gradient. The decoded gradient, after dequantization, can be used to update the global federated learning model. This model is then synchronized to the federated learning terminals corresponding to all participants, and the next round of the federated learning process is initiated.
[0092] In one implementation, multiple federated participating terminals are further used to compute the original gradient based on the neural network using their respective local datasets; quantize the original gradient to obtain the quantized gradient, and map it to the integer domain; input the quantized gradient into a pre-trained encoder to obtain the encoded gradient.
[0093] In one implementation, the server is further configured to pre-train the pre-trained encoder and pre-trained decoder.
[0094] Specifically, the server initializes the encoder, decoder, and quantization parameters, and randomly generates a training dataset based on the quantization parameters;
[0095] Vectors are sampled from the training dataset, and the encoder and decoder are trained and updated based on a preset optimization objective.
[0096] When the encoder and decoder reach the preset optimization target and the encoding noise satisfies the standard Gaussian distribution, the pre-trained encoder and pre-trained decoder are output.
[0097] When the encoder and decoder fail to achieve the preset optimization target, and / or the encoded noise does not satisfy the standard Gaussian distribution, m data samples are selected from the training dataset, and the encoder and decoder are trained and updated based on the preset optimization target.
[0098] The preset optimization objective is: Wherein, the D c For the decoder function, the E c Let m be the encoder function, l be the number of participating terminals in a federated round, l be the data samples in the training dataset, and L be the loss function.
[0099] The encoded noise is Wherein, the D c For the decoder function, the Ec For encoder function, m is the number of participating terminals in a federated round, and l is the data sample in the training dataset.
[0100] When the encoding noise satisfies When the standard Gaussian distribution is used, the autoencoder forms a differential privacy method with a Gaussian mechanism. By normalizing the noise distribution of the autoencoder, the original lossy noise is constructed into effective noise that satisfies the differential privacy method, thereby further improving the privacy security.
[0101] In one implementation, the server is further configured to send the pre-trained encoder to multiple federated participating terminals.
[0102] Specifically, the server defines the encoder, decoder, quantization space size, and input vector size;
[0103] A training dataset is randomly generated based on the quantization space size and the input vector size, wherein each data sample in the training dataset is an integer vector.
[0104] Select m data samples from the training dataset and train the encoder and decoder based on a preset optimization objective;
[0105] The loss function is calculated based on the training results, the gradient is calculated by back-deriving the loss function, and the encoder and decoder are updated based on the gradient.
[0106] The present invention also discloses a computer-readable storage medium storing a computer program that can be executed to implement the steps of the federated learning multi-directional gradient encoding domain aggregation method as described above.
[0107] In summary, this invention discloses a method and system for encoding domain aggregation of multi-party gradients in federated learning. The method includes: multiple federated participating terminals converting their respective local datasets into quantized gradients and inputting the quantized gradients into a pre-trained encoder to obtain encoded gradients; a server acquiring the encoded gradients corresponding to each federated participating terminal, aggregating all encoded gradients to obtain an aggregation result, and inputting the aggregation result into a pre-trained decoder to obtain decoded gradients; the server updating the federated learning model based on the decoded gradients and sending the updated federated learning model to each federated participating terminal. This invention, by having multiple federated participating terminals use a pre-trained encoder, with each terminal calculating its own encoded gradient, and the server directly summing all encoded gradients, ensures that communication only requires communicating the encoded gradients. The server directly sums the encoded results and aggregates the sum using the pre-trained decoder, avoiding additional communication costs and consuming only small computational resources, thus reducing computational cost and communication burden.
[0108] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.
Claims
1. A federated learning method for encoding domain aggregation using multi-faceted gradients, characterized in that, include: Multiple federated participating terminals each convert their local datasets into quantized gradients, and input the quantized gradients into a pre-trained encoder to obtain the encoded gradients; The server obtains the encoding gradient corresponding to each federated participating terminal, aggregates all encoding gradients to obtain the aggregation result, and inputs the aggregation result into the pre-trained decoder to obtain the decoding gradient; The server updates the federated learning model based on the decoding gradient and sends the updated federated learning model to each federated participating terminal. The pre-training steps for the pre-trained encoder and pre-trained decoder include: The server initializes the encoder, decoder, and quantization parameters, and randomly generates a training dataset based on the quantization parameters. Vectors are sampled from the training dataset, and the encoder and decoder are trained and updated based on a preset optimization objective. When the encoder and decoder reach the preset optimization target and the encoding noise satisfies the standard Gaussian distribution, the pre-trained encoder and pre-trained decoder are output. The preset optimization objective is: ; Among them, the For decoder functions, the For encoder functions, the The number of federal participating terminals in a federal round, the These are data samples from the training dataset. It is a loss function; The encoded noise is ; The multiple federated participating terminals convert their respective local datasets into quantized gradients and input the quantized gradients into the pre-trained encoder to obtain the encoded gradients. Before this process, the process further includes: The server sends the pre-trained encoder to multiple federated participating terminals; The server initializes the encoder, decoder, and quantization parameters, and randomly generates a training dataset based on the quantization parameters, including: The server defines the encoder, decoder, quantization space size, and input vector size; A training dataset is randomly generated based on the quantization space size and the input vector size, wherein each data sample in the training dataset is an integer vector. When multiple federated terminals communicate with the server, they only need to communicate the encoded gradients. The server uniformly pre-trains pairs of pre-trained encoders and pre-trained decoders. Multiple federated terminals use the unified pre-trained encoders for encoding, and the server uses the pre-trained decoders to directly sum the encoded gradients.
2. The federated learning multi-directional gradient encoding domain aggregation method according to claim 1, characterized in that, The multiple federated participating terminals each convert their respective local datasets into quantized gradients, and input the quantized gradients into a pre-trained encoder to obtain encoded gradients, including: Multiple federated participating terminals use their respective local datasets to compute the original gradient based on a neural network; The original gradient is quantized to obtain the quantized gradient, which is then mapped to the integer field. The quantized gradient is input into the pre-trained encoder to obtain the encoded gradient.
3. The federated learning multi-directional gradient encoding domain aggregation method according to claim 1, characterized in that, Sampling vectors from the training dataset and training and updating the encoder and decoder based on a preset optimization objective includes: Select m data samples from the training dataset and train the encoder and decoder based on a preset optimization objective; The loss function is calculated based on the training results, the gradient is calculated by back-deriving the loss function, and the encoder and decoder are updated based on the gradient.
4. The federated learning multi-directional gradient encoding domain aggregation method according to claim 1, characterized in that, After sampling vectors from the training dataset and training and updating the encoder and decoder based on a preset optimization objective, the method further includes: When the encoder and decoder fail to achieve the preset optimization target, and / or the encoded noise does not satisfy the standard Gaussian distribution, m data samples are selected from the training dataset, and the encoder and decoder are trained and updated based on the preset optimization target.
5. A federated learning multi-gradient encoding domain aggregation system, characterized in that, The federated learning multi-party gradient coding domain aggregation system includes: multiple federated participating terminals and a server, wherein the multiple federated participating terminals are all connected to the server via a network. The federated participating terminals are used to convert their respective local datasets into quantized gradients and input the quantized gradients into a pre-trained encoder to obtain encoded gradients. The server is used to obtain the encoding gradient corresponding to each federated participating terminal, aggregate all encoding gradients to obtain an aggregation result, input the aggregation result into a pre-trained decoder to obtain a decoding gradient, update the federated learning model according to the decoding gradient, and send the updated federated learning model to each federated participating terminal. The pre-training steps for the pre-trained encoder and pre-trained decoder include: The server initializes the encoder, decoder, and quantization parameters, and randomly generates a training dataset based on the quantization parameters. Vectors are sampled from the training dataset, and the encoder and decoder are trained and updated based on a preset optimization objective. When the encoder and decoder reach the preset optimization target and the encoding noise satisfies the standard Gaussian distribution, the pre-trained encoder and pre-trained decoder are output. The preset optimization objective is: ; Among them, the For decoder functions, the For encoder functions, the The number of federal participating terminals in a federal round, the These are data samples from the training dataset. It is a loss function; The encoded noise is ; The server sends the pre-trained encoder to multiple federated participating terminals; The server initializes the encoder, decoder, and quantization parameters, and randomly generates a training dataset based on the quantization parameters, including: The server defines the encoder, decoder, quantization space size, and input vector size; A training dataset is randomly generated based on the quantization space size and the input vector size, wherein each data sample in the training dataset is an integer vector. When multiple federated terminals communicate with the server, they only need to communicate the encoded gradients. The server uniformly pre-trains pairs of pre-trained encoders and pre-trained decoders. Multiple federated terminals use the unified pre-trained encoders for encoding, and the server uses the pre-trained decoders to directly sum the encoded gradients.
6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed to implement the steps of the federated learning multi-directional gradient encoding domain aggregation method as described in any one of claims 1 to 4.