A residual federated learning method and system based on controllable aggregation weight

Abstract

The application provides a residual federal learning method and system based on controllable aggregation weight, and the steps comprise: a server sends global model parameters to a client, the client performs iterative training and generates a local prediction model; the client calculates the parameter residual of the local training model and the local prediction model and uploads the parameter residual to the server; the server takes the proportion of the data volume of the client as an initial value, takes minimizing a loss function as a target, combines the KL divergence between the virtual data distribution of the client and the proxy data distribution, iteratively learns an optimal aggregation weight; the client is weighted and aggregated according to the optimal aggregation weight, a new global model is generated and iterative training is continued until convergence. The application reduces communication overhead through residual compression and position coding mechanism, and alleviates the influence of client data heterogeneity on model performance through controllable aggregation weight, thereby improving the communication efficiency, model accuracy and generalization ability of federal learning in the network traffic classification detection scene.

Description

Technical Field

[0001] This invention relates to the field of federated learning, specifically to a residual federated learning method and system based on adjustable aggregation weights. Background Technology

[0002] Federated learning, as a privacy-preserving distributed machine learning framework, allows multiple clients to collaboratively train a global model without sharing local data, providing a new approach for network traffic data security detection in distributed nodes of the Industrial Internet. However, with the increasing number of clients participating in federated learning and the growing complexity of client data environments, traditional federated learning faces two major challenges in practical deployment: first, increased communication overhead, as the increasing number of clients and model parameters brings huge communication costs; and second, heterogeneity of client data, including non-independent and identically distributed data distribution, uneven device computing power, and unstable network conditions, which seriously affect the convergence speed and performance of the model.

[0003] To reduce the communication overhead between clients and servers in federated learning, engineers are attempting to improve federated learning schemes through model compression, model gradient quantization, and sparse communication. However, these schemes focus on model parameter compression techniques and lack effectiveness in addressing issues such as heterogeneous client data and inconsistent model update directions. In recent years, residual federated learning has been proposed as an emerging, highly efficient communication paradigm. Its core idea is to transmit only the differences between the client's locally trained model and the prediction model, thereby improving global model performance while ensuring communication efficiency. However, existing residual federated learning methods often use fixed weights to aggregate client models, neglecting the differences in data heterogeneity and training capabilities among different clients. This leads to low global learning efficiency, and the models are particularly prone to getting trapped in local optima in highly heterogeneous environments, affecting model convergence and generalization capabilities. Summary of the Invention

[0004] To address the problems of fixed aggregation weights, inability to adapt to client differences, high communication overhead, and difficulty in adapting to large-scale federated learning in existing residual federated learning methods, this invention proposes a residual federated learning method and system based on adjustable aggregation weights. On one hand, the client generates a prediction model based on local and global model trajectories, uploading the parameter difference between the client's training model and the prediction model as the residual. By incorporating sparsity reduction and position indicator vector encoding, the amount of communication data per round is significantly compressed, effectively reducing the communication overhead between the client and server while maintaining model accuracy. On the other hand, to address the differences in data distribution and model training quality among different clients during residual aggregation, adjustable aggregation weights are introduced. The aggregation strategy is dynamically optimized on the proxy dataset, thereby improving the convergence speed and generalization ability of the global model. The technical solution provided by this invention is as follows:

[0005] Firstly, a residual federated learning method based on adjustable aggregation weights includes the following steps:

[0006] Step S1: The server initializes the global model parameters and global model trajectory, and sends the global model parameters to all associated clients; each client receives the global model and initializes its local training model trajectory.

[0007] Step S2: The client uses the local dataset to perform local iterative training on the global model, generating a client-side local training model; the client generates a local prediction model based on its local training model trajectory and the locally recorded global model trajectory; the client calculates the difference between the parameters of the local training model and the local prediction model, generating the local model residual; the client uses an improved Top-K algorithm to sparsify and compress the local model residual, and introduces position indicator vectors to mark the positions of non-zero model parameters; the non-zero model parameters, position indicator vectors, and local data are packaged together and uploaded to the server.

[0008] Step S3: The server receives the model residuals uploaded by the client, and reconstructs the local training model parameters of each client by combining them with the global model trajectory; based on the amount of local data uploaded by each client, the server calculates the proportion of each client as the initial value of the client model aggregation weight; the server iteratively optimizes the model aggregation weight on the server proxy dataset to obtain the optimal aggregation weight; the server performs weighted aggregation on the reconstructed client training models according to the optimal model aggregation weight to generate the global model for the current global training round.

[0009] Step S4: The server checks whether the current global training rounds have reached the specified number. If so, federated learning is stopped, and the server sends the final global model parameters to all participating clients and notifies the clients that the global model training has ended. After receiving the global training end message, the client can use the global model to perform real-time monitoring and classification of local network traffic. If not, steps S2 and S3 are repeated to continue training until the required number of training rounds is reached and the global model converges.

[0010] Preferably, the global model trajectory is the set of global model parameters from the most recent M training epochs; the client-side local training model trajectory refers to the set of local training model parameters from the most recent M training epochs.

[0011] Preferably, in step 2, the client uses a linear prediction method to generate a local prediction model, and the local prediction model generated by client i in the t-th round of global training... for:

[0012]

[0013] in, This represents the local model parameters generated by client i during the (t-1)th round of global training; and These represent the global model parameters formed by server aggregation during the (t-1)th and (t-2)th rounds of global training, respectively; This is a hyperparameter used to control the degree to which the local prediction model references the direction of global model updates.

[0014] Preferably, the specific process of iterative optimization of model aggregation weights in step S3 is as follows:

[0015] Step S31: Use proxy data samples to perform multi-class classification tests on the given aggregation model, obtain the predicted classification output of the samples, and calculate the cross-entropy loss under the current aggregation weight.

[0016] Step S32: Generate a virtual data distribution for the client based on the data distribution uploaded by the client and the current model aggregation weights, and calculate the KL divergence between the virtual data distribution and the actual sample distribution in the proxy dataset to assess the degree of difference between the two.

[0017] Step S33: Calculate the current loss function by combining cross-entropy loss and KL divergence, and update the client model aggregate weights using the Adam optimizer;

[0018] Step S34: Repeat steps S31 to S33 until a fixed number of iterations is reached. The Adam optimizer stops the optimization process of the model aggregation weights and outputs the model aggregation weights in the last iteration of the Adam optimizer, which is the optimal model aggregation weights for the current global training round.

[0019] Preferably, the formula for calculating the cross-entropy loss in step S31 is:

[0020]

[0021] in, This indicates that the aggregation model is in the proxy dataset. Cross-entropy on Represents the proxy dataset The number of samples; Indicates the total number of sample categories; This represents the one-hot encoding of the k-th proxy data sample; This represents the probability that the aggregate model predicts the k-th proxy data sample belongs to category c.

[0022] Preferably, the formula for calculating the KL divergence in step S32 is:

[0023]

[0024] in, The KL divergence between the virtual data distribution and the training sample distribution is the relative entropy. This indicates the proportion of class c samples in the server proxy dataset; This indicates the virtual data distribution generated by the current model's aggregated weights for the client.

[0025] Secondly, a residual federated learning system based on adjustable aggregation weights includes the following modules:

[0026] Client module: Executes the operations of step S2 in sequence, including classifier model training and predictor, generating model residuals, generating prediction models, and compressing model residuals respectively; each client maintains a cache queue of local model trajectories and global model trajectories, uses the predictor to generate prediction models, calculates residuals, performs sparsity compression, and then uploads them to the server.

[0027] Server module: Performs the operation in step S3, decompresses and reconstructs the model from the compressed model residuals uploaded by the client, learns the optimal aggregation weights using the proxy dataset, then aggregates the model and tests it; the server also maintains the trajectory information of all clients to ensure the consistency of the predictor.

[0028] Preferably, both the client module and the server module employ a convolutional neural network model, comprising multiple convolutional layers and a fully connected layer; wherein the convolutional layers are used to extract features from the dataset, and the fully connected layer is used to connect the features and output the model's prediction results.

[0029] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0030] This invention utilizes a residual communication mechanism to transmit only the difference between model updates and predicted values. Combined with Top-K sparsity and position indicator vectors, it significantly reduces the amount of data per round of communication. Compared to traditional federated learning, communication overhead is drastically reduced. Because the residuals themselves are small in value and have high information density, more effective information can be retained at the same compression ratio, thus achieving higher communication efficiency while maintaining model accuracy. Adjustable aggregation weights are introduced to dynamically optimize aggregation parameters on the proxy dataset. Unlike traditional methods that use fixed aggregation based on data volume ratios, this invention adaptively allocates aggregation weights based on the performance of different clients in reconstructing the model in the current training round. Clients with high training quality and strong residual reliability are given greater weights, while the impact of clients with unstable training or severely deviated data distributions is reduced. This effectively mitigates the negative impacts of client data heterogeneity and training instability, improving the convergence speed and final accuracy of the global model.

[0031] This invention boasts strong compatibility and scalability, applicable to various deep learning model architectures (such as LeNet-5, ResNet-18, CNN, etc.). It requires no modification to the model's internal structure, only changes to the specific model architecture, making it easy to deploy within existing federated learning systems. Furthermore, this method is compatible with existing federated optimization algorithms (such as FedProx, FedAvg, etc.), exhibiting excellent scalability. The selection of proxy datasets is flexible, allowing the use of publicly available datasets unrelated to the original task, thus preserving data privacy.

[0032] This invention achieves higher testing accuracy and superior convergence performance within the same number of communication rounds by accurately characterizing the client's contribution through adjustable aggregate weights and by effectively reconstructing residual information using coefficients. Theoretical analysis shows that the introduction of adjustable aggregate weights can accelerate model convergence and improve model accuracy. Attached Figure Description

[0033] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0034] Figure 1 This is a flowchart of the federated learning method with adjustable aggregation weights according to the present invention;

[0035] Figure 2 This is a schematic diagram of the simulation results of the present invention under the case of independent and identical distribution;

[0036] Figure 3 This is a schematic diagram of the simulation results of the present invention under the condition of non-independent and identically distributed components. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] To make the above-mentioned objectives, features and effects of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] Example 1: A residual federated learning method based on adjustable aggregation weights, the overall implementation process is as follows: Figure 1As shown. In this embodiment, the federated learning system includes a central server and N clients. Each client has a private local dataset, and the participating clients jointly train the global model. To address the problems of heterogeneous client data and high communication overhead, this invention introduces a shared predictor on both the client and server sides, achieving efficient federated learning through residual communication and adjustable aggregation weights. The specific implementation steps are as follows:

[0040] Step S1: Before federated learning begins, the central server performs initial configuration with N clients. The total number of communication rounds is set. Sliding time window parameters The server initializes the global model parameters, which can be randomly initialized or a pre-trained model can be used. The server and each client use the same predictor architecture; this embodiment uses a linear predictor, which is computationally simple and has a small memory footprint. The server and each client initialize their local model trajectories. and global model trajectory The client and server establish a communication channel through a secure encryption protocol to ensure the privacy and security of data transmission.

[0041] Step S2: The client performs local model training. At the start of the t-th round of communication, the server first selects a subset of clients to participate in this round of training based on the data distribution characteristics or model performance uploaded by the clients in the previous round. In this embodiment, for simplicity, it is assumed that all N clients participate in each round of training. The server will use the global model from the previous round... The model is distributed to all selected clients as the iterative model for each client's current training round; after local training is completed, the locally trained model generated by client i will be updated to... .

[0042] Client i generates a local prediction model based on its locally trained model trajectory and the globally recorded model trajectory. The local prediction model generated by client i in the t-th round of global training. for:

[0043]

[0044] in, This represents the local model parameters generated by client i during the (t-1)th round of global training; and These represent the global model parameters formed by server aggregation during the (t-1)th and (t-2)th rounds of global training, respectively; These are hyperparameters used to control the degree to which the local prediction model references the global model's update direction. The difference between the parameters of the locally trained model and the prediction model is calculated, i.e., the generated model residual. .

[0045] Finally, client i uses an improved Top-K algorithm to sparsify and compress the local model residuals, introducing a position indicator vector; non-zero model parameters The decimal number corresponding to the position indicator vector The data is packaged together with the local data and uploaded to the server. Specifically:

[0046] Check local model residuals If it is a matrix, stack it row by row in sequence to form the model residual vector. ,in This is a line-by-line stack function;

[0047] Calculate the residual vector The absolute values ​​of each element are used to select the top K model parameters with the largest absolute values ​​to form the compressed model residuals. Set the remaining model parameters to zero and discard them.

[0048] In the t-th round of global training, the compressed model residual of client i is... It can be represented as:

[0049]

[0050] in, It is a sparsity compression function, which retains the K elements with the largest absolute values ​​in the vector to form a vector.

[0051] Model parameter position indicator vector Used to indicate the residual of the compression model The parameters in the original model residual vector The position in the middle is represented as:

[0052]

[0053] in, Represents the model residual vector Is the j-th element included in the compressed residual? In the middle, if ,otherwise M represents the model residuals. The total number of elements in the vector. Clearly, the model parameter position indicator vector... Only K elements in the model are 1, and the position of the k-th 1 is the residual of the compressed model. The k-th parameter in the original model residual vector The corresponding position in the model. By compressing the model residuals, the amount of information exchanged between the client and the server is greatly reduced, thereby lowering the communication overhead of federated learning.

[0054] Step S3: The server receives the model residuals uploaded by the client, and reconstructs the local training model parameters of each client by combining them with the global model history trajectory. Based on the local data volume information uploaded by each client, the data volume ratio of each client is calculated. Considering the heterogeneity of client data, the data volume occupied by each client is a percentage of the total dataset. , as the initial aggregation weight.

[0055] Using the Adam optimizer on the server proxy dataset The model aggregation weights are iteratively optimized to obtain a set of optimal aggregation weights. The iterative optimization of the model aggregation weights starts with the initial aggregation model, and the specific optimization process includes:

[0056] Using surrogate data samples, a multi-class classification test is performed on the given aggregation model to obtain the predicted classification output of the samples. The cross-entropy loss under the current aggregation weights is then calculated.

[0057]

[0058] in, This indicates that the aggregation model is in the proxy dataset. Cross-entropy on Represents the proxy dataset The number of samples; Indicates the total number of sample categories; This represents the one-hot encoding of the k-th proxy data sample; This represents the probability that the aggregate model predicts the k-th proxy data sample belongs to category c.

[0059] Based on the distribution of data uploaded by the client The current model aggregates weights to generate the client's virtual data distribution. Calculate the distribution of virtual data and proxy datasets KL divergence of the actual sample distribution:

[0060]

[0061] in, The KL divergence between the virtual data distribution and the training sample distribution is the relative entropy. This indicates the proportion of class c samples in the server proxy dataset.

[0062] The current loss function is calculated by combining cross-entropy loss and KL divergence, and the Adam optimizer is used to update the aggregate weights of the client model:

[0063]

[0064] Repeat the above steps until a fixed number of iterations is reached, set to 20. The Adam optimizer then stops optimizing the model aggregation weights, and the model aggregation weights from the last iteration of the Adam optimizer are the optimal model aggregation weights for the current global training epoch. The reconstructed client-trained model is weighted and aggregated based on the optimal model aggregation weights to generate the global model for the current global training round.

[0065]

[0066] Step S4: Local model training and residual generation. The server checks whether the current global training rounds have reached the specified number. If so, federated learning is stopped, and the server sends the final global model parameters to all participating clients, notifying them that global model training has ended. After receiving the global training end message, the client can use the global model for real-time monitoring and classification of local network traffic. If not, steps S2-S4 are repeated to continue training until the required number of training rounds is reached and the global model converges.

[0067] In this embodiment, both the client and server use a classifier model based on a convolutional neural network (CNN). The model structure includes:

[0068] Input layer: Considering CIFAR-10 as an image dataset, the first convolutional layer has 3 input channels, which is equal to the number of channels in the input image;

[0069] Multi-layer convolutional layers: The first and second layers use 32 convolutional kernels, the third and fourth layers use 64 convolutional kernels, and the fifth and sixth layers use 128 convolutional kernels. All convolutional kernels are 3×3 in size. Each convolutional layer is followed by a ReLU activation function and a max pooling layer to gradually reduce the feature map space size.

[0070] Fully connected layer: Feature integration is achieved through the connected layer after the output of the convolutional layer. Each fully connected layer contains 512 neurons and uses ReLU activation. The final output layer contains 10 neurons and uses Softmax activation, which corresponds to the number of categories in the classification task.

[0071] This embodiment uses the CIFAR-10 dataset, which contains 50,000 training samples and 10,000 test samples across 10 categories. Each sample is a 32×32 color image. To simulate the data heterogeneity in a federated learning scenario, the training data is processed as follows:

[0072] The private dataset is divided into clients using a round-robin approach: there are 10 clients. Considering that the CIFAR-10 dataset has 10 categories, each client is assigned several different categories of training data to ensure that the category combinations of each client are different, thereby simulating the diversity and non-independent and identically distributed characteristics of client data distribution in real-world scenarios.

[0073] Training parameter settings: Total number of communication rounds T=300; Learning rate: Adam optimizer, fixed learning rate set to 0.001; Local training rounds: 10 epochs of local training on the client within each communication round; Proxy training rounds: 20 epochs of training on the proxy dataset within each communication round; Time sliding window: Considering memory saving on both the client and server sides, the time sliding window M=2; Data compression ratio: 10% of model parameters are retained; Client samples: Under the case of independent and identically distributed (IOD) data, each client is assigned training data for 10 classes; under the case of non-independent and identically distributed (IOD) data, each client is assigned training data for 2 classes.

[0074] Using accuracy as the evaluation metric, the server model's accuracy in identifying each category is calculated after each round of training on the test dataset. This invention is compared with the ResFed scheme (the ResFed scheme uses a traditional federated average algorithm for aggregation, and except for the training part of the proxy dataset, the other parts are consistent with this scheme).

[0075] Experimental results are as follows Figure 2 and Figure 3 As shown, experiments demonstrate that the method of this invention achieves high accuracy in multi-class classification scenarios. Compared with state-of-the-art methods, this method has significant advantages in global model aggregation quality and model training robustness. Furthermore, this invention, through adjustable aggregation weights in federated learning, dynamically optimizes the aggregation strategy on the proxy dataset, effectively mitigating the negative impact of heterogeneous client data, significantly improving the final accuracy of the global model, and exhibiting higher adaptability and practicality.

[0076] Example 2: A residual federated learning system based on adjustable aggregation weights, comprising the following modules:

[0077] Client module: Executes the operations in step S2 sequentially, including classifier model training and predictor generation, generating model residuals, generating prediction models, and compressing model residuals, respectively. Each client maintains a cache queue of local and global model trajectories, uses the predictor to generate prediction models, calculates residuals, performs sparsity compression, and then uploads them to the server.

[0078] Server module: Performs step S3, decompresses and reconstructs the compressed model residuals uploaded by the client, learns the optimal aggregation weights using the proxy dataset, then aggregates the model and tests it. The server also maintains trajectory information for all clients to ensure predictor consistency.

[0079] Example 3: The computer-readable storage medium of this example stores a computer program that, when executed by a processor, implements the steps of a residual federated learning method based on adjustable aggregate weights in Example 1.

[0080] The computer-readable storage medium in this embodiment can be an internal storage unit of the terminal, such as the terminal's hard disk or memory; the computer-readable storage medium in this embodiment can also be an external storage device of the terminal, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc. equipped on the terminal; furthermore, the computer-readable storage medium can include both the terminal's internal storage unit and external storage devices.

[0081] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0082] Example 4: The computer device of this example includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the residual federated learning method based on adjustable aggregate weights in Example 1.

[0083] In this embodiment, the processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The memory can include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0084] Those skilled in the art will clearly understand that each implementation can be achieved using software plus the necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A residual federated learning method based on adjustable aggregation weights, characterized in that, Includes the following steps: Step S1: The server initializes the global model parameters and global model trajectory, and sends the global model parameters to all associated clients; each client receives the global model and initializes its local training model trajectory. Step S2: The client uses the local dataset to perform local iterative training on the global model to generate a client-side local training model; the client generates a local prediction model based on its local training model trajectory and the locally recorded global model trajectory; the client calculates the difference between the parameters of the local training model and the local prediction model to generate the local model residual; the client uses an improved Top-K algorithm to sparsify and compress the local model residual, and introduces a position indicator vector to mark the position of non-zero model parameters. Pack the non-zero model parameters, location indicator vectors, and local data together and upload them to the server; Step S3: The server receives the model residuals uploaded by the client, and reconstructs the local training model parameters of each client by combining them with the global model trajectory; based on the amount of local data uploaded by each client, the server calculates the proportion of each client as the initial value of the client model aggregation weight; and iteratively optimizes the model aggregation weight on the server proxy dataset to obtain the optimal aggregation weight. The server performs weighted aggregation on the reconstructed client-trained model based on the optimal model aggregation weights to generate the global model for the current global training round. Step S4: The server checks whether the current global training rounds have reached the specified number of times; If yes, stop federated learning, the server sends the final global model parameters to all participating clients, and notifies the clients that the global model training is over. After receiving the global training over message, the clients can use the global model to perform real-time monitoring and classification of local network traffic. If no, repeat steps S2 and S3 to continue training until the training rounds are reached and the global model converges.

2. The residual federated learning method based on adjustable aggregation weights according to claim 1, characterized in that, The global model trajectory is the set of global model parameters from the most recent M training epochs; the client-side local training model trajectory refers to the set of local training model parameters from the most recent M training epochs.

3. The residual federated learning method based on adjustable aggregation weights according to claim 2, characterized in that, Step 2: The client uses a linear prediction method to generate a local prediction model. In the t-th round of global training, the local prediction model generated by client i... for: ； in, This represents the local model parameters generated by client i during the (t-1)th round of global training; and These represent the global model parameters formed by server aggregation during the (t-1)th and (t-2)th rounds of global training, respectively; This is a hyperparameter used to control the degree to which the local prediction model references the direction of global model updates.

4. The residual federated learning method based on adjustable aggregation weights according to claim 3, characterized in that, The specific process of iterative optimization of the model aggregation weights in step S3 is as follows: Step S31: Use proxy data samples to perform multi-class classification tests on the given aggregation model, obtain the predicted classification output of the samples, and calculate the cross-entropy loss under the current aggregation weight. Step S32: Generate a virtual data distribution for the client based on the data distribution uploaded by the client and the current model aggregation weights, and calculate the KL divergence between the virtual data distribution and the actual sample distribution in the proxy dataset to assess the degree of difference between the two. Step S33: Calculate the current loss function by combining cross-entropy loss and KL divergence, and update the client model aggregate weights using the Adam optimizer; Step S34: Repeat steps S31 to S33 until a fixed number of iterations is reached. The Adam optimizer stops the optimization process of the model aggregation weights and outputs the model aggregation weights in the last iteration of the Adam optimizer, which is the optimal model aggregation weights for the current global training round.

5. The residual federated learning method based on adjustable aggregation weights according to claim 4, characterized in that, The formula for calculating the cross-entropy loss in step S31 is: ； in, This indicates that the aggregation model is in the proxy dataset. Cross-entropy on Represents the proxy dataset The number of samples; Indicates the total number of sample categories; This represents the one-hot encoding of the k-th proxy data sample; This represents the probability that the aggregate model predicts the k-th proxy data sample belongs to category c.

6. The residual federated learning method based on adjustable aggregation weights according to claim 4, characterized in that, The formula for calculating the divergence of step S32KL is: ； in, The KL divergence between the virtual data distribution and the training sample distribution is the relative entropy. This indicates the proportion of class c samples in the server proxy dataset; This indicates the virtual data distribution generated by the current model's aggregated weights for the client.

7. A residual federated learning system based on adjustable aggregation weights, characterized in that, Implementing the steps of a residual federated learning method based on adjustable aggregation weights as described in any one of claims 1-6 includes the following modules: Client module: Executes the operations of step S2 in sequence, including classifier model training and predictor, generating model residuals, generating prediction models, and compressing model residuals respectively; each client maintains a cache queue of local model trajectories and global model trajectories, uses the predictor to generate prediction models, calculates residuals, performs sparsity compression, and then uploads them to the server. Server module: Performs the operation in step S3, decompresses and reconstructs the model from the compressed model residuals uploaded by the client, learns the optimal aggregation weights using the proxy dataset, then aggregates the model and tests it; the server also maintains the trajectory information of all clients to ensure the consistency of the predictor.

8. A residual federated learning system based on adjustable aggregation weights according to claim 7, characterized in that, Both the client module and the server module employ a convolutional neural network model, comprising multiple convolutional layers and one fully connected layer. The convolutional layers are used to extract features from the dataset, while the fully connected layer is used to connect the features and output the model's prediction results.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in a residual federated learning method based on adjustable aggregate weights as described in any one of claims 1-6.

10. A computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the residual federated learning method based on adjustable aggregate weights as described in any one of claims 1-6.

Images