A personalized federated learning method and system based on layer-by-layer multi-threshold sketch

Abstract

The application discloses a kind of personalized federated learning method and system based on layer-by-layer multi-threshold sketch, for the distributed architecture training of one server and multiple clients, first, server and multiple clients are initialized training before training starts, after training starts, the independent quantization threshold set of local model of client is distributed layer by layer, and compared with the layer-by-layer parameter projection vector of local model, the comparison result is uploaded to server for processing to obtain global consensus variable and send to client, and the local model parameter is updated by the global consensus variable, and the training learning is completed;The application solves the precision problem of traditional single threshold and global single compression rule, adaptively matches the heterogeneity structure and parameter distribution difference in different network layers of deep model, provides more fine quantization resolution, while improving the fidelity of personalized model representation, maintains the limit of extremely low bit two-way communication overhead.

Description

Technical Field

[0001] This invention relates to the field of federated learning communication compression technology, and specifically to a personalized federated learning method based on layer-by-layer multi-threshold sketches.

[0002] The present invention also relates to a personalized federated learning system based on layer-by-layer multi-threshold sketches. Background Technology

[0003] Personalized federated learning aims to learn personalized models for each participating client under conditions of data heterogeneity, while protecting data privacy through distributed training. However, in high-dimensional deep neural network models, the communication overhead caused by the repeated exchange of model parameters or updates between the client and the server has become a key bottleneck limiting its application in large-scale deployments and bandwidth-constrained scenarios such as the Internet of Things and edge computing systems.

[0004] Currently, to reduce communication overhead, extreme communication compression methods based on symbol aggregation, random one-bit quantization, and one-bit random sketching have been proposed. These methods achieve significant communication savings in both uplink and downlink communication by compressing model parameters or gradients into one bit or symbol information for transmission.

[0005] Current problems include: Existing symbolic or bit-based compression methods typically apply a uniform binary compression rule to all network layers of the entire model; while this layer-independent design simplifies communication and aggregation, it ignores an important characteristic of deep neural networks: the significant heterogeneity of parameter distributions across different layers; for example, shallow layer weights tend to be more concentrated, while deep layer weights may cover a wider numerical range, and the distributions of each layer evolve continuously during training; in this context, applying the same compression rule to all layers is inherently mismatched with the heterogeneous statistical characteristics between layers, resulting in significant information loss in the compressed representation and ultimately affecting the accuracy of personalized models; furthermore, for each network layer, a single threshold is often too coarse to fully characterize the parameter distribution features of that layer; when the distribution is wide or changes over time, a single threshold can only roughly bisect the distribution, while multiple thresholds can divide the sketched parameter space into multiple intervals, thus providing a broader coverage and a more refined representation of layer-by-layer parameter changes. Summary of the Invention

[0006] The purpose of this invention is to address the aforementioned problems by providing a personalized federated learning method and system based on layer-by-layer multi-threshold sketches. This invention employs an uplink and downlink compression mechanism that combines layer-by-layer dynamic multi-threshold quantization with (T+1)-ary sketch encoding. By guiding the personalized modeling of the local model through a consistency regularization term, it ensures the generalization performance of the model on non-independent and identically distributed data, while achieving extreme compression of communication traffic. This solves the accuracy problems of traditional single-threshold and global single compression rules.

[0007] The technical solution adopted in this invention is as follows: A personalized federated learning method based on layer-by-layer multi-threshold sketches, the method being used in a distributed architecture with one server and multiple clients, the method comprising: Before each communication round, the server generates a public random seed and shared model parameters and sends them to each client. Each client trains the shared model parameters based on its local data to obtain the initial local model corresponding to each client. After the communication round begins, each client obtains the local statistics after projecting the parameters of each network layer in the local model of the current round based on the local model of the current round and the common random seed, and sends them to the server; the server receives the local statistics of each client, aggregates them to obtain the server's global statistics, and sends them to the client. The client receives global statistics and generates a threshold set for each network layer in the local model in the current round. The client obtains the projection vector of the model parameters of the current network layer through a public random seed, compares the projection vector with the threshold set of the corresponding network layer, and transmits the comparison result to the server for processing to obtain global consensus variables. The global consensus variables are then sent to each client. The client receives the global consensus variable and updates the client's local model parameters based on the consensus variable.

[0008] Furthermore, each client performs a preset number of stochastic gradient descent training steps on its local dataset. The specific training process is as follows: the client extracts a small batch of training samples from the local dataset, performs forward propagation based on the current shared model parameters and calculates the task loss, then calculates the gradient of the shared model parameters through backpropagation, and updates the local model parameters using a stochastic gradient descent method with momentum. After a preset number of training steps, the initial local model of each client is obtained.

[0009] Furthermore, the local statistics include the mean and variance of the projection vector of each network layer in the client's local model. The acquisition of the projection vector specifically includes: after each communication round, each client obtains the parameters of each network layer in its current round's local model. In the local model, each network layer, combined with the common random seed, performs structured random projection on the parameters of each network layer to generate the projection vector of each network layer.

[0010] Furthermore, the global statistics specifically include: the server aggregating the mean and variance of the projection vectors sent by each network layer of the local model in the client, obtaining the global mean and variance of each global network layer of the corresponding server, and the server sending the global mean and variance of each global network layer as global statistics to the client.

[0011] Furthermore, the generation of the threshold set includes: after receiving the global statistics, the client combines the inverse cumulative distribution function of the standard normal distribution in its local model in the current round to independently generate an ordered set of T quantization thresholds for each network layer in the local model in the current round.

[0012] Further, the projection vector is compared with the corresponding network layer threshold set to obtain a single-bit comparison result. The single-bit comparison result is aggregated into a (T+1) base symbol sequence. The client encodes the symbol sequence into a binary bit stream and uploads it to the server. The server decodes the binary bit stream and performs weighted aggregation on the current network layer of the client's local model in the current round and its corresponding threshold to form a multi-threshold global consensus variable for the current network layer. Then, the global consensus variable is encoded in (T+1) base and sent to each client.

[0013] Furthermore, updating the client's local model parameters specifically includes: after the client decodes and receives the global consensus variable, it performs a preset number of local stochastic gradient descent training updates on the local data based on the global consensus variable to complete the update of the local model parameters, and determines whether the maximum number of communication rounds has been reached. If so, the update is completed, and the client saves the local model with the final updated parameters. If not, the communication rounds continue to be executed.

[0014] Furthermore, in the training update, the client constructs a local optimization objective function with a regularization term based on local data and global consensus variables, and completes the update of local model parameters through the objective function.

[0015] Furthermore, the objective function includes a standard task loss and a layer-wise multi-threshold alignment smoothing regularization term, and the parameters of the local model are updated based on the gradient of the standard loss and the gradient of the regularization term.

[0016] This invention also provides a personalized federated learning system based on layer-by-layer multi-threshold sketches, the system comprising: The client is used to receive interactive information from the server and update the parameters of the local model. The server is used to receive information from the network layer in the client's local model and generate interactive information to guide the updating of local model parameters; The system initialization module generates a public random seed and shared model parameters through the server and transmits them to each client. Each client trains the shared model parameters based on its local data to obtain the initial local model corresponding to each client. The statistics aggregation module obtains the local statistics after projecting the parameters of each network layer in the local model of the current round using the local model of each client and the common random seed, and sends them to the server; the server receives the local statistics of each client, aggregates them to obtain the server's global statistics, and sends them to the client. The threshold generation and global consensus aggregation module receives global statistics through the client, generates a threshold set for each network layer in the local model in the current round, obtains the projection vector of the current network layer model parameters through a public random seed, compares the projection vector with the corresponding network layer threshold set, transmits the comparison result to the server for processing to obtain the global consensus variable, and sends the global consensus variable to each client. The parameter update module receives the consensus variables from the client and updates the local model parameters on the client based on the consensus variables.

[0017] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention adaptively matches the heterogeneous structure and parameter distribution differences of different network layers in a deep model, providing a finer quantization resolution. While improving the fidelity of personalized model representation, it maintains an extremely low bit bidirectional communication overhead limit.

[0018] 2. This invention, by allocating independent quantization threshold sets layer by layer, differs from the traditional global single compression mode. It can adaptively match the heterogeneous statistical characteristics of parameter distributions of different network layers, reduce the loss of extreme value features under non-independent and identically distributed data, and improve the generalization accuracy of personalized models.

[0019] 3. By configuring multiple thresholds for each layer, the present invention divides the parameter space into multiple intervals, providing a higher resolution layer-by-layer quantization representation and a finer global consistency alignment than a single threshold, thus improving the communication-precision tradeoff.

[0020] 4. The present invention uses a (T+1) base encoding method to compactly represent the results of multiple threshold comparisons. The uplink and downlink communication volumes both increase logarithmically rather than nonlinearly with the threshold number T, thereby achieving bidirectional communication volume compression while maintaining extremely low uplink and downlink bandwidth usage. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a personalized federated learning method based on layer-by-layer multi-threshold sketches according to the present invention. Figure 2This is a comparison chart of the accuracy of the method of the present invention with that of existing benchmark algorithms; Figure 3 This is a comparison chart of the training loss of the method of this invention and existing benchmark algorithms; Figure 4 This is a comparison chart of the communication volume of the method of this invention and existing benchmark algorithms on different datasets. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the accompanying drawings.

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0024] This invention provides a personalized federated learning method based on layer-by-layer multi-threshold sketches, such as... Figure 1 As shown, this method performs personalized collaborative training in a distributed architecture consisting of a central parameter server and K clients. The specific training process is as follows: Initialization process: Communication rounds Before starting, the central parameter server initializes the shared model parameters and distributes them to all clients. Each client performs a preset number of stochastic gradient descent training steps on its local dataset. The client extracts mini-batch training samples from its local dataset, performs forward propagation based on the current model parameters, and calculates the task loss. Subsequently, it calculates the gradient of the model parameters through backpropagation and updates the local model parameters using a momentum-driven stochastic gradient descent approach. After several preset initial local training steps, the initial personalized local model for client K is obtained. The server generates a public random seed. It is used to deterministically generate structured random projection matrices on both the client and server. ,in The model represents the first Layer, projection matrix The communication is refreshed once per communication round.

[0025] Layer-by-layer statistical aggregation and adaptive multi-threshold generation: In the At the start of a round of communication, the participating client K uses the local model of the current round. Calculate the local mean of the parameters after projection for each layer. and local variance These local statistics are compiled and uploaded to the server. The server aggregates the local statistics from each client and calculates the first local statistics in the server's global network layer. layer mean and variance The server will aggregate the global mean of each network layer. With global variance This is distributed as a global statistic to all clients.

[0026] Layer-by-layer multi-threshold quantization and global consensus aggregation: The client receives global statistics and combines them with the inverse cumulative distribution function of the standard normal distribution in the local model to independently generate ordered values ​​for each network layer. A set of quantization thresholds Simultaneously, the client calculates the projection vector of the current network layer model parameters. ; For each coordinate component in the obtained projection vector and Each threshold is compared to obtain a single-bit comparison result. The results of the single-bit comparisons are then aggregated into a (T+1) base symbol sequence. The client encodes the (T+1) base symbol sequence into a compact binary bit stream and uploads it to the server. The server receives and decodes the binary bitstream uploaded by the client, performs weighted aggregation on the symbol vector of each network layer and each threshold of the corresponding network layer in the local model, and updates the global layer-by-layer multi-threshold consensus variable, i.e., the global consensus variable: ,in , For the client The number of local samples, The total number of samples participating in training; the server re-encodes the aggregated global consensus variables into a (T+1) base bit stream and broadcasts it to all clients.

[0027] Regular local model parameter updates: Client decodes and receives global consensus variables It updates SGD by performing a preset number of local stochastic gradient descent steps based on its local dataset; During local training, the client uses local data and a global consensus vector. Construct a local optimization objective function with regularization. As shown in the following formula: (1) In the formula, Loss due to local standard tasks The weight decay coefficient is... Regularization coefficient, Weights for consistency regularization terms; The introduced layer-by-layer multi-threshold alignment smoothing regularization term aims to ensure that the projection of the local model parameters falls within the consistency interval induced by global consensus, as shown in the following equation: (2) In the formula, The total number of network layers. The threshold number for each layer, For the whole vector, Represents the dot product of vectors. Indicates the use of smoothing parameters smoothness The norm, through this smoothing process, can make the originally non-differentiable interval consistency penalty term differentiable, thereby supporting subsequent efficient gradient truncation optimization.

[0028] Combine the standard loss gradient and the regularization gradient to update the local model parameters: ,in For learning rate, The regularization term weights are used, and the gradient of the regularization term can be efficiently calculated using the element-wise clipping function Clip.

[0029] Iteration and Convergence: Determine if the maximum number of communication rounds has been reached. If yes, terminate training, and each client retains its final personalized model; otherwise, update the number of communication rounds. Continue with the communication rounds and begin a new round of learning.

[0030] In this invention, the random projection operation is involved. Both the forward projection and the inverse projection of the regularization gradient are implemented using a fast Hadamard transform. Specifically, the target layer parameters are zero-padded to a power of 2, and a normalized Hadamard matrix is ​​used instead of a dense Gaussian random matrix for forward projection and backward gradient mapping. This mechanism avoids explicitly constructing the projection matrix and performing matrix multiplication, reducing the computational complexity of single-layer projection from... Significantly reduced to .

[0031] This invention also provides a personalized federated learning system based on layer-by-layer multi-threshold sketches. The system employs a personalized federated learning method based on layer-by-layer multi-threshold sketches and includes the following modules: The system initialization module generates public random and shared model parameters through the server and transmits them to each client. Each client trains the shared model parameters based on its local data to obtain the initial local model corresponding to each client. The statistics aggregation module obtains the local statistics after projecting the parameters of each network layer in the local model of the current round using the local model of each client and the common random seed, and sends them to the server; the server receives the local statistics of each client, aggregates them to obtain the server's global statistics, and sends them to the client. The threshold generation and global consensus aggregation module generates a threshold set for each network layer in the local model of the current round through the global statistics received by the client. The client obtains the projection vector of the model parameters of the current network layer through a public random seed, compares the projection vector with the threshold set of the corresponding network layer, transmits the comparison result to the server for processing to obtain the global consensus variable, and sends the global consensus variable to each client. The parameter update module receives the consensus variables from the client and updates the local model parameters on the client based on the consensus variables.

[0032] Example This embodiment provides a personalized federated learning method based on layer-by-layer multi-threshold sketches. The selected programming language is Python, and the deep learning framework is PyTorch. The training and testing images used in this embodiment are from public datasets such as MNIST, FMNIST, SVHN, CIFAR-10, and CIFAR-100. The Dirichlet distribution is used to simulate a non-independent and identically distributed data partitioning scenario. The method in this embodiment is implemented according to the following steps: S1, assuming the federated learning system includes There are 1 client, and the index is denoted as . Deep neural network models include There are trainable network layers, indexed as _ . ; Parameter server settings for total communication rounds Local iteration count Threshold number And projection compression ratio.

[0033] S2, in the In round-robin communication, each client Get its current local model The parameters of each layer in the network model Combining a pre-shared public random seed, the fast Hadamard transform is used to analyze the parameters of this layer. Perform structured random projection to generate dimensionality-reduced projection vectors. The specific projection process is as follows: S2.1, Zero-fill expansion: Expands the current layer model parameters Perform zero-padding expansion to obtain the vector This makes the expanded dimension satisfy And it is a power of 2, thus ensuring compatibility with the input format of the Fast Hadamard Transform; S2.2, Hadamard Dimensionality Reduction: Structured random projection is defined as ,in To normalize the Hadamard matrix, For random seed The generated subsampling matrix is ​​responsible for... Randomly selected from the dimensional vector Each component is used as the dimensionality reduction result; this process does not require explicit execution of intensive matrix multiplications, reducing the computational complexity of a single-layer projection from... Down to .

[0034] S3 involves client-server interaction to aggregate global statistics and generate multiple thresholds, specifically including: S3.1, The client calculates the mean of the projection vector for each network layer. and variance Upload to the server; S3.2, The server aggregates the local mean and variance of the data uploaded from each network layer and calculates the global mean. With global variance , S3.3, the server will calculate the global mean for each layer. With global variance This is distributed as a global statistic to all clients.

[0035] S4, after each client receives the global statistics, it calculates the layer-by-layer quantization threshold in its local model and projects it onto its own vector. The process of judgment and encoding upload includes: S4.1, the client combines the inverse cumulative distribution function of the standard normal distribution to independently compute and generate ordered values ​​for each network layer. A set of quantization thresholds Its calculation formula is ; S4.2, for the network layer The first in Each element, using the generated The threshold is used to calculate the interval identifier that falls within: ,at this time ; S4.3, the client transmits this (T+1) base data through... Convert the encoding to a non-negative integer Its specific polynomial encoding formula is as follows: ,in The dimension of the projection vector; then this integer... Converted to a binary bitstream for uploading, thus strictly limiting the transmission bit width of a single parameter to [specific value]. Bit.

[0036] S5, the server-side decodes the binary bitstreams from each client, targeting the network layer. and corresponding network layer threshold Implement a weighted majority voting mechanism: This forms a multi-threshold global consensus vector for the layer, which is then encoded again in (T+1) base and broadcast to all clients. S6, each client receives the global consensus vector. Then, start local epoch training with the preset e-step and update its local model parameters. The specific process is as follows: S6.1, Client-side construction of local optimization objective function: Under the current local model parameters, not only calculating the standard loss for local data classification or regression tasks. and its gradient At the same time, a layer-by-layer multi-threshold alignment smoothing regularization term is introduced. The overall local optimization objective function is as follows: (3) Among them, the smoothing regularization term The mathematical expression is: (4) In the above formula, the smoothing parameter Used for The norm is used for a smooth approximation, and this regularization term is used to measure and penalize the orientation of the local model in the projected space relative to the global quantization consensus in the continuous space. Inconsistency; S6.2, Calculate the regularization term residuals: For each network layer, calculate the residual vector using the smoothing pruning function. ,in For smoothing coefficients; S6.3, Calculating the gradient of the regularization term using the inverse Hadamard map: In order to integrate the residual vector Efficiently mapping back to the original parameter space, the client performs reverse embedding: First, dimensional residuals Embedded into zero initialization dimensional vector In the middle, according to the forward projection neutron sampling matrix Selected set of indexes Fill in Fill the remaining positions with zeros. This embedding process is represented as: (5) in, , Subsequently Applying the inverse Hadamard transform Before the final cut The coordinate components constitute the core gradient of the regularization term in this layer; S6.4, Total Gradient Combination and Model Update: Combine the standard task gradient and the regularization term gradient to update the local model, as follows: (6) In the formula, and These represent the client's position in the [number]th month. Step and the first The model parameters are updated after local iteration. For local learning rate, The weighting coefficients for the interval consistency regularization term are: This is the weight decay coefficient.

[0037] S7, determine whether the maximum number of communication rounds has been reached. If the model has converged, proceed to step S2 to execute the next round of training; otherwise, terminate the process.

[0038] The technical advantages of this invention are illustrated in conjunction with the accompanying drawings: like Figure 2 As shown in the figure, the accuracy of the method of this invention is compared with other benchmark algorithms. The pFedLMS method proposed in this invention corresponds to the red upper triangle marked curve in the figure. It can be seen from the figure that as the number of communication rounds increases, the model accuracy of each algorithm gradually increases and tends to stabilize. Except for the traditional full-precision federated averaging algorithm FedAvg, the model accuracy of the method of this invention is higher than that of benchmark algorithms such as FedProto, FedEden, pFed1BS, FedOBCSAA, zSignFed and OBDA. This shows that the method of this invention can maintain high model learning accuracy and good convergence performance while reducing model communication overhead.

[0039] like Figure 3 As shown, the training loss of the proposed method is compared with other benchmark algorithms. The pFedLMS method proposed in this invention corresponds to the red upper triangle curve in the figure. As can be seen from the figure, in the early stage of training, the proposed method shows a faster loss reduction rate. In the later stage of training, its average training loss is lower than most benchmark algorithms such as FedAvg, FedEden, pFed1BS, FedOBCSAA, zSignFed, and OBDA, and only higher than FedProto. The results show that the proposed method can effectively promote model optimization, enable the model to converge quickly within a limited number of communication rounds, and obtain a low and stable training loss.

[0040] like Figure 4 The figure shows a comparison of the communication overhead of the proposed method with other benchmark algorithms on different datasets. The vertical axis of the figure uses a logarithmic scale, where the pFedLMS method proposed in this invention corresponds to the orange cross-lined bars in the figure. As can be seen from the figure, the proposed method maintains low communication overhead on datasets of different sizes and complexities, and its cumulative communication overhead is significantly lower than benchmark algorithms such as FedAvg, EDEN, zSignFed, OBCSAA, and OBDA. Although some highly compressed algorithms such as pFed1BS have even lower communication overhead, combined with… Figure 2 and Figure 3 It can be seen that the method of the present invention can achieve higher model accuracy and lower training loss; therefore, the method of the present invention achieves a better balance between model learning performance and communication efficiency.

[0041] This article uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A personalized federated learning method based on layer-by-layer multi-threshold sketches, the method being used in a distributed architecture with one server and multiple clients, characterized in that, The method includes: Before each communication round, the server generates a public random seed and shared model parameters and sends them to each client. Each client trains the shared model parameters based on its local data to obtain the initial local model corresponding to each client. At the start of a communication round, each client obtains the local statistics after projecting the parameters of each network layer in the local model of the current round based on the local model of the current round and the common random seed, and sends them to the server; the server receives the local statistics of each client, aggregates them to obtain the server's global statistics, and sends them to the client. The client receives global statistics and generates a threshold set for each network layer in the local model in the current round. The client obtains the projection vector of the model parameters of the current network layer through a public random seed, compares the projection vector with the threshold set of the corresponding network layer, and transmits the comparison result to the server for processing to obtain global consensus variables. The global consensus variables are then sent to each client. The client receives the consensus variables and updates the client's local model parameters based on the consensus variables.

2. The personalized federated learning method based on layer-by-layer multi-threshold sketches according to claim 1, characterized in that, Each client performs a preset number of stochastic gradient descent training steps on its local dataset, including: the client extracts a small batch of training samples from the local dataset, performs forward propagation based on the current shared model parameters and calculates the task loss, then calculates the gradient of the shared model parameters through backpropagation, updates the local model parameters using a momentum-driven stochastic gradient descent method, and after a preset number of training steps, obtains the initial local model for each client.

3. The personalized federated learning method based on layer-by-layer multi-threshold sketches according to claim 1, characterized in that, The local statistics include the mean and variance of the projection vector of each network layer in the client's local model. The acquisition of the projection vector includes: after each communication round, each client obtains the parameters of each network layer in its current round's local model. In the local model, each network layer, combined with the common random seed, performs structured random projection on the parameters of each network layer to generate the projection vector of each network layer.

4. The personalized federated learning method based on layer-by-layer multi-threshold sketches according to claim 3, characterized in that, The global statistics include: the server aggregates the mean and variance of the projection vectors sent by each network layer of the local model in the client, obtains the global mean and variance of each global network layer of the corresponding server, and sends the global mean and variance of each global network layer to the client as global statistics.

5. The personalized federated learning method based on layer-by-layer multi-threshold sketches according to claim 1, characterized in that, The generation of the threshold set includes: after receiving the global statistics, the client combines the inverse cumulative distribution function of the standard normal distribution in its local model in the current round to independently generate T ordered quantization threshold sets for each network layer in the local model in the current round.

6. The personalized federated learning method based on layer-by-layer multi-threshold sketches according to claim 5, characterized in that, The projection vector is compared with the corresponding network layer threshold set to obtain a single-bit comparison result. The single-bit comparison result is aggregated into a (T+1) base symbol sequence. The client encodes the symbol sequence into a binary bit stream and uploads it to the server. The server decodes the binary bit stream and performs weighted aggregation on the current network layer of the client's local model in the current round and its corresponding threshold to form a multi-threshold global consensus variable for the current network layer. Then, the global consensus variable is encoded in (T+1) base and sent to each client.

7. The personalized federated learning method based on layer-by-layer multi-threshold sketches according to claim 1, characterized in that, Updating the client's local model parameters includes: after the client decodes and receives the global consensus variable, it performs a preset number of local stochastic gradient descent training updates on the local data based on the global consensus variable to complete the update of the local model parameters, and determines whether the maximum number of communication rounds has been reached. If so, the update is completed and the client saves the local model with the final updated parameters. If not, the communication rounds continue to be executed.

8. A personalized federated learning method based on layer-by-layer multi-threshold sketches according to claim 7, characterized in that, In the training update, the client constructs a local optimization objective function with a regularization term based on local data and global consensus variables, and updates the local model parameters through the objective function.

9. A personalized federated learning method based on layer-by-layer multi-threshold sketches according to claim 8, characterized in that, The objective function includes a standard task loss and a layer-wise multi-threshold alignment smoothing regularization term. Based on the gradient of the standard loss and the gradient of the regularization term, the parameters of the local model are updated.

10. A personalized federated learning system based on layer-by-layer multi-threshold sketches, characterized in that, The system includes: The client is used to receive interactive information from the server and update the parameters of the local model. The server is used to receive information from the network layer in the client's local model and generate interactive information to guide the updating of local model parameters; The system initialization module generates a public random seed and shared model parameters through the server and transmits them to each client. Each client trains the shared model parameters based on its local data to obtain the initial local model corresponding to each client. The statistics aggregation module obtains the local statistics after projecting the parameters of each network layer in the local model of the current round using the local model of each client and the common random seed, and sends them to the server; the server receives the local statistics of each client, aggregates them to obtain the server's global statistics, and sends them to the client. The threshold generation and global consensus aggregation module receives global statistics through the client, generates a threshold set for each network layer in the local model in the current round, obtains the projection vector of the current network layer model parameters through a public random seed, compares the projection vector with the corresponding network layer threshold set, transmits the comparison result to the server for processing to obtain the global consensus variable, and sends the global consensus variable to each client. The parameter update module receives the consensus variables from the client and updates the local model parameters on the client based on the consensus variables.

Images