Federal learning method and system based on fine-grained layer rarefaction and similarity perception aggregation

Abstract

The invention discloses a federated learning method and system based on fine-grained layer sparsification and similarity perception aggregation, and the method comprises the steps: carrying out the personalized local model training through a local private data set, and extracting a feature map from a specified feature extraction layer; calculating the sparseness of the feature map to obtain the feature sparseness; calculating a change rate of local model parameters in a training process, setting a sparse ratio for different layers of the local model to obtain a sparse mask matrix, and performing sparsification on the local model to obtain a sparse model; and uploading the feature sparseness and the sparseness model to a server, so that the server constructs a similarity matrix, calculates a similarity index, weights the local model parameter of each client according to the similarity index, updates the local model parameter, and distributes the local model parameter back to each client. The method can effectively reduce the communication overhead, improves the performance of a personalized model, and is especially suitable for scenes with high data privacy protection requirements, such as medical image analysis.

Description

Technical Field

[0001] The present invention relates to a federated learning method and system based on fine-grained layer sparsification and similarity-aware aggregation, belonging to the technical field of federated learning. Background Art

[0002] In recent years, artificial intelligence (AI) has been increasingly applied in the medical field. An increasing number of medical image analysis tasks rely on large-scale local models to improve diagnostic accuracy. However, unlike natural image analysis, medical image analysis has always faced the challenge of small sample sizes, making it difficult for researchers to accurately capture the true data distribution and hindering the development of robust and reliable medical image analysis models by medical institutions. The ideal solution to this problem is to integrate medical image data from multiple institutions into a large-scale dataset and use this dataset to train high-quality machine learning models. However, due to strict privacy protection policies restricting cross-institutional data sharing, such as the HIPAA Act, which strictly restricts the exchange of personal health data and images, this approach is difficult to implement in real-world clinical practice. Federated learning (FLT) has become an important tool in the field of medical image analysis because it protects patient privacy while enabling collaborative model training across medical institutions. However, medical images typically have high resolution and complex features. To accurately extract medical information, models require a large number of parameters. After each local training session, the client (i.e., the medical institution) must upload a large number of parameter updates to the server, which significantly increases communication overhead and becomes a major bottleneck hindering the efficient operation of federated learning systems. Therefore, how to reduce communication overhead while ensuring model performance has become a key issue in the application of federated learning in the medical field.

[0003] Current work on personalized federated learning for model compression primarily relies on quantization and sparsification techniques. Quantization methods reduce the precision of model parameters to reduce the amount of data transmitted, thereby reducing communication overhead. However, in the medical field, medical imaging data features are complex and have high resolution. Low-precision quantization often loses critical pathological information, affecting diagnostic accuracy. Sparsification methods, such as the TopK strategy, further reduce communication overhead by transmitting only the most variable local model parameters. However, when data is not independent and identically distributed (IID), the features learned by each client during training vary significantly, and their parameter choices with large gradient variations may also differ significantly, making it difficult for the server to align these updates when aggregating the model. Furthermore, while data categories for key diseases are similar across hospitals, the sample sizes for each disease often vary significantly, and data on key diseases in some hospitals is extremely scarce. Traditional methods fail to fully utilize this small but crucial sample size, resulting in poor model performance in detecting key conditions. Therefore, there is an urgent need to propose a novel sparsification strategy that can reduce communication costs while addressing data heterogeneity and class imbalance, thereby improving the detection accuracy of key conditions while maintaining model performance. Summary of the Invention

[0004] In view of this, the present invention provides a federated learning method, device, system and storage medium based on fine-grained layer sparsification and similarity-aware aggregation, which retains complete parameter accuracy and avoids the loss of key information due to low-bit quantization. The hierarchical dynamic sparsity strategy can more carefully reflect the differences of each layer in different clients, avoiding the loss of important parameters and index misalignment problems caused by data heterogeneity.

[0005] The first object of the present invention is to provide a federated learning method based on fine-grained layer sparsification and similarity-aware aggregation.

[0006] The second object of the present invention is to provide a federated learning device based on fine-grained layer sparsification and similarity-aware aggregation.

[0007] The third object of the present invention is to provide a federated learning system based on fine-grained layer sparsification and similarity-aware aggregation.

[0008] A fourth object of the present invention is to provide a computer-readable storage medium.

[0009] The first object of the present invention can be achieved by adopting the following technical solutions:

[0010] A federated learning method based on fine-grained layer sparsification and similarity-aware aggregation, applied to a client, includes:

[0011] Use local private datasets to perform personalized local model training and extract feature maps from the specified feature extraction layer;

[0012] Calculate the sparsity of the feature map to obtain the feature sparsity;

[0013] Calculate the rate of change of the local model parameters during training, set the sparse ratio for different layers of the local model according to the rate of change, and obtain a sparse mask matrix;

[0014] Use the sparse mask matrix to sparse the local model to obtain a sparse model;

[0015] The feature sparsity and the sparsification model are uploaded to the server so that after collecting the feature sparsity and the sparsification model of all clients, the server constructs a similarity matrix based on the Euclidean distance, calculates the similarity index, weights the local model parameters of each client according to the similarity index, updates the local model parameters, and distributes the updated model parameters back to each client.

[0016] Furthermore, the feature map is extracted from the specified feature extraction layer as follows:

[0017]

[0018] in, is the feature map, l is the feature extraction layer, w is the local model parameter, x i represents the i-th input sample, b l is the bias vector of the lth layer.

[0019] Furthermore, the calculation of the sparsity of the feature map to obtain the feature sparsity specifically includes:

[0020] Calculate the sparsity of each sample in the local private dataset as follows:

[0021]

[0022] Among them, H and W are the height and width of the feature map respectively;

[0023] Take the average value as the overall sparsity of client i on channel c, as follows:

[0024]

[0025] Where N is the number of samples in the local private dataset, is a sample in the local private dataset of client i.

[0026] Furthermore, the rate of change of the local model parameters during the training process is calculated as follows:

[0027]

[0028] in, is the rate of change, θ l is the parameter set of layer l, Δw i [l] is the parameter change of layer l, and They are the states before and after the update of layer l respectively.

[0029] Furthermore, the sparse ratio is set for different layers of the local model according to the change rate to obtain a sparse mask matrix, which specifically includes:

[0030] Set the sparsity ratio γ for different layers of the local model according to the rate of change l When the parameter change rate of a layer is higher than the preset change rate, a lower sparse ratio is set for the layer, and the γ of the absolute value of the parameter is calculated. l Quantile as the clipping threshold ρ l , when the absolute value of the parameter of layer l is greater than the clipping threshold ρ l When , the corresponding mask value is 1, otherwise it is 0, and a sparse mask matrix M is obtained. i ;

[0031] The local model is sparsely constructed using a sparse mask matrix, as shown in the following formula:

[0032] w i =w i ⊙M i

[0033] Among them, w i are the local model parameters of client i.

[0034] Furthermore, constructing a similarity matrix based on the Euclidean distance and calculating the similarity index specifically includes:

[0035] The similarity matrix is ​​constructed based on the Euclidean distance as follows:

[0036]

[0037] in, is the feature sparsity of client i on channel c, is the feature sparsity of client j on channel c, R i is the feature sparsity vector of client i, R j is the feature sparsity vector of client j;

[0038] Based on the similarity matrix, the distribution similarity index of each client is calculated as follows:

[0039]

[0040] Among them, DSI i is the distribution similarity index of client i, μ i is the mean similarity between client i and all other clients.

[0041] Furthermore, the local model parameters of each client are weighted according to the similarity index to update the local model parameters, specifically including:

[0042] The local model parameters of each client are weighted according to the similarity index as follows:

[0043] β i =DSI i ·(1-|a g -a i |)

[0044] Among them, DSI i is the distribution similarity index of client i, a g and a i are the local models of client i and global models The accuracy on the current client validation set, t is the tth round of training, |a g -a i |Used to quantify the performance difference between the two, β i is the contribution ratio of the global model to the local model;

[0045] Update the local model parameters of each client as follows:

[0046]

[0047] in, The updated local model parameters of client i.

[0048] The second object of the present invention can be achieved by adopting the following technical solutions:

[0049] A federated learning device based on fine-grained layer sparsification and similarity-aware aggregation, applied to a client, comprising:

[0050] The feature map extraction module is used to perform personalized local model training using a local private dataset and extract feature maps from a specified feature extraction layer;

[0051] The first calculation module is used to calculate the sparsity of the feature map to obtain feature sparsity;

[0052] The second calculation module is used to calculate the change rate of the local model parameters during the training process, set the sparse ratio for different layers of the local model according to the change rate, and obtain a sparse mask matrix;

[0053] The sparse module is used to sparse the local model using a sparse mask matrix to obtain a sparse model;

[0054] The local model parameter update module is used to upload the feature sparsity and the sparsification model to the server, so that after collecting the feature sparsity and the sparsification model of all clients, the server constructs a similarity matrix based on the Euclidean distance, calculates the similarity index, weights the local model parameters of each client according to the similarity index, updates the local model parameters, and distributes the updated model parameters back to each client.

[0055] The third object of the present invention can be achieved by adopting the following technical solutions:

[0056] A federated learning system based on fine-grained layer sparsification and similarity-aware aggregation, the system comprising N clients and a server, wherein the server is connected to the N clients respectively;

[0057] The client is used to execute the above-mentioned federated learning method;

[0058] The server is used to construct a similarity matrix based on Euclidean distance after collecting the feature sparsity and sparsification models of all clients, calculate the similarity index, weight the local model parameters of each client according to the similarity index, update the local model parameters, and distribute the updated model parameters back to each client.

[0059] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0060] A computer-readable storage medium stores a program, which, when executed by a processor, implements the above-mentioned federated learning method.

[0061] The present invention has the following beneficial effects compared to the prior art:

[0062] 1. This invention proposes fine-grained layer sparsification. By carefully screening the parameters of each layer in the deep neural network, only the parameters that are most critical to the model prediction results are transmitted, which greatly reduces the amount of communication data. It is particularly suitable for medical scenarios with limited bandwidth.

[0063] 2. Compared with traditional methods, this invention improves the aggregation mode of federated learning. Through similarity perception, it can dynamically adjust the model update strategy to make the model more consistent with the data distribution of each client, thereby improving the adaptability of the model to non-independent and identically distributed data.

[0064] 3. The present invention uses the natural sparsity of neural networks to characterize the client's original data, which not only protects patient privacy, but also accurately reflects the statistical characteristics of the data distribution of each hospital, effectively preventing the risk of sensitive information leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0066] Figure 1 Schematic diagram of the federated learning system architecture based on fine-grained layer sparsification and similarity-aware aggregation according to Example 1 of the present invention.

[0067] Figure 2 This is a flowchart of the federated learning method based on fine-grained layer sparsification and similarity-aware aggregation of Example 1 of the present invention.

[0068] Figure 3 This is a structural block diagram of a federated learning device based on fine-grained layer sparsification and similarity-aware aggregation according to embodiment 2 of the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0070] Example 1:

[0071] like Figure 1 As shown, this embodiment provides a federated learning system based on fine-grained layer sparsification and similarity-aware aggregation. The system includes N clients and a server S. Server S is a central server connected to N clients respectively. N clients constitute a client set C. Client numbered i has a local private dataset D. i ={x i ,y i}, where x i represents the data sample set, y iRepresents the corresponding category label set; in order to learn the classification task, each client also has a neural network model as a local model, whose parameters are recorded as w i .

[0072] like Figure 2 As shown, this embodiment provides a federated learning method based on fine-grained layer sparsification and similarity-aware aggregation. The method is mainly implemented through the above-mentioned client and specifically includes the following steps:

[0073] S201. Use a local private dataset to perform personalized local model training and extract feature maps from a specified feature extraction layer.

[0074] In this embodiment, each client i uses a local private dataset D i Perform personalized training and optimize its local model w i , use ReLU to extract the feature map from the specified feature extraction layer, as follows:

[0075]

[0076] in, is the feature map, l is the feature extraction layer, w is the local model parameter, x i represents the i-th input sample, b l is the bias vector of the lth layer, which can help the model converge faster.

[0077] S202: Calculate the sparsity of the feature map to obtain feature sparsity.

[0078] All clients calculate the sparsity of each sample in the local private dataset as follows:

[0079]

[0080] Among them, H and W are the height and width of the feature map respectively.

[0081] Take the average value as the overall sparsity of client i on channel c, as follows:

[0082]

[0083] Where N is the number of samples in the local private dataset, is a sample in the local private dataset of client i.

[0084] Therefore, the sparsity vector calculated by client i based on local data is R i =[sp1,sp2,…,sp q], which reflects the feature sparsity of the client on the selected important channels and serves as its privacy-preserving distribution representation. Subsequently, each client uploads the calculated sparsity vector to the server for subsequent similarity analysis without sharing any original data.

[0085] S203. Calculate the rate of change of the local model parameters during the training process, set sparse ratios for different layers of the local model according to the rate of change, and obtain a sparse mask matrix.

[0086] In this embodiment, all clients calculate the rate of change of local model parameters during the training process, assuming that the parameter set of a certain layer is θ l The states of this layer before and after parameter update are and Then the parameter change of this layer is Normalize it to get the rate of change:

[0087]

[0088] Finally, the parameter change rate of each layer is Set the sparse ratio γ for different layers of the model according to the rate of change l When the parameter change rate of a layer is higher than the preset change rate, that is, when the change rate is high, it means that the layer has undergone significant adjustments during the local training process. The client will set a lower sparsity ratio for the layer, that is, set a lower sparsity ratio, and the client calculates the γ of the absolute value of the parameter. l Quantile as the clipping threshold ρ l , when the absolute value of the parameter of layer l is greater than the clipping threshold ρ l When , the corresponding mask value is 1, otherwise it is 0, and a sparse mask matrix M is obtained. i .

[0089] S204: Use the sparse mask matrix to sparse the local model to obtain a sparse model.

[0090] In this embodiment, all clients use a sparse mask matrix to sparse the local model, as shown in the following formula:

[0091] w i =w i ⊙M i

[0092] S205: Upload the feature sparsity and the sparsification model to the server.

[0093] All clients will have feature sparsity R i And the sparse model w iAfter uploading to the server, the server collects the feature sparsity and sparsification models of all clients and constructs a similarity matrix based on Euclidean distance, as shown below:

[0094]

[0095] in, is the feature sparsity of client i on channel c, is the feature sparsity of client j on channel c, R i is the feature sparsity vector of client i, R j is the feature sparsity vector of client j.

[0096] Randomly select a reference client and then calculate the Euclidean distance of other clients relative to the reference client. Specifically, first randomly select client i as the reference client and then calculate its distribution representation R i The Euclidean distance between the distribution representations of other clients is used to generate a similarity vector (sim(R i ,R1),sim(R i ,R2),…,sim(R i ,R n )). In this vector, if (R i ,R j ) is low, it means that the data distribution of client i and client j is similar; at the same time, if (R i ,R j ) and (R i ,R z ) are close, we can infer that the data distributions of clients j and z are also similar.

[0097] The server calculates the distribution similarity index DSI of each client based on the similarity matrix, which represents the similarity between different clients, as shown in the following formula:

[0098]

[0099] Among them, DSI i is the distribution similarity index of client i, μ i is the mean similarity between client i and all other clients.

[0100] The server weights the local model parameters of each client according to the similarity index and updates the local model parameters as follows:

[0101] β i =DSI i ·(1-|a g -a i |)

[0102] Among them, DSI i is the distribution similarity index of client i, a g and a i are the local models of client i and global models The accuracy on the current client validation set, t is the tth round of training, |a g -a i |Used to quantify the performance difference between the two, β i Indicates the contribution ratio of the global model to the local model.

[0103] The server updates the local model parameters of each client as follows:

[0104]

[0105] in, The updated local model parameters of client i.

[0106] The server distributes the updated local model parameters back to each client and repeats the above steps S201 to S205 until convergence.

[0107] It should be noted that although the method operations of the above embodiments are described in a particular order, this does not require or imply that the operations must be performed in that particular order, or that all of the illustrated operations must be performed to achieve the desired results. Rather, the depicted steps may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into a single step, and / or a single step may be broken down into multiple steps.

[0108] Example 2:

[0109] like Figure 3 As shown, this embodiment provides a federated learning device based on fine-grained layer sparsification and similarity-aware aggregation, which is applied to the client and includes a feature map extraction module 301, a first calculation module 302, a second calculation module 303, a sparsification module 304, and a local model parameter update module 305. The specific functions of each module are as follows:

[0110] A feature map extraction module 301 is used to perform personalized local model training using a local private dataset and extract feature maps from a specified feature extraction layer;

[0111] A first calculation module 302 is used to calculate the sparsity of the feature map to obtain feature sparsity;

[0112] The second calculation module 303 is used to calculate the change rate of the local model parameters during the training process, and set the sparse ratio for different layers of the local model according to the change rate to obtain a sparse mask matrix;

[0113] A sparse module 304 is used to sparse the local model using a sparse mask matrix to obtain a sparse model;

[0114] The local model parameter update module 305 is used to upload the feature sparsity and the sparsification model to the server, so that after collecting the feature sparsity and the sparsification model of all clients, the server constructs a similarity matrix based on the Euclidean distance, calculates the similarity index, weights the local model parameters of each client according to the similarity index, updates the local model parameters, and distributes the updated model parameters back to each client.

[0115] It should be noted that the device provided in this embodiment is only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0116] Example 3:

[0117] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the federated learning method of the above-mentioned embodiment 1 is implemented as follows:

[0118] A personalized local model is trained using a local private dataset, and a feature map is extracted from a specified feature extraction layer; the sparsity of the feature map is calculated to obtain the feature sparsity; the rate of change of the local model parameters during the training process is calculated, and the sparse ratio is set for different layers of the local model according to the rate of change to obtain a sparse mask matrix; the local model is sparsified using the sparse mask matrix to obtain a sparse model; the feature sparsity and the sparse model are uploaded to the server, so that after the server collects the feature sparsity and the sparse model of all clients, it constructs a similarity matrix based on the Euclidean distance, calculates the similarity index, weights the local model parameters of each client according to the similarity index, updates the local model parameters, and distributes the updated model parameters back to each client.

[0119] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0120] In this embodiment, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or device. Furthermore, in this embodiment, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution device, apparatus, or device. The computer program contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0121] The computer readable storage medium can be written in one or more programming languages ​​or a combination thereof to execute the computer program for performing the present embodiment, including object-oriented programming languages ​​such as Java, Python, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect via the Internet).

[0122] In summary, this invention provides an efficient and feasible federated learning solution for privacy-sensitive tasks such as medical image analysis by reducing communication overhead, enhancing personalized modeling capabilities, and improving privacy protection. It has been verified to perform well on multiple medical datasets.

[0123] The foregoing description is merely a preferred embodiment of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive. The scope of the present invention is defined by the appended claims, not the foregoing description, and all variations that come within the meaning and range of equivalents of the claims are intended to be included within the present invention.

Claims

1. A federated learning method based on fine-grained layer sparsification and similarity-aware aggregation, applied to the client, characterized by: The method comprises: Use local private datasets to perform personalized local model training and extract feature maps from the specified feature extraction layer; Calculate the sparsity of the feature map to obtain the feature sparsity; Calculate the rate of change of the local model parameters during training, set the sparse ratio for different layers of the local model according to the rate of change, and obtain a sparse mask matrix; Use the sparse mask matrix to sparse the local model to obtain a sparse model; The feature sparsity and the sparsification model are uploaded to the server so that after collecting the feature sparsity and the sparsification model of all clients, the server constructs a similarity matrix based on the Euclidean distance, calculates the similarity index, weights the local model parameters of each client according to the similarity index, updates the local model parameters, and distributes the updated model parameters back to each client.

2. The federated learning method according to claim 1, characterized in that: The feature map is extracted from the specified feature extraction layer as follows: in, is the feature map, l is the feature extraction layer, w is the local model parameter, x i represents the i-th input sample, b l is the bias vector of the lth layer.

3. The federated learning method according to claim 1, wherein: The calculation of the sparsity of the feature map to obtain the feature sparsity specifically includes: Calculate the sparsity of each sample in the local private dataset as follows: Among them, H and W are the height and width of the feature map respectively; Take the average value as the overall sparsity of client i on channel c, as follows: Where N is the number of samples in the local private dataset, is a sample in the local private dataset of client i.

4. The federated learning method according to claim 1, wherein: The calculation of the change rate of the local model parameters during the training process is as follows: in, is the rate of change, θ l is the parameter set of layer l, Δw i [l] is the parameter change of layer l, and They are the states before and after the update of layer l respectively.

5. The federated learning method according to claim 1, wherein: The method sets the sparse ratio for different layers of the local model according to the change rate to obtain a sparse mask matrix, specifically including: Set the sparsity ratio γ for different layers of the local model according to the rate of change l When the parameter change rate of a layer is higher than the preset change rate, a lower sparse ratio is set for the layer, and the γ of the absolute value of the parameter is calculated. l Quantile as the clipping threshold ρ l , when the absolute value of the parameter of layer l is greater than the clipping threshold ρ l When , the corresponding mask value is 1, otherwise it is 0, and a sparse mask matrix M is obtained. i ; The local model is sparsely constructed using a sparse mask matrix, as shown in the following formula: In i =in i ⊙M i Among them, w i are the local model parameters of client i.

6. The federated learning method according to claim 1, wherein: The similarity matrix is ​​constructed based on the Euclidean distance, and the similarity index is calculated, specifically including: The similarity matrix is ​​constructed based on the Euclidean distance as follows: in, is the feature sparsity of client i on channel c, is the feature sparsity of client j on channel c, R i is the feature sparsity vector of client i, R j is the feature sparsity vector of client j; Based on the similarity matrix, the distribution similarity index of each client is calculated as follows: Among them, DSI i is the distribution similarity index of client i, μ i is the mean similarity between client i and all other clients.

7. The federated learning method according to claim 1, wherein: The weighting of the local model parameters of each client according to the similarity index and updating of the local model parameters specifically include: The local model parameters of each client are weighted according to the similarity index as follows: b i =DSI i ·(1-|a g -a i |) Among them, DSI i is the distribution similarity index of client i, a g and a i are the local models of client i and global models The accuracy on the current client validation set, t is the tth round of training, |a g -a i |Used to quantify the performance difference between the two, β i is the contribution ratio of the global model to the local model; Update the local model parameters of each client as follows: in, The updated local model parameters of client i.

8. A federated learning device based on fine-grained layer sparsification and similarity-aware aggregation, applied to a client, characterized in that: The device comprises: The feature map extraction module is used to perform personalized local model training using a local private dataset and extract feature maps from a specified feature extraction layer; The first calculation module is used to calculate the sparsity of the feature map to obtain feature sparsity; The second calculation module is used to calculate the change rate of the local model parameters during the training process, set the sparse ratio for different layers of the local model according to the change rate, and obtain a sparse mask matrix; The sparse module is used to sparse the local model using a sparse mask matrix to obtain a sparse model; The local model parameter update module is used to upload the feature sparsity and the sparsification model to the server, so that after collecting the feature sparsity and the sparsification model of all clients, the server constructs a similarity matrix based on the Euclidean distance, calculates the similarity index, weights the local model parameters of each client according to the similarity index, updates the local model parameters, and distributes the updated model parameters back to each client.

9. A federated learning system based on fine-grained layer sparsification and similarity-aware aggregation, characterized by: The system includes N clients and a server, wherein the servers are connected to the N clients respectively; The client is used to execute the federated learning method according to any one of claims 1 to 7; The server is used to construct a similarity matrix based on Euclidean distance after collecting the feature sparsity and sparsification models of all clients, calculate the similarity index, weight the local model parameters of each client according to the similarity index, update the local model parameters, and distribute the updated model parameters back to each client.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the federated learning method according to any one of claims 1 to 7 is implemented.

Images