Personalized federal learning method based on personalized enhancement and sharing self-adaption

Abstract

The invention provides a personalized federated learning method based on personalized enhancement and sharing self-adaption, which comprises the following steps: enabling a sharing module in a Transform model to participate in federated aggregation, promoting knowledge migration of each client, storing a personalized module in a local client, and improving the adaptability to heterogeneous data; a Lora low-rank adapter is introduced into a personalized module, only Lora parameters are uploaded, feature expression homogenization caused by global aggregation is avoided, initial global shared module parameters and local parameters of a shared module are subjected to self-adaptive fusion, semantic offset caused by model decoupling is effectively relieved, and the robustness of the system is improved. Therefore, the generalization performance is improved, and the stability and adaptability of a local model are kept; dynamic incremental clustering is performed on each client based on Lora parameters, and shared feature representation with cross-client migration value is mined in a personalized module, so that the calculation overhead is reduced, and the personalized knowledge sharing and migration capability among similar clients is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a personalized federated learning method based on personalized enhancement and shared adaptation. Background Art

[0002] With growing awareness of data privacy and the improvement of relevant laws and regulations, federated learning can effectively address the "data silo" problem while protecting data privacy. This framework allows multiple participants to achieve collaborative training by training local models and submitting model updates to a central server without sharing the original data. This avoids the centralized transmission of raw data, reduces the risk of privacy leakage, and reduces communication overhead. However, because the data of each client is typically not independent and identically distributed, this can lead to deviations between the direction of local model updates and the global optimization goal, reducing the consistency of the global model's performance across clients and limiting the model's generalization capabilities.

[0003] To alleviate the model bias problem caused by non-independent and identically distributed data in federated learning, personalized federated learning was proposed. Its core idea is to customize exclusive models for different clients, while maintaining global knowledge sharing and improving local personalization capabilities. Currently, personalized federated learning methods based on the Transformer structure have gradually become a research hotspot. This structure has stronger cross-distribution modeling capabilities and module decoupling potential due to its self-attention mechanism, which can effectively capture long-distance dependencies and global interaction characteristics between inputs. Although existing research has achieved parameter decoupling by splitting the Transformer into a shared module and a personalized module, there are still significant problems:

[0004] 1. Existing methods generally ignore the problem of inconsistent optimization directions between shared and personalized modules during federated training. During the periodic update process of federated learning, the shared module aggregates with multiple clients after each round of training. Its representation space is continuously affected by the global data distribution during training, introducing parameter features that are incompatible with the current client data distribution and causing semantic drift. The personalized module, on the other hand, remains completely local, and its parameter updates rely solely on the local data distribution. This causes a shift in the optimization objectives and representation spaces of the two. This conflict reduces the stability of model training, hinders the complementary advantages of global knowledge and local features, and thus affects the performance of personalized models.

[0005] 2. In terms of personalized module design, existing methods are mainly divided into two categories: fully localized training of personalized parameters or dynamic generation of personalized parameters by hypernetwork. The former chooses to keep personalized parameters completely local and not participate in global aggregation, ensuring the model's adaptability to local data distribution, but it leads to a serious cold start problem when new clients join. That is, since the personalized module cannot be directly migrated to the new client, the personalized module of the new client must be retrained or rely on a time-consuming and unstable fine-tuning process, which greatly reduces the scalability of the system; the latter introduces a hypernetwork on the server side to dynamically customize the personalized module parameters for each client. However, this method also has obvious limitations. First, the hypernetwork needs to learn the mapping relationship between client features and parameter space. This parameter generation method causes the computational complexity of the training process to grow exponentially. Second, because the training of the hypernetwork relies on global client data, when the data distribution of each client is significantly different, the generated personalized parameters tend to the mean of the global distribution, thereby weakening the personalized representation ability. Summary of the Invention

[0006] The present invention provides a personalized federated learning method based on personalized enhancement and shared adaptation, the purpose of which is to improve the personalization ability and overall training stability of the model in a non-independent and identically distributed environment.

[0007] To achieve the above objectives, the present invention provides a personalized federated learning method based on personalized enhancement and shared adaptation, comprising:

[0008] Step 1: Decouple the initial Transformer model into a shared module and a personalized module on each client, upload the local parameters of the shared module to the central server, and deploy the Lora low-rank adapter in the personalized module;

[0009] Step 2: The central server aggregates the local parameters of each shared module into the initial global shared module parameters and sends the initial global shared module parameters to each client synchronously;

[0010] Step 3: Each client adaptively fuses the initial global shared module parameters with the local parameters of the shared module to obtain a local transformer model with updated shared module parameters.

[0011] Step 4: Freeze the parameters of the Lora low-rank adapter in the local Transformer model, use the local data set to train the local Transformer model with the frozen Lora low-rank adapter parameters, obtain the trained shared module parameters and the trained personalized module parameters, freeze the parameters of the shared module and the personalized module in the local Transformer model, use the local data set to train the local Transformer model with the frozen shared module and personalized module parameters, obtain the trained Lora parameters, and upload the trained shared module parameters and the trained Lora parameters to the central server;

[0012] Step 5: The central server performs dynamic incremental clustering based on the trained Lora parameters to obtain multiple clusters. Each cluster includes at least one trained Lora parameter uploaded by a client. The central server aggregates the Lora parameters in each cluster to obtain multiple aggregated Lora parameters, and sends each aggregated Lora parameter to the client in the corresponding cluster. One aggregated Lora parameter corresponds to one cluster.

[0013] Step 6: The central server aggregates the trained shared module parameters to obtain the global shared module parameters and sends them to each client.

[0014] Step 7: In each client, fine-tune the personalized module parameters of the local Transformer model based on each aggregated Lora parameter, and update the shared module parameters of the local Transformer model based on the global shared module parameters to obtain the global model, and determine whether the global model meets the preset training conditions; if so, the training ends and the global model is used as the segmentation model for segmenting medical imaging data; otherwise, the global model is used as the initial Transformer model and returns to step 1.

[0015] Furthermore, the initial Transformer model includes an embedding layer, a self-attention layer, a feedforward neural network, and a normalization layer;

[0016] Among them, the embedding layer, feedforward neural network, and normalization layer constitute a shared module, and the self-attention layer is a personalized module.

[0017] Furthermore, the initial global shared module parameters are adaptively fused with the local parameters of the shared module. The adaptive fusion expression is:

[0018]

[0019] in, represents the local parameters of the shared module m of client i in round t, represents the shared weight of client i in round t, represents the initial global shared module parameters of round t, represents the local parameters of the shared module m of client i in round t-1.

[0020] Furthermore, the trained Lora parameters are expressed as:

[0021]

[0022] Among them, z i represents the trained Lora parameters uploaded by client i, Flatten(·) represents the flattening operation, Both represent offset matrices.

[0023] Furthermore, the central server performs dynamic incremental clustering based on the trained Lora parameters to obtain multiple clusters, including:

[0024] Construct an adjacency matrix based on the trained Lora parameters;

[0025] According to the adjacency matrix, multiple neighbor clients with the highest similarity are selected for each client to form a candidate neighbor set;

[0026] The elements in the adjacency matrix are updated using the candidate neighbor set, and an undirected sparse graph is constructed based on the updated adjacency matrix. The vertex set of the undirected sparse graph is all clients, and the edge set of the undirected sparse graph is the node pairs whose updated adjacency matrix is ​​greater than a preset threshold.

[0027] A depth-first search is used on an undirected sparse graph to calculate all connected components, and each connected component is regarded as a cluster to obtain multiple clusters. Each cluster includes at least one trained Lora parameter uploaded by a client.

[0028] Furthermore, the adjacency matrix can be expressed as:

[0029]

[0030] Among them, S i,j represents the adjacency matrix, z j Represents the trained LoRa parameters uploaded by client j.

[0031] Furthermore, the expression for aggregating the Lora parameters in each cluster is:

[0032]

[0033] Among them, D irepresents the data volume of client i, D j represents the amount of data from client j, Both represent personalized offset, C m represents the mth cluster.

[0034] Furthermore, the preset training conditions include:

[0035] When the global model reaches the maximum number of training rounds, the training stops;

[0036] When the global model converges to the expected level, stop training.

[0037] The above solution of the present invention has the following beneficial effects:

[0038] The present invention decouples the initial Transformer model into a shared module and a personalized module on each client, uploads the local parameters of the shared module to the central server, and deploys a Lora low-rank adapter in the personalized module; the central server aggregates the local parameters of each shared module into initial global shared module parameters and sends them to each client; each client adaptively fuses the initial global shared module parameters with the local parameters of the shared module to obtain a local Transformer model with updated shared module parameters; the local Transformer model is trained using a local data set to obtain trained shared module parameters, trained personalized module parameters and trained Lora parameters, and the trained shared module parameters and trained Lora parameters are uploaded to the central server; the central server performs dynamic incremental clustering to obtain multiple cluster clusters, each cluster cluster includes at least one trained Lora parameter uploaded by a client, aggregates the Lora parameters in each cluster cluster to obtain multiple aggregated Lora parameters and sends them to the clients in the corresponding cluster cluster; at the same time, the trained shared module parameters are aggregated to obtain global shared module parameters and send them to each client; in each client, the local Transformer model is trained based on each aggregated Lora parameter. The personalized module parameters of the local Transformer model are fine-tuned, and the shared module parameters of the local Transformer model are updated based on the global shared module parameters to obtain a global model, and it is judged whether the global model meets the preset training conditions; if so, the training is terminated, and the global model is used as the segmentation model for segmenting medical imaging data; otherwise, the global model is used as the initial Transformer model and the process returns to step 1; compared with the prior art, the present invention involves the shared module in the Transformer model in federal aggregation, promotes the knowledge transfer of each client, and stores the personalized module locally on the client to improve the adaptability to heterogeneous data; a Lora low-rank adapter is introduced in the personalized module, and only Lora parameters are uploaded to avoid the homogenization of feature expression caused by global aggregation, and the initial global shared module parameters are adaptively fused with the local parameters of the shared module, effectively alleviating the semantic deviation caused by model decoupling, thereby improving the generalization performance while maintaining the stability and adaptability of the local model; by performing dynamic incremental clustering on each client based on the Lora parameters, shared feature representations with cross-client migration value are mined in the personalized module, which significantly reduces the computational overhead and enhances the personalized knowledge sharing and migration capabilities between similar clients.

[0039] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention;

[0041] Figure 2 Flow chart for model training in embodiments of the present application. DETAILED DESCRIPTION

[0042] To make the technical problems solved by the present application, technical solutions and advantages clearer, the following will be described in detail with reference to the drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0043] In the description of the present application, it should be noted that the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0044] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting" should be understood in a broad sense, for example, it can be a locking connection, or it can be a detachable connection, or an integral connection; it can be a mechanical connection, or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium; it can be the communication inside two elements. For those of ordinary skill in the art, the specific meaning of the above-mentioned terms in the present application can be understood according to the specific circumstances.

[0045] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0046] The present application aims at the existing problems, and provides a personalized federated learning method based on personalized enhancement and shared adaptation.

[0047] In embodiments of the present application, the framework of federated learning includes a central server and a plurality of clients;

[0048] The federated learning process includes: taking multi-center medical institutions as federated learning clients, in an embodiment of the present invention, the magnetic resonance imaging equipment and computer tomography equipment of the medical institutions are used to collect local data sets, and the local data sets are medical images such as magnetic resonance images and computer tomography images. The local Transformer model in the client is trained using the local data sets, and the trained shared module parameters and Lora parameters are obtained and uploaded to the central server. The central server aggregates the shared module parameters and Lora parameters uploaded by each client, obtains each aggregated Lora parameter and sends it together with the global shared module parameter to each client, and each client updates the parameters of the local Transformer model based on each aggregated Lora parameter and the global shared module parameter to obtain a global model;

[0049] In an embodiment of the present invention, the central server in federated learning can be a computing device such as a data center server, a private cloud server, or a highly reliable third-party computing platform within a medical institution, which is used to aggregate the parameters of each shared module into initial global shared module parameters, perform dynamic incremental clustering based on the trained Lora parameters, and obtain multiple clusters. The central server aggregates the Lora parameters in each cluster to obtain multiple aggregated Lora parameters, and aggregates the trained shared module parameters to obtain global shared module parameters.

[0050] Finally, the magnetic resonance images and computed tomography images are segmented through the global model, and the segmentation results are the lesion areas in the magnetic resonance images and computed tomography images, which assist in clinical lesion detection, disease analysis and treatment decision-making.

[0051] like Figure 1 As shown, an embodiment of the present invention provides a personalized federated learning method based on personalized enhancement and shared adaptation, including:

[0052] Step 1: Decouple the initial Transformer model into a shared module and a personalized module on each client, upload the local parameters of the shared module to the central server, and deploy the Lora low-rank adapter in the personalized module;

[0053] Step 2: The central server aggregates the local parameters of each shared module into the initial global shared module parameters and sends the initial global shared module parameters to each client synchronously;

[0054] Step 3: Each client adaptively fuses the initial global shared module parameters with the local parameters of the shared module to obtain a local Transformer model with updated shared module parameters.

[0055] Step 4: Freeze the parameters of the Lora low-rank adapter in the local Transformer model, use the local data set to train the local Transformer model with the frozen Lora low-rank adapter parameters, obtain the trained shared module parameters and the trained personalized module parameters, freeze the parameters of the shared module and the personalized module in the local Transformer model, use the local data set to train the local Transformer model with the frozen shared module and personalized module parameters, obtain the trained Lora parameters, and upload the trained shared module parameters and the trained Lora parameters to the central server;

[0056] Step 5: The central server performs dynamic incremental clustering based on the trained Lora parameters to obtain multiple clusters. Each cluster includes at least one trained Lora parameter uploaded by a client. The central server aggregates the Lora parameters in each cluster to obtain multiple aggregated Lora parameters, and sends each aggregated Lora parameter to the client in the corresponding cluster.

[0057] Step 6: The central server aggregates the trained shared module parameters to obtain the global shared module parameters and sends them to each client.

[0058] Step 7: In each client, fine-tune the personalized module parameters of the local Transformer model based on each aggregated Lora parameter, and update the shared parameters of the local Transformer model based on the global shared module parameters to obtain the global model, and determine whether the global model meets the preset training conditions; if so, the training ends and the global model is used as the segmentation model for segmenting medical imaging data; otherwise, the global model is used as the initial Transformer model and returns to step 1.

[0059] Specifically, the initial Transformer model includes an embedding layer, a self-attention layer, a feedforward neural network, and a normalization layer;

[0060] Among them, the embedding layer, feedforward neural network, and normalization layer constitute a shared module, and the self-attention layer is a personalized module.

[0061] The embodiment of the present invention divides the Transformer model into a shared module and a personalized module based on the expression ability of each functional module for input features and the difference in sensitivity to data distribution in the Transformer model structure. In the Transformer model, the embedding layer, the feedforward neural network, and the normalization layer have universal representation capabilities. Therefore, these three functional modules are used as shared modules to participate in federated aggregation, thereby promoting knowledge transfer across clients. The self-attention layer has the ability to be highly sensitive to the distribution of input features, so it is retained as a personalized module locally on the client to avoid homogenization of feature expression caused by global aggregation.

[0062] Since the shared module and the personalized module are clearly distinguished in structure, it is difficult to balance local adaptability and global consistency by adopting a unified global update strategy. Therefore, the embodiment of the present invention proposes an adaptive fusion mechanism to introduce independent sharing weights for the shared module, and adaptively fuse the initial global shared module parameters with the local parameters of the shared module. The adaptive fusion expression is:

[0063]

[0064] in, represents the local parameters of the shared module m of client i in round t, represents the shared weight of client i in round t, For control The degree of influence on the shared module parameters of client i, when When it is larger, the client is more inclined to adopt global knowledge, otherwise, it is more inclined to maintain its local features. represents the initial global shared module parameters of round t, represents the local parameters of the shared module m of client i in round t-1.

[0065] In the embodiment of the present invention, in order to calculate the sharing weight scientifically and reasonably, It should be set according to the change of the shared module between the client and the previous round of training and its similarity with the initial global shared module. Specifically, the difference between the client shared module parameters and the initial global shared module parameters is set to Assume that the change of the client's own shared module in the last two rounds is Then the shared weight of client i in round t is It can be defined as:

[0066]

[0067] Here, σ(·) represents the sigmoid function, which is used to normalize the mapping value to the interval [0, 1]. ∈ represents the smoothing term, ∈>0, which is used to avoid the denominator being zero. α represents the scaling factor that adjusts the response amplitude of the function, α>0. Multiple candidate values ​​are selected at the beginning of training, and the local training loss and global convergence performance are evaluated on the validation set. The α value with the best performance is selected for subsequent rounds of calculation.

[0068] The above definition of sharing weight reflects the dynamic trade-off strategy: if the parameters of the client sharing module are recently When the change is large or close to the global shared module parameters, reduce To avoid introducing external interference and enhance local continuity, this adaptive fusion strategy improves the smoothness and semantic continuity of the model update path in a data heterogeneous environment, providing a more stable shared feature expression for subsequent global aggregation.

[0069] In the local training phase, the embodiment of the present invention adopts a layered perception fine-tuning strategy to achieve separate training of the backbone structure and the low-rank adapter, improving the model while maintaining structural stability and achieving more refined personalized adjustment. Specifically, by using the query, key, and value projection matrices W of the self-attention layer in the Transformer model q 、W k 、W v The Lora low-rank adapter is deployed on each of them to achieve parameter-efficient personalized modeling. The personalized offset of the original projection weight is represented by the following low-rank decomposition:

[0070]

[0071] Among them, A and B both represent low-rank matrices, r<<d,r, thus ensuring low parameter redundancy and high adaptability;

[0072] In the above training process, the hierarchical perception fine-tuning strategy realizes the separate training of the backbone structure and the low-rank adapter. In the first stage, the parameters of the Lora low-rank adapter in the local Transformer model are frozen, and the local Transformer model with the frozen Lora low-rank adapter parameters is trained using the local dataset to obtain the trained shared module parameters and the trained personalized module parameters, which have adapted to the local data distribution and enhanced the individual feature expression ability; in the second stage, the parameters of the shared module and the personalized module in the local Transformer model are fixed (frozen), and the local Transformer model with the frozen shared module and personalized module parameters is trained using the local dataset, that is, only the low-rank matrices A and B in the Lora low-rank adapter are trained to obtain the trained Lora parameters, which achieves more refined personalized tuning while maintaining the original expression structure and reduces the computational and communication overhead;

[0073] After the training is completed, the personalized module remains in the client and does not participate in the upload. Only the trained shared module parameters and Lora parameters are uploaded to the central server. The trained Lora parameters are expressed as:

[0074]

[0075] Among them, z i represents the trained Lora parameters uploaded by client i. Flatten(·) represents the flattening operation, which is used to flatten the sum of the three offset matrices into a one-dimensional vector as the client's low-dimensional personalized expression feature. Both represent offset matrices.

[0076] In order to further capture the semantic similarity between clients in the Lora low-rank adapter and enhance personalization capabilities, the central server in the embodiment of the present invention performs dynamic incremental clustering based on the trained Lora parameters to obtain multiple clusters, including:

[0077] The cosine similarity calculation is performed based on the trained Lora parameters to construct the adjacency matrix;

[0078] According to the adjacency matrix, multiple neighbor clients with the highest similarity are selected for each client to form a candidate neighbor set;

[0079] The elements in the adjacency matrix are updated using the candidate neighbor set, and an undirected sparse graph is constructed based on the updated adjacency matrix. The vertex set of the undirected sparse graph is all clients, and the edge set of the undirected sparse graph is the node pairs whose updated adjacency matrix is ​​greater than a preset threshold.

[0080] A depth-first search is used on an undirected sparse graph to calculate all connected components, and each connected component is regarded as a cluster to obtain multiple clusters. Each cluster includes at least one trained Lora parameter uploaded by a client.

[0081] Specifically, since cosine similarity calculation eliminates the modulus interference of the update amplitude through vector directional consistency, only focusing on the directional consistency of the feature vector in high-dimensional space, and at the same time has the efficiency and robustness of high-dimensional data processing, it improves the computational efficiency of constructing the adjacency matrix in large-scale data scenarios. Therefore, cosine similarity is selected as the feature matching metric between clients. The calculation expression of the adjacency matrix is:

[0082]

[0083] Among them, S i,j represents the adjacency matrix, z j represents the trained Lora parameters uploaded by client j. The adjacency matrix defines the structural similarity between clients. Subsequently, the k neighbor clients with the highest similarity are selected for each client to form the candidate neighbor set:

[0084] N i ={j|j∈TopK k (S i,j )}

[0085] Among them, TopK k (·) means taking the top k index sets sorted by similarity.

[0086] The adjacency relationship obtained at this time is usually asymmetric. To ensure that the adjacency matrix S i,j Due to the symmetry and sparsity of , only the edges that are close neighbors are retained: if and only if client i is in the neighbor list of client j, and client j is also in the neighbor list of client i, the connection of this edge is retained and its weight is set to the cosine similarity between the two; otherwise, the edge weight is reset to zero. At this time, the adjacency matrix elements are updated as follows:

[0087]

[0088] Based on this, we construct an undirected sparse graph G = (V, E), where the vertex set is the client number V = {1, 2, ..., n}, and the edge set is the one that satisfies S' i,j >0 node pair E={(i,j)||S i,j >0}.

[0089] A depth-first search is used on the undirected sparse graph to calculate all connected components, and each connected component is defined as a cluster, which reflects the high similarity and structural consistency of the clients in the cluster in the personalized LoRA parameter space. If the adjacency matrix S is directly used to calculate the connected components, the cluster is divided into two groups:i,j Constructing connected subgraphs for clustering can easily lead to an overly dense adjacency matrix containing a large number of low-quality weak connections, which weakens the discriminability and structural clarity of the clustering. Therefore, filtering can be used to effectively suppress noise edges and improve the accuracy of the clustering results.

[0090] In order to avoid the computational overhead caused by repeatedly calculating the complete graph in each round, the embodiment of the present invention introduces a dynamic incremental clustering update strategy. In the initial round, complete clustering is performed. In subsequent rounds, only the similarity between the client with significant feature changes and the cluster center is updated, and z is recalculated. i The strategy significantly reduces the clustering computation complexity and clustering convergence time of the central server by comparing the similarity with the existing cluster centers and performing cluster migration locally.

[0091] Specifically, in the aggregation stage, in order to ensure the structural integrity of the Lora low-rank adapter, the embodiment of the present invention performs a m The weighted average aggregation of the Lora parameters in each cluster is performed, and the expression for aggregating the Lora parameters in each cluster is:

[0092]

[0093] Among them, D i represents the amount of data from client i, D j Represents the amount of data of client j, which is used for weighting. The aggregation results are then distributed to each client in the corresponding cluster as a global supplement to the fine-tuning of the personalization module. Both represent personalized offset, C m represents the mth cluster.

[0094] Subsequently, combined with the adaptive weight balancing mechanism, the shared module and the personalized module are guided to gradually align in the optimization direction, alleviating the problem of representation space offset and promoting model convergence. The training flow chart is as follows: Figure 2 shown.

[0095] It should be noted that the embodiment of the present invention does not limit the order of training the shared module and the personalized module and training the Lora low-rank adapter in step 4, nor does it limit the order of steps 5 and 6. Steps 5 and 6 can be performed simultaneously or one after the other, depending on the configuration of the central server.

[0096] Specifically, the preset training conditions include:

[0097] When the global model reaches the maximum number of training rounds, the training stops;

[0098] When the global model converges to the expected level, stop training.

[0099] The present invention decouples the initial Transformer model into a shared module and a personalized module on each client, uploads the local parameters of the shared module to the central server, and deploys a Lora low-rank adapter in the personalized module; the central server aggregates the local parameters of each shared module into initial global shared module parameters and sends them to each client; each client adaptively fuses the initial global shared module parameters with the local parameters of the shared module to obtain a local Transformer model with updated shared module parameters; the local Transformer model is trained using a local data set to obtain trained shared module parameters, trained personalized module parameters and trained Lora parameters, and the trained shared module parameters and trained Lora parameters are uploaded to the central server; the central server performs dynamic incremental clustering to obtain multiple cluster clusters, each cluster cluster includes at least one trained Lora parameter uploaded by a client, aggregates the Lora parameters in each cluster cluster to obtain multiple aggregated Lora parameters and sends them to the clients in the corresponding cluster cluster; at the same time, the trained shared module parameters are aggregated to obtain global shared module parameters and send them to each client; in each client, the local Transformer model is trained based on each aggregated Lora parameter. The personalized module parameters of the local Transformer model are fine-tuned, and the shared module parameters of the local Transformer model are updated based on the global shared module parameters to obtain a global model, and it is judged whether the global model meets the preset training conditions; if so, the training is terminated, and the global model is used as the segmentation model for segmenting medical imaging data; otherwise, the global model is used as the initial Transformer model and the process returns to step 1; compared with the prior art, the present invention involves the shared module in the Transformer model in federal aggregation, promotes the knowledge transfer of each client, and stores the personalized module locally on the client to improve the adaptability to heterogeneous data; a Lora low-rank adapter is introduced in the personalized module, and only Lora parameters are uploaded to avoid the homogenization of feature expression caused by global aggregation, and the initial global shared module parameters are adaptively fused with the local parameters of the shared module, effectively alleviating the semantic deviation caused by model decoupling, thereby improving the generalization performance while maintaining the stability and adaptability of the local model; by performing dynamic incremental clustering on each client based on the Lora parameters, shared feature representations with cross-client migration value are mined in the personalized module, which significantly reduces the computational overhead and enhances the personalized knowledge sharing and migration capabilities between similar clients.

[0100] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A personalized federated learning method based on personalized enhancement and shared adaptation, characterized by: include: Step 1: Decouple the initial Transformer model into a shared module and a personalized module on each client, upload the local parameters of the shared module to the central server, and deploy the Lora low-rank adapter in the personalized module; Step 2: the central server aggregates the local parameters of each shared module into initial global shared module parameters and synchronously sends the initial global shared module parameters to each client; Step 3: For each client, the client adaptively fuses the initial global shared module parameters with the local parameters of the shared module to obtain a local Transformer model with updated shared module parameters; Step 4: Freeze the parameters of the Lora low-rank adapter in the local Transformer model, train the local Transformer model with the frozen Lora low-rank adapter parameters using a local data set to obtain trained shared module parameters and trained personalized module parameters, freeze the parameters of the shared module and personalized module in the local Transformer model, train the local Transformer model with the frozen shared module and personalized module parameters using a local data set to obtain trained Lora parameters, and upload the trained shared module parameters and the trained Lora parameters to the central server; Step 5: The central server performs dynamic incremental clustering based on the trained Lora parameters to obtain multiple clusters. Each cluster includes at least one trained Lora parameter uploaded by a client. The central server aggregates the Lora parameters in each cluster to obtain multiple aggregated Lora parameters, and sends the aggregated Lora parameters to the clients in the corresponding clusters. One aggregated Lora parameter corresponds to one cluster. Step 6: The central server aggregates the trained shared module parameters to obtain the global shared module parameters and sends them to each client. Step 7: In each client, the personalized module parameters of the local Transformer model are fine-tuned based on each aggregated Lora parameter, and the shared module parameters of the local Transformer model are updated based on the global shared module parameters to obtain a global model, and determine whether the global model meets the preset training conditions; if so, the training is completed, and the global model is used as a segmentation model for segmenting medical imaging data; otherwise, the global model is used as the initial Transformer model and returns to step 1.

2. The personalized federated learning method based on personalized enhancement and shared adaptation according to claim 1 is characterized in that: The initial Transformer model includes an embedding layer, a self-attention layer, a feedforward neural network, and a normalization layer; Among them, the embedding layer, the feedforward neural network, and the normalization layer constitute a shared module, and the self-attention layer is a personalized module.

3. The personalized federated learning method based on personalized enhancement and shared adaptation according to claim 1 is characterized in that: The initial global shared module parameters are adaptively fused with the local parameters of the shared module. The adaptive fusion expression is: in, represents the local parameters of the shared module m of client i in round t, represents the shared weight of client i in round t, represents the initial global shared module parameters of round t, represents the local parameters of the shared module m of client i in round t-1.

4. The personalized federated learning method based on personalized enhancement and shared adaptation according to claim 1 is characterized in that: The trained Lora parameters are expressed as: Among them, z i represents the trained Lora parameters uploaded by client i, Flatten(·) represents the flattening operation, Both represent offset matrices.

5. The personalized federated learning method based on personalized enhancement and shared adaptation according to claim 4 is characterized in that: The central server performs dynamic incremental clustering based on the trained Lora parameters to obtain multiple clusters, including: Construct an adjacency matrix based on the trained Lora parameters; Selecting a plurality of neighboring clients with the highest similarity for each client according to the adjacency matrix to form a candidate neighbor set; Using the candidate neighbor set, the elements in the adjacency matrix are updated, and an undirected sparse graph is constructed based on the updated adjacency matrix, wherein the vertex set of the undirected sparse graph is all clients, and the edge set of the undirected sparse graph is the node pairs whose updated adjacency matrix is ​​greater than a preset threshold; A depth-first search is used on the undirected sparse graph to calculate all connected components, and each connected component is used as a cluster to obtain multiple clusters, each of which includes at least one trained Lora parameter uploaded by a client.

6. The personalized federated learning method based on personalized enhancement and shared adaptation according to claim 5 is characterized in that: The calculation expression of the adjacency matrix is: Among them, S i,j represents the adjacency matrix, z j Represents the trained LoRa parameters uploaded by client j.

7. The personalized federated learning method based on personalized enhancement and shared adaptation according to claim 6 is characterized in that: The expression for aggregating the Lora parameters in each cluster is: Where D represents the amount of data from client i, D j represents the amount of data from client j, Both represent personalized offset, C m represents the mth cluster.

8. The personalized federated learning method based on personalized enhancement and shared adaptation according to claim 1 is characterized in that: The preset training conditions include: When the global model reaches the maximum number of training rounds, the training stops; When the global model converges to the expected level, stop training.

Images