Adaptive clustering federated learning modeling method for precision medicine
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- LIAONING NORMAL UNIVERSITY
- Filing Date
- 2025-07-01
- Publication Date
- 2026-07-24
Smart Images
Figure CN120781928B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of precision medicine technology, and in particular to an adaptive clustering federated learning modeling method for precision medicine. Background Technology
[0002] In the process of digital transformation and precision medicine development in the medical field, federated learning, as a key technology for achieving collaborative modeling of medical data across institutions, has attracted much attention because it can complete joint training while ensuring data privacy. However, the inherent heterogeneity of medical data has become the core bottleneck restricting the effectiveness of federated learning. Patient groups in different medical institutions have significant differences in feature dimensions, sample size, and label distribution due to differences in region, lifestyle, and disease spectrum. The integration of wearable devices and clinical data further exacerbates the complexity of data structure.
[0003] Traditional federated learning methods typically tend to construct a single global model. However, different clients exhibit significant differences in data features, sample distributions, and label distributions. A single global model struggles to adequately adapt to the diverse needs of all clients, resulting in limited model performance. Furthermore, it may lack generalization ability when faced with new or unseen client data, failing to adapt to novel data distributions. Therefore, this study explores an adaptive clustering federated learning modeling method for precision medicine. Summary of the Invention
[0004] This invention proposes an adaptive clustering federated learning modeling method for precision medicine to address the technical problems existing in the background art.
[0005] To solve the above problems, the technical solution of the present invention is:
[0006] An adaptive clustering federated learning modeling method for precision medicine includes the following steps:
[0007] (1) Data collection and preprocessing:
[0008] S1-1. Obtain patient data related to the target disease in a medical setting, perform data cleaning operations such as missing value filtering, and obtain a refined dataset;
[0009] S1-2. Divide the fine dataset into multiple client datasets according to the same distribution and non-same distribution patterns. Each client dataset is divided into a training set and a test set according to a preset ratio.
[0010] (2) Model building and training:
[0011] S2-1. m clients receive initial model parameters from the server, perform local training on the initial model received from the server using the cross-entropy loss function, update the model parameters using gradient descent, and send the updated model parameters back to the server.
[0012] S2-2. After receiving the model parameters returned by the client, the server uses the gap statistics method to determine the optimal number of clusters K. * Then, the clients are grouped according to the optimal number of clusters using the K-means algorithm, and the clustering loss is calculated.
[0013] S2-3. Based on the clustering results, the server generates multiple global models by weighted averaging and distributes the global models to the corresponding clients.
[0014] Preferably, in S2-1, the cross-entropy loss function is: Among them, y i This represents the true label of sample i. This represents the predicted probability of sample i, where N is the total number of samples;
[0015] The gradient descent method is: Where η represents the learning rate used in model training, w (t) represents the model parameters of the multilayer perceptron in the t-th training round, and b represents the size of the local mini-batch.
[0016] Preferably, in S2-1, after training is completed, m clients send their trained model parameters back to the server, and in the t-th round, these parameters are respectively represented as...
[0017] Preferably, in step S2-2, the optimal number of clusters K is determined using the gap statistics method. * The formula is:
[0018] Preferably, in step S2-2, the optimal number of clusters K is determined using the gap statistics method. * The specific steps are as follows: Generate a set of random samples uniformly distributed within the actual sample clustering region, use the K-means algorithm to divide the random samples into k clusters, and calculate the loss value G. k Repeat the above process N times, calculate the average of these N loss values, and use it as the estimated loss for the random sample. Calculate the difference Gap(k) between the actual sample loss and the estimated loss for the random sample, and select the k value that maximizes Gap(k) as the optimal number of clusters K. * Next, the K-means algorithm is used to determine the optimal number of clusters K. * Group the clients.
[0019] Preferably, in S2-2, each cluster S j It contains a set of clients, for each cluster S j Calculate the clustering loss J(S) j The formula for calculating clustering loss is:
[0020]
[0021] Among them, w i This represents the model parameters for client i.
[0022] Preferably, in steps S2-3, clustering S = [S1, ..., S...] is used. J ,…,S K *] The server will generate K by weighted average. * A global model W1,…,W K *, when cluster S j If the number of clients is q, then the aggregation model of the j-th cluster in round t is:
[0023]
[0024] in, Indicates from cluster S j The model parameters obtained by the i-th client in the process, where n represents S j The total number of training samples across all clients, n i Let ξ1+…+ξi+…+ξq=1.
[0025] Preferably, in steps S2-3, after aggregation is completed, the server will transfer the global model W. j Send the data to each q client. After receiving the data, the client updates its local model and repeats the iterative process of steps 1-3 until the model converges or the maximum number of iterations is reached.
[0026] The above-described technical solution of the present invention has the following beneficial technical effects:
[0027] 1. This invention accurately determines the optimal number of global models through adaptive clustering and divides the clusters according to the similarity of client model parameters, enabling independent training of groups. This effectively improves the model's adaptability to differences in data features, sample and label distributions of different clients, avoids the problem of poor performance of a single model on some clients, and significantly enhances the model's ability to capture complex medical patterns.
[0028] 2. This invention optimizes the model's generalization ability through group training, enabling it to better cope with new data distributions. Under the premise of ensuring the privacy and security of medical data, it significantly improves the accuracy and reliability of the model in precision medicine scenarios such as disease diagnosis and prognosis prediction. Attached Figure Description
[0029] Figure 1 This is a schematic diagram of the data collection and preprocessing process in the adaptive clustering federated learning modeling method for precision medicine of the present invention.
[0030] Figure 2 This is a flowchart illustrating the model training process of the adaptive clustering federated learning modeling method for precision medicine according to the present invention.
[0031] Figure 3 The graph shows the ACC, AUC, and F1 data of FedAC and the other three federated models in this invention using non-IID data during the last 20 training rounds.
[0032] Figure 4 This is a comparison chart of seven evaluation metrics used by FedAC and three other federal models in this invention, based on IID data. Detailed Implementation
[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of the invention.
[0034] The following is in conjunction with the appendix Figure 1 To further detail this application, an adaptive clustering federated learning modeling method for precision medicine includes the following steps:
[0035] (1) Data collection and preprocessing:
[0036] S1-1. Obtain patient data related to the target disease in a medical setting, perform data cleaning operations such as missing value filtering, and obtain a refined dataset;
[0037] This application targets acute coronary syndrome (ACS). In practice, patients diagnosed with ACS in the cardiology department of a tertiary hospital in China between October and December 2019 were collected. These patients underwent fatty acid profiling. A total of 4,238 samples were collected, containing 36 features. After thorough filtering to remove any missing values, a fine dataset of 1,988 samples was obtained. Among these samples, 729 were non-ACS patients, while 1,259 were patients diagnosed with ACS.
[0038] S1-2, All fine datasets X totalThe dataset is divided into m client datasets using two modes: identical distribution and non-identical distribution. Each client dataset is further divided into a training set and a test set according to a preset ratio. 70% of the data is used for training, and 30% of the data is used for testing, providing a data foundation for subsequent federated learning training and validation.
[0039] (2) Model building and training:
[0040] S2-1. First, m clients receive initial model parameters from the server, and perform local training based on the initial model received from the server using the cross-entropy loss function. The cross-entropy loss function is: Among them, y i This represents the true label of sample i. Let N represent the predicted probability of sample i, and N be the total number of samples. The model parameters are updated using gradient descent, which is as follows: Where η represents the learning rate used in model training, w (t) Let represent the model parameters of the multilayer perceptron in the t-th training round, and b represent the size of the local mini-batch. The updated model parameters are then sent back to the server. After training is complete, m clients send their trained model parameters back to the server. In the t-th round, these parameters are represented as follows:
[0041] S2-2. After receiving the model parameters returned by the client, the server uses the gap statistics method to determine the optimal number of clusters K. * Determine the optimal number of clusters K * The formula is:
[0042] The specific implementation steps are as follows:
[0043] Step 1: Generate a set of random samples that are uniformly distributed within the actual sample clustering region;
[0044] Step 2: Use the K-means algorithm to divide the random samples into k clusters and calculate the loss value G. k ;
[0045] Step 3: Repeat the above process N times, and calculate the average of these N loss values as the estimated loss for the random sample;
[0046] Step 4: Calculate the difference Gap(k) between the actual sample loss and the estimated loss of the random sample, and select the k value that maximizes Gap(k) as the optimal number of clusters K. * .
[0047] In determining K * After setting the value, the K-means algorithm is used to determine the optimal number of clusters K.* Group the clients, each cluster S j It contains a set of clients, for each cluster S j Calculate the clustering loss J(S) j The formula for calculating clustering loss is: Among them, w i Represents the model parameters for client i;
[0048] S2-3, Based on clustering S=[S1,…,S… J ,…,S K *] The server will generate K by weighted average. * A global model W1,…,W K * Assuming cluster S j If the number of clients is q, then the aggregation model of the j-th cluster in round t is expressed as:
[0049]
[0050] in, Indicates from cluster S j The model parameters obtained by the i-th client in the process, where n represents S j The total number of training samples across all clients, n i Let ξ1+…+ξi+…+ξq=1.
[0051] S2-4. After aggregation is complete, the server will send the global model W. j Send the data to each q client. After receiving the data, the client updates its local model and repeats the iterative process of steps 1-3 until the model converges or reaches the maximum number of iterations, thus completing the federated learning training.
[0052] To verify the effectiveness of the method of the present invention, the following comparative experimental steps were designed:
[0053] Step 1: Select classic federated learning models FedAvg, FedProx, and FedNova, and compare them with FedAC proposed in this invention;
[0054] Step 2: Conduct experiments in two data scenarios: independent and identically distributed (i.i.d.) and non-independent and identically distributed (i.i.d.) data.
[0055] Step 3: Use classification performance metrics commonly used in precision medicine scenarios, including ACC, AUC, F1, etc.
[0056] refer to Figure 3 and Figure 4The results show that FedAC performs well on both IID and non-IID data, especially in handling client heterogeneity. FedAC improves the overall performance of the model by grouping clients through adaptive clustering and training an independent global model in each cluster. FedAC's performance is particularly outstanding on non-IID data, indicating its significant advantage in handling federated learning environments with unevenly distributed data.
[0057] In summary, the adaptive clustering federated learning modeling method for precision medicine provided by this invention addresses the challenge of traditional federated learning's single global model's inability to adapt to the heterogeneity of client data. It accurately determines the optimal number of global models through adaptive clustering and divides clusters based on the similarity of client model parameters, enabling independent training for each group. This method effectively improves the model's adaptability to differences in data features, sample, and label distributions across different clients, avoiding the problem of poor performance of a single model on some clients and significantly enhancing the model's ability to capture complex medical patterns. Simultaneously, group training optimizes the model's generalization ability, enabling it to better cope with new data distributions. While ensuring the privacy and security of medical data, it significantly improves the accuracy and reliability of the model in precision medicine scenarios such as disease diagnosis and prognosis prediction, providing an efficient and practical solution for cross-institutional collaborative modeling of medical data and powerfully promoting the development of precision medicine.
[0058] The above are merely preferred embodiments of the present invention, intended only to aid in understanding the method and core ideas of this application. The scope of protection of the present invention is not limited to the above embodiments; all technical solutions falling within the scope of the present invention's concept are within its protection. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention. The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An adaptive clustering federated learning modeling method for precision medicine, characterized in that, Includes the following steps: (1) Data collection and preprocessing: S1-1. Obtain patient data related to the target disease in a medical setting, perform missing value filtering and cleaning operations on the data to obtain a refined dataset; S1-2. Divide the fine dataset into multiple client datasets according to the same distribution and non-same distribution patterns. Each client dataset is divided into a training set and a test set according to a preset ratio. (2) Model building and training: S2-1, Each client receives initial model parameters from the server, performs local training using the cross-entropy loss function based on the initial model received from the server, updates the model parameters using gradient descent, and sends the updated model parameters back to the server. S2-2. After receiving the model parameters returned by the client, the server uses the gap statistics method to determine the optimal number of clusters. Then, the clients are grouped according to the optimal number of clusters using the K-means algorithm, and the clustering loss is calculated. In step S2-2, the optimal number of clusters is determined using gap statistics. The formula is: ; In step S2-2, the optimal number of clusters is determined using gap statistics. The specific steps are as follows: Generate a set of random samples that are uniformly distributed within the actual sample clustering region, and use the K-means algorithm to divide the random samples into... Calculate the loss value for each cluster. Repeat the above process. Next, calculate this The average of the two loss values is used as the estimated loss for the random sample. The difference between the actual loss and the estimated loss for the random sample is then calculated. , choose to The largest Value as the optimal number of clusters Then, the K-means algorithm is used to determine the optimal number of clusters. Group the clients; In S2-2, each cluster Includes a set of clients for each cluster. Calculate clustering loss The formula for calculating clustering loss is: ， in, Indicates client Model parameters; S2-3. Based on the clustering results, the server generates multiple global models by weighted averaging and distributes the global models to the corresponding clients. Based on clustering The server will generate a weighted average. A global model When clustering The number of clients in the middle is Then the first The clustering in the th case is at the th case. The aggregation model of the wheel is: ， in, Indicates from clustering The first in Model parameters obtained by each client express The total number of training samples across all clients. Indicates the client during training. The number of samples, ; After aggregation is complete, the server will display the global model. Send to each The client receives the data and updates its local model, repeating the iterative process of steps 1-3 until the model converges or reaches the maximum number of iterations.
2. The adaptive clustering federated learning modeling method for precision medicine according to claim 1, characterized in that, In S2-1, the cross-entropy loss function is: ,in, Indicates sample The true label, Indicates sample The predicted probability, It is the total number of samples; The gradient descent method is: ,in, This represents the learning rate used during model training. Indicates the first Model parameters of the multilayer perceptron during round training. This indicates the size of the local small batch.
3. The adaptive clustering federated learning modeling method for precision medicine according to claim 1, characterized in that, In S2-1, after training is completed... The clients send their trained model parameters back to the server, and the first client sends its trained model parameters back to the server. These parameters in the wheel are respectively expressed as ,…, .