A medical image classification method based on adaptive weight weighted federated learning

Abstract

This invention provides a medical image classification method based on adaptive weighted federated learning, belonging to the field of medical image classification technology. The method includes: a server distributing a global image classification model to multiple clients; each client training the global image classification model using stored patient medical image data and updating the parameters ω of the global image classification model. k The performance of the global image classification model trained on each client was validated using saved patient medical image data to obtain the parameter m characterizing the model performance. k The server determines the parameters based on parameter m. k For parameter ω k The global image classification model is re-aggregated by weighting to obtain the final global image classification model; this final global image classification model is then used to classify medical images. This method can increase the impact of high-quality models on the global model when using federated learning for medical image classification.

Description

A Medical Image Classification Method Based on Adaptive Weighted Federated Learning Technical Field

[0001] This invention belongs to the field of medical image classification technology, specifically relating to a medical image classification method based on adaptive weighted federated learning. Background Technology

[0002] Artificial intelligence technologies, represented by deep learning, have made many breakthroughs in the field of computer vision and are gradually being applied to medical data analysis for computer-aided diagnosis. As a data-driven technology, the performance of deep learning models largely depends on large-scale training data. Medical data, such as electronic medical records and medical images, plays a foundational role in achieving precise health monitoring; however, this data is often scattered and difficult to share due to its sensitivity or privacy concerns, such as CT (computed tomography) images of patients recorded by different hospitals.

[0003] To train models without compromising patient privacy, federated learning, as a framework for distributed learning, has attracted increasing research attention. Within this framework, a centralized, global model is trained using data distributed across multiple clients (e.g., healthcare centers), without directly accessing the data on those clients.

[0004] The federated learning process involves three main steps: 1) Distributing the global model from the server to the clients storing the data. 2) Having each client perform training iterations. 3) Returning the models trained on each client to a central repository on the server to update the global model. Its main advantage is protecting data privacy, allowing model training using data stored on different clients, and avoiding the need to transfer data from each client to a central repository. A classic federated algorithm is called Federated Avg, which aggregates the locally trained models from each client using a weighted approach, where the weight of each model is proportional to the size of the client's local dataset. This method has also been increasingly applied to the field of medical image processing. For example, pioneering work by Roth et al. demonstrated the feasibility of federated learning for medical image analysis, with FedAvg applied to breast density recognition in a real-world collaborative environment.

[0005] Although FedAvg's method enables models to learn knowledge from distributed datasets without directly accessing private data, it assigns weights based solely on the size of the data, ignoring the data heterogeneity issues present in medical data.

[0006] This heterogeneity is first reflected in the degree of class imbalance. For example, in the well-known public dermoscopy dataset ISIC2018, the number of samples for the most common disease is 60 times that of the least common disease. In reality, the number of common disease samples and rare disease samples collected from different regions and hospitals often varies greatly. However, rare disease samples play a crucial role in model performance, while an excessive number of common disease samples provides limited improvement to model performance.

[0007] Furthermore, this heterogeneity also stems from variations in medical image quality, which depends on factors such as the characteristics of imaging equipment in different hospitals and the skills of operators. Low-quality medical images inevitably reduce model performance. Therefore, assigning weights solely based on the amount of data is unfair; the weights of high-performance client models may be underestimated. This also means that traditional federated learning algorithms are insufficient to handle the heterogeneity of medical images. Summary of the Invention

[0008] To overcome the shortcomings of the existing technology, this invention provides a medical image classification method based on adaptive weighted federated learning.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A medical image classification method based on adaptive weighted federated learning includes:

[0011] The server distributes the global image classification model to multiple clients.

[0012] Each client uses the stored patient medical image data to train the global image classification model and updates the parameters ω of the global image classification model. k ;

[0013] The performance of the global image classification model trained on each client was validated using saved patient medical image data to obtain the parameter m characterizing the model performance. k ;

[0014] The server depends on parameter m k The parameters are weighted to re-aggregate the global image classification model, resulting in the final global image classification model; and the final global image classification model is then used to classify medical images.

[0015] Furthermore, it also includes: for multi-class patient medical image data in the training of the global image classification model, using the first loss function to train the global image classification model and updating the parameters ω of the global image classification model. k ;

[0016] The first loss function is:

[0017]

[0018] In the formula, β represents the weight of class C in the loss function, used to amplify the contribution of important classes to the prediction results and reduce the contribution of unimportant classes. β = 0.000, n c This represents the number of samples of class C in the training set.

[0019] Furthermore, it also includes: for multi-label patient medical image data used in the training of the global image classification model, a second loss function is used to train the global image classification model and update the parameters ω of the global image classification model. k ;

[0020] The second loss function is:

[0021]

[0022] In the formula, |P| and |N| represent the number of samples in the training set that belong to class j and the number of samples that do not belong to class j, respectively.

[0023] Furthermore, the model parameters of the global image classification model are updated as follows:

[0024]

[0025] in, To update the model parameters at time t+1; η is the step size; f k () represents the loss function.

[0026] Furthermore, the parameter m that characterizes the model performance k for:

[0027]

[0028] In the formula, n k The number of samples in the local data; T is the temperature coefficient; t is the number of iterations;

[0029] in:

[0030]

[0031] In the formula, The verification results are for the corresponding algebra. This is the verification result of the t-th iteration; α is the momentum parameter, representing the contribution of the previous t-1 updates.

[0032] Furthermore, the server, based on parameter mk For parameter ω k The expression for weighted summation is:

[0033]

[0034] Where, ω k (t) This is for updating the global image classification model parameters t times; j is the client number.

[0035] Furthermore, the multi-classification data is dermoscopic image data.

[0036] Furthermore, the multi-label data is, for example, chest CT scan data.

[0037] The medical image classification method based on adaptive weighted federated learning provided by this invention has the following beneficial effects:

[0038] This invention proposes a medical image classification method based on adaptive weighted federated learning. The server selects several clients to participate in the communication and broadcasts a global image classification model to them. Each client independently trains the global image classification model based on its local dataset. Subsequently, when the server performs model aggregation, the weights assigned to each client model are determined by the amount of data on that client and the performance of the corresponding client model on the validation set. The parameters of higher-performing models have a greater impact on the aggregation process, thus addressing the problem in existing federated learning algorithms where the weights of high-performance client models may be underestimated. Attached Figure Description

[0039] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 is a schematic diagram of a medical image classification method based on adaptive weighted federated learning according to the present invention. Detailed Implementation

[0041] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0042] Example 1

[0043] This invention provides a medical image classification method based on adaptive weighted federated learning, as shown in Figure 1, which includes: the server distributing a global image classification model to multiple clients;

[0044] Each client uses the stored patient medical image data to train the global image classification model and updates the parameters ω of the global image classification model. k ;

[0045] The performance of the global image classification model trained on each client was validated using saved patient medical image data to obtain the parameter m characterizing the model performance. k ;

[0046] The server depends on parameter m k For parameter ω k The global image classification model is re-aggregated by weighting to obtain the final global image classification model; and the final global image classification model is used to classify medical images.

[0047] The following are specific embodiments of the present invention:

[0048] 1) Server broadcast global model

[0049] First, the server selects several clients that need to participate in this communication and sends the global model to these clients.

[0050] 2) Local training on the client side

[0051] After receiving the global model from the server, the local client trains the model using local data. Assume the number of clients participating in this round of communication is K, where k represents the client's ID. k This represents the number of samples in the local data. The training formula for the local model is as follows:

[0052]

[0053] In the above formula, l(x) i ,y i ;ω k ) indicates that through local training data (x) i ,y i ) and model parameters ω k Loss values ​​are trained. However, considering the diversity and heterogeneity of medical image data, conventional loss function calculation methods are insufficient for different types of medical data. To address this, we present different loss processing methods for multi-classified data (such as dermoscopy images) and multi-label data (such as chest X-ray CT data).

[0054] The final prediction result of the model for multi-class data is not a simple binary classification problem, but rather a multi-class problem. For this case, we assume there are C possible prediction results for this type of data, then n c This represents the number of predicted values ​​of type c in the dataset. Therefore, we use the following formula to calculate the loss for multi-class problems:

[0055]

[0056] In the formula, β represents the weight of class C in the loss function, used to amplify the contribution of important classes to the prediction results and reduce the contribution of unimportant classes. β = 0.000, n c This represents the number of samples of class C in the training set.

[0057] Multi-label data means that the same sample x i Its corresponding y i There may be more than one. For this case, we assume there are a total of C possible labels. We use the following formula to calculate the loss for the multi-label problem:

[0058]

[0059] In the formula, |P| and |N| represent the number of samples in the training set that belong to class j and the number of samples that do not belong to class j, respectively.

[0060] Using the loss function described above, we set η as the step size and adjust the model parameters using the following formula. Update:

[0061]

[0062] in, To update the model parameters at time t+1; η is the step size; f k () represents the loss function.

[0063] Through the above process, the task can be completed on the local client. Parameters updated.

[0064] 3) Local client verification

[0065] To achieve better results, we perform local model validation after training to determine the model's performance. This performance is then used to determine the model's weight in the final model aggregation. We use 't' to represent the index of the iteration number. This represents the verification result of the corresponding algebra. This represents the verification result of the t-th iteration:

[0066]

[0067] In the formula, The verification results are for the corresponding algebra. This is the verification result of the t-th iteration; α is the momentum parameter, representing the contribution of the previous t-1 updates.

[0068] Through the above process, we obtained the final verification result after iteration t. However, the amount of data used for weight allocation is still crucial in the early stages before the model converges. In the final weight calculation, we use T as the temperature to balance the relationship between the amount of data and the model's validation results, that is:

[0069]

[0070] In this formula, the temperature coefficient T decreases to half its original value with each iteration of the model over a certain period of time. Through these processes, the local client eventually uploads a trained local model ω to the server. k And a parameter m that balances data size and model performance k .

[0071] This formula uses a temperature coefficient to balance the contribution of the amount of data on the client side and the performance of the corresponding client model on the validation set. In the early stage of training, because the model has not yet converged, the size of the local dataset has a greater impact on the weight allocation. In the later stage of training, in order to further improve the performance and robustness of the model, the performance of the corresponding local model on the validation set becomes the main factor affecting the weight allocation.

[0072] 4) Server aggregation of local models

[0073] After all local models have completed the corresponding message uploads, we base our decisions on m. k and ω k Apply weighted averages. That is:

[0074] Finally, we completed the re-aggregation of the global model ω.

[0075] 5) Iterative training

[0076] Repeat steps 1-4 above to iteratively train the global model until it converges.

[0077] The above-described embodiments are merely preferred embodiments of the present invention, and the scope of protection of the present invention is not limited thereto. Any simple changes or equivalent substitutions of the technical solutions that can be obviously obtained by those skilled in the art within the scope of the technology disclosed in the present invention shall fall within the scope of protection of the present invention.

Claims

1. A medical image classification method based on adaptive weighted federated learning, characterized in that, include: The server distributes the global image classification model to multiple clients; each client uses the stored patient medical image data to train and update the parameters of the global image classification model. ； The performance of the global image classification model trained on each client was validated using saved patient medical image data to obtain the parameter m characterizing the model performance. k The server determines the parameters based on parameter m. k right The global image classification model is re-aggregated by weighting to obtain the final global image classification model. This final model is then used to classify medical images. For multi-class patient medical image data used in the global image classification model training, the first loss function is employed to train the global image classification model and update its parameters. The first loss function is: In the formula, These are the weights of class C in the loss function, used to amplify the contribution of important classes to the prediction results and reduce the contribution of unimportant classes. The number of samples in class C in the training set; for multi-label patient medical image data in the global image classification model training, a second loss function is used to train the global image classification model and update its parameters. The second loss function is: In the formula, = 、 、 and Let represent the number of samples belonging to category j and the number of samples not belonging to category j in the training set, respectively; the model parameters of the global image classification model are updated as follows: in, To update the model parameters at time t+1; Step size; ƒ k () represents the loss function; the parameters characterizing the model's performance for: In the formula, The sample size of the local data is denoted by T; the temperature coefficient is denoted by t; where: In the formula, This is the verification result of the t-th iteration; α is the momentum parameter, representing the contribution of the previous t-1 updates.

2. The medical image classification method based on adaptive weighted federated learning according to claim 1, characterized in that, The Parameter m k The expression for weighted summation is: in, k (t) To update the global image classification model parameters t times.

3. The medical image classification method based on adaptive weighted federated learning according to claim 1, characterized in that, The multi-class patient medical image data is dermoscopic image data.

4. The medical image classification method based on adaptive weighted federated learning according to claim 1, characterized in that, The multi-label patient medical image data is chest CT data.

Images