A participation-adaptive federated learning method and system for medical image analysis

Abstract

A participation-adaptive federated learning method and system for medical image analysis, belonging to the technical field of medical image processing. The method, applied to a central server, includes: Step S01, sending the model parameters #imgabs0# of the analysis model to multiple medical institutions, which then locally update the model parameters; Step S02, receiving the gradient #imgabs1# transmitted back by the medical institutions after the local update, and updating #imgabs2# to #imgabs3# based on the participation of each institution; Step S03, constructing the loss function of the analysis model at #imgabs4# as a function #imgabs5#, and taking the derivative of this function to calculate the gradient value #imgabs7# of #imgabs6#; then using #imgabs8# to update #imgabs9#, resulting in an updated participation level of #imgabs10#; Step S04, updating #imgabs11# to #imgabs12# based on the updated participation level; Step S05, sending #imgabs13# to multiple medical institutions, and repeating Steps S01-S05 until the loss function of the analysis model converges. The system is implemented based on the above method. The present invention solves the problem of data heterogeneity by dynamically adjusting the participation of each organization in the model training process, thereby improving the final image analysis effect.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to a participation-adaptive federated learning method and system for medical image analysis. Background Art

[0002] Existing deep learning algorithms based on large-scale data typically centralize the data before developing models. However, due to the privacy and sensitivity of medical imaging data, the data often cannot leave the medical institution where it resides, resulting in algorithm failure. Furthermore, due to regional disparities in my country, medical institutions in different regions have significant differences in the quantity, quality, and variety of data. Developing models based on such heterogeneous data will undoubtedly limit the ultimate effectiveness. Therefore, an effective distributed learning paradigm is urgently needed to ensure the efficient and accurate execution of recognition and diagnosis algorithms based on large-scale medical imaging without leaving the local data repository.

[0003] Faced with large-scale data, mainstream machine learning and deep learning methods develop models based on the datasets. However, due to the privacy and sensitivity of medical data, data exchange and integration between different medical institutions is difficult. In 2017, McMahan et al. proposed federated learning. This learning framework enables model development by exchanging model parameters and gradients while retaining data locally, thus ensuring data privacy. It has now begun to gain widespread application in the field of medical image analysis.

[0004] The heterogeneity of data across institutions currently limits the effectiveness of the final model. Current research, both domestically and internationally, often focuses on the similarity of data distribution and data volume across institutions, ignoring the impact on the final image analysis results. This heterogeneity makes it difficult for current federated learning to effectively utilize large-scale data to develop effective image analysis applications. Ensuring the effectiveness of the final model while maintaining data localization is a critical scientific issue with practical significance.

[0005] Invention patent application CN202111471753.7 discloses a federated learning image segmentation method based on model similarity measurement, and specifically discloses that the method includes: Step 1: Selecting the U-Net network architecture and the Deeplab v3+ network architecture as the network models used by each participant for local training; Step 2: Proposing the concept of model similarity, designing a model similarity measurement algorithm, and applying the idea of federated learning to aortic image segmentation; Step 3: Conducting network training based on the selected loss function interaction mode. Although this method can cope with situations with different data distributions and overfitting, it ignores the impact of different institutions on the final image analysis effect.

[0006] Invention patent application CN202110813710.6 discloses a multi-center brain tumor prognosis survival prediction method based on federated learning, and specifically discloses that the method includes the following steps: Step S1: Construct a federated learning model based on a client-server architecture of multi-center federated learning; the federated learning model is based on an active learning strategy to minimize the total loss of all devices; Step S2: Locally on each client, collect original medical images of brain tumors, and unify and de-privacy the collected images to obtain brain tumor medical tumor image data and historical tumor data; Step S3: Establish a multimodal imaging database of brain tumor medical tumors, and establish a brain tumor segmentation model through a neural network algorithm, take a brain tumor medical image sample from the multimodal imaging database of brain tumor medical tumors, and extract brain tumor features in the brain tumor medical image sample; Step S4: Analyze the brain tumor features extracted in step S3, select the most useful predictive features from the extracted features, and select a linear combination of the predictive features to calculate the patient's radiomics score; Repeat the execution of steps S2 to step S4 until a training set is formed;

[0007] Step S5: Based on the training set, the machine learning model is combined to generate a three-category model for the final survival period, which is used to classify and predict the patient's brain tumor survival. This method utilizes federated learning to aggregate data from different centers to construct a complete data sample. Based on this complete data sample, classification prediction is performed by combining multi-center and multivariate features. Therefore, this method cannot solve the problem of model interoperability caused by the heterogeneity of data from different institutions when the data is retained locally. Summary of the Invention

[0008] This paper addresses the data heterogeneity of different medical imaging institutions by dynamically adjusting the participation of each institution in the model training process through a performance-oriented federated learning algorithm, thereby resolving the data heterogeneity issue and improving the final image analysis results. Therefore, a participation-adaptive federated learning method and system for medical image analysis is proposed.

[0009] The present invention provides a participation-adaptive federated learning method for medical image analysis, which is applied to a central server and includes:

[0010] Step S01: Send the model parameters of the analysis model to multiple medical institutions , so that medical institutions can update model parameters locally;

[0011] Step S02: Receive the gradients sent back by multiple medical institutions after the model parameters are updated. , and the model parameters of the analysis model are updated for the first time according to the following formula:

[0012]

[0013] Among them, K is the number of medical institutions participating in the training, is the gradient from the medical institution side k, η is the learning rate, is a scalar that controls the medical institution's participation in this round of training;

[0014] Step S03, the model parameters are The loss function of the analysis model is constructed as Differentiable functions , and for this function Derivative calculation The gradient value ; Then use renew to get updated engagement ;

[0015] Step S04: Based on the updated participation rate, the model parameters of the analysis model are updated again according to the following formula:

[0016] ;

[0017] Step S05: Send the updated model parameters of the analysis model to multiple medical institutions. , repeat the above steps S01 to S05 until the loss function of the analysis model converges.

[0018] The present invention is not limited to the use of the model for any purpose of analysis. It is mainly guided by the final effect of the analysis model, and dynamically optimizes the proportion of participation of different medical institutions in the training, so as to achieve the effect of controlling their participation. First, the model parameters are distributed through the central server, and then updated locally by each medical institution. In this way, the privacy of each institution can be ensured without leaving the local data. After that, the gradient value after the parameter update is transmitted back, and the unified model parameter update is performed on the central server. On the basis of retaining the heterogeneity of the data, the participation of the institution can be incorporated to perform a preliminary update of the unified parameters; then the participation update is used to further update the model parameters again. By repeating multiple update steps, the loss function of the analysis model converges, thereby ensuring that the recognition and diagnosis algorithms based on large-scale medical images can be executed efficiently and accurately.

[0019] In particular, after the participation is updated separately, the model parameters are updated. This process is updated through the contribution to the final performance, that is, the participation of each institution is adjusted according to the model effect, and then the ladder model parameters are adjusted.

[0020] Preferably, the method further comprises step S00, before step S01, presetting the participation degree of each medical institution for the first updating process of the model parameters.

[0021] As a preference, the gradient of the medical institution's return The acquisition process is as follows: the medical institution locally calculates the loss function of the analysis model and then obtains the gradient .

[0022] Preferably, the step S03 includes:

[0023] Step S31: Input the test data set into the model parameter The analysis model is built and the loss function of the analysis model is constructed;

[0024] Step S32, setting the above loss function to A differentiable function, denoted by ;

[0025] Step S33, according to the formula , calculated to obtain The gradient value ;

[0026] Step S34: After normalization, Update and get .

[0027] Preferably, the method further comprises step S00', before step S01, constructing part of the data in the data set under the application scenario as a test data set for use in the loss function calculation of steps S03 and S05.

[0028] Preferably, the analysis model is a convolutional neural network model or a Transformer model.

[0029] Preferably, the analysis model is an image classification model, an image segmentation model, or an image detection model.

[0030] The present invention provides a participation-adaptive federated learning system for medical image analysis, which is applied to a central server and includes:

[0031] Model parameter sending module, used to send model parameters of analysis models to multiple medical institutions , so that medical institutions can update model parameters locally;

[0032] Model parameter receiving module, used to receive the gradients sent back by multiple medical institutions after the model parameters are updated ;

[0033] The model parameter update module is used to update the model parameters of the analysis model for the first time according to the following formula:

[0034]

[0035] Among them, K is the number of medical institutions participating in the training, is the gradient from the medical institution side k, η is the learning rate, is a scalar that controls the medical institution's participation in this round of training;

[0036] Participation update module, which is used to update the model parameters to The loss function of the analysis model is constructed as Differentiable functions , and for this function Derivative calculation The gradient value ; Then use renew to get updated engagement ;

[0037] The model parameter updating module is further configured to update the model parameters of the analysis model again according to the following formula based on the updated participation degree:

[0038] ;

[0039] The model parameter sending module is also used to send the updated model parameters of the analysis model to multiple medical institutions. , repeat the above-mentioned local model parameter update, model parameter first update, participation update, and model parameter re-update process until the loss function of the analysis model converges.

[0040] Preferably, the analysis model is a convolutional neural network model or a Transformer model.

[0041] Preferably, the analysis model is an image classification model, an image segmentation model, or an image detection model.

[0042] The present invention has the following beneficial effects:

[0043] The present invention provides a participation-adaptive federated learning method and system for medical image analysis. Guided by the final effect of the analysis model, it dynamically optimizes the participation of each institution in the analysis model development process, fully leverages the value of each institution, and enables the analysis model to achieve optimal results during the adaptive adjustment of each institution. Without leaving the local data, it ensures that the recognition and diagnosis algorithms based on large-scale medical images can be executed efficiently and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of a participation-adaptive federated learning method for medical image analysis according to the present invention;

[0045] Figure 2 This is a principle framework diagram of the central server and various medical institutions using the method of the present invention. DETAILED DESCRIPTION

[0046] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solutions of the present invention, but the present invention is not limited to these embodiments.

[0047] In the process of analyzing medical images, it is necessary to obtain large-scale data to develop analysis models based on deep learning algorithms. This large-scale data often comes from multiple medical institutions. Since medical image data involves patient privacy and medical staff's diagnosis records, each medical institution often does not share this data. At the same time, the diagnostic level and experience of medical staff in each medical institution are different, and the diagnostic results are different. In particular, the development levels of various regions in my country vary greatly, and there are many differences in the quantity, quality, and type of data from each institution. Based on the differences between various medical institutions and the characteristics of data being limited to localization, the present invention proposes a participation-adaptive federated learning method for medical image analysis. When the data does not leave the local area, it can be guided by the final effect of the analysis model to dynamically optimize the participation of each institution in the analysis model development process, so as to construct an analysis model that solves the problem of data heterogeneity and facilitates subsequent targeted medical analysis.

[0048] like Figure 1 , a participation-adaptive federated learning method for medical image analysis, the method comprising:

[0049] Step S01: Send the model parameters of the analysis model to multiple medical institutions , so that medical institutions can update model parameters locally;

[0050] Step S02: Receive the gradients sent back by multiple medical institutions after the model parameters are updated. , and the model parameters of the analysis model are updated for the first time according to the following formula:

[0051]

[0052] Among them, K is the number of medical institutions participating in the training, is the gradient of k from the medical institution, η is the learning rate, which is generally set to 0.01~0.05, is a scalar that controls the medical institution's participation in this round of training;

[0053] Step S03, the model parameters are The loss function of the analysis model is constructed as Differentiable functions , and for this function Derivative calculation The gradient value ; Then use renew to get updated engagement ;

[0054] Step S04: Based on the updated participation rate, the model parameters of the analysis model are updated again according to the following formula:

[0055] ;

[0056] Step S05: Send the updated model parameters of the analysis model to multiple medical institutions. , repeat the above steps S01 to S05 until the loss function of the analysis model converges.

[0057] This method is applied to the central server, which executes the above process. Figure 2 The central server communicates with multiple medical institutions, sending model parameters of the analysis model to them and receiving gradients sent back by the medical institutions. The medical institutions do not communicate with each other, but each updates its own model parameters locally. Based on the updated model parameters, it calculates gradients and sends them back to the central server.

[0058] In step S01, the model parameters of the analysis model are The central server distributes the model parameters to each medical institution. The model parameters for the initial distribution are pre-configured by the central server before step S01. The configuration is performed based on the type of analysis model; for example, different model parameters are configured based on the analysis purpose, such as image detection, image classification, or image segmentation.

[0059] After receiving the model parameters from the central server, medical institutions perform local updates. The specific update process is based on the characteristics of the classification model and local data. For example, when the analysis model uses a neural network to segment and detect intracranial hemorrhage in CT images, the hemorrhage area is annotated on the locally acquired CT images, and a training dataset is constructed. The training data in the training dataset is then input into the analysis model for training. After training, the updated model parameters and gradient values are obtained.

[0060] Among them, the gradient transmitted back from the medical institution The acquisition process is as follows: The medical institution locally calculates the derivative of the analysis model's loss function to obtain the gradient. For example, when the analysis model uses a neural network for intracranial hemorrhage segmentation detection in CT images, the loss function uses a hybrid loss function composed of cross-entropy and dice. Based on the calculated loss value, the gradient of the last layer of neurons is derived. The gradient of the neurons in the previous layers is then calculated using the chain rule. The gradient formula is the derivative formula obtained using the chain rule. For example, for F(H(x)), the gradient of the F-layer neurons is dF / dH·dH / dx. The gradient descent algorithm is then used to update the values of all neurons in the network, continuously decreasing the loss value. This means that the model's predicted value and the labeled value of the training data are continuously approached, ultimately obtaining the optimal network parameters. The training process is repeated locally until the gradient stops decreasing, marking the end of the local update of the analysis model parameters. At this point, the gradient that no longer decreases is transmitted from the medical institution back to the central server.

[0061] After the central server receives the returned gradients, step S02 is executed, where the model parameters are uniformly updated for the first time. During this update, the participation level of each medical institution is the same as that after the previous cycle. The participation level for the initial update is pre-set before step S01. For example, the participation level of each medical institution is set to the same value, such as 1.

[0062] In step S02, the model parameters uniformly updated by the central server are obtained. After that, the step S03 updates the participation. Specifically, the step S03 includes:

[0063] Step S31: Input the test data set into the model parameter The analysis model is built and the loss function of the analysis model is constructed;

[0064] Step S32, setting the above loss function to A differentiable function, denoted by ;

[0065] Step S33, according to the formula , calculated to obtain The gradient value ;

[0066] Step S34: After normalization, Update and get .

[0067] The test data is constructed before step S01. The method also includes step S00', before step S01, constructing part of the data in the data set under the application scenario as a test data set. The application scenario varies according to the application purpose of the analysis model. For example, when the application scenario is an intracranial hemorrhage segmentation detection scenario of CT images, the brain CT image data is constructed on the central server, and the hemorrhage area of the image data is marked according to the historical diagnosis. The image data that can be analyzed is obtained through image processing methods and constructed as a test data set. The test data set is a small-scale data set, which is used to measure the effect of the model parameters. The performance of the time, and the optimization is derived by the loss value of the performance .

[0068] In the pair After normalization, it can be used to update ,get = - Through this update method, if the organization The gradient used to update the model does not contribute much to the final performance in this round, and its corresponding will be lowered, otherwise it will be raised.

[0069] After obtaining the updated participation, the central server executes step S04 based on the model parameters set in step S01. Based on the gradient of the local update returned by the medical institution and the participation obtained in step S03, the model parameters are updated again to obtain The model parameters are distributed to the medical institution, which then performs local updates and gradient feedback after receiving the parameters. The present invention repeats the above steps S01 to S05 until the loss function of the analysis model converges on the test data set.

[0070] Based on the above, the method of the present invention is applicable to analysis models of any type (e.g., image classification, segmentation, or detection) and any model architecture (e.g., convolutional neural network or Transformer). The present invention primarily performs unified updates of model parameters on the central server side. During this update, model parameters are updated uniformly based on the results of local data updates by the medical institutions, without requiring direct access to data from the medical institutions themselves. Furthermore, the method considers the differences between medical institutions and dynamically updates participation based on local update results. This ensures data specificity for each medical institution and accurately reflects the differences in their contribution to the analysis. The examples presented herein are intended only to illustrate how the above process is implemented and are not limited to this model. The present invention is applicable to all types of analysis models for medical imaging. Based on the final model performance, the method dynamically optimizes the weight of participation of different institutions in training, enabling diverse image analysis methods to achieve good results using this framework.

[0071] Based on the above method, the present invention also provides a participation-adaptive federated learning system for medical image analysis, which is applied to a central server. Specifically, the system includes a model parameter sending module, a model parameter receiving module, a model parameter updating module, and a participation updating module.

[0072] The model parameter sending module is used to send the model parameters of the analysis model to multiple medical institutions so that the medical institutions can locally update the model parameters.

[0073] The model parameter receiving module is used to receive the gradients sent back by multiple medical institutions after the model parameters are updated. The model parameter updating module is used to perform the first update on the model parameters of the analysis model according to the following formula: Among them, K is the number of medical institutions participating in the training, is the gradient from the medical institution side k, η is the learning rate, It is a scalar that controls the medical institution's participation in this round of training.

[0074] The participation update module is used to set the model parameters to The loss function of the analysis model is constructed as Differentiable functions , and for this function Derivative calculation The gradient value ; Then use renew to get updated engagement .

[0075] The model parameter updating module is further configured to update the model parameters of the analysis model again according to the following formula based on the updated participation degree: The model parameter sending module is also used to send the updated model parameters of the analysis model to multiple medical institutions. , repeat the above-mentioned local model parameter update, model parameter first update, participation update, and model parameter re-update process until the loss function of the analysis model converges.

[0076] Those skilled in the art will appreciate that the embodiments of the present invention described above and shown in the accompanying drawings are intended to be illustrative only and are not intended to limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and illustrated in the embodiments. Any variations or modifications may be made to the embodiments of the present invention without departing from the principles described.

Claims

1. A participation-adaptive federated learning method for medical image analysis, applied to a central server, characterized in that: Methods include: Step S01: Send the model parameters of the analysis model to multiple medical institutions , so that medical institutions can update model parameters locally; Step S02: Receive the gradients sent back by multiple medical institutions after the model parameters are updated. , and the model parameters of the analysis model are updated for the first time according to the following formula: Among them, K is the number of medical institutions participating in the training, is the gradient from the medical institution side k, η is the learning rate, is a scalar that controls the medical institution's participation in this round of training; Step S03, the model parameters are The loss function of the analysis model is constructed as Differentiable functions , and for this function Derivative calculation The gradient value ; Then use renew to get updated engagement ; Step S03 includes: Step S31: Input the test data set into the model parameter The analysis model is built and the loss function of the analysis model is constructed; Step S32, setting the above loss function to A differentiable function, denoted by ; Step S33, according to the formula , calculated to obtain The gradient value ; Step S34: After normalization, Update and get ; Step S04: Based on the updated participation rate, the model parameters of the analysis model are updated again according to the following formula: ； Step S05: Send the updated model parameters of the analysis model to multiple medical institutions. , repeat the above steps S01 to S05 until the loss function of the analysis model converges.

2. The participation-adaptive federated learning method for medical image analysis according to claim 1, characterized in that: The method further includes step S00, before step S01, presetting the participation degree of each medical institution for the first updating process of the model parameters.

3. The participation-adaptive federated learning method for medical image analysis according to claim 1, characterized in that: The gradient of the backhaul from the medical institution The acquisition process is as follows: the medical institution locally calculates the loss function of the analysis model and then obtains the gradient .

4. The participation-adaptive federated learning method for medical image analysis according to claim 1, characterized in that: The method further includes step S00', before step S01, constructing part of the data in the data set under the application scenario as a test data set for use in the loss function calculation of steps S03 and S05.

5. The participation-adaptive federated learning method for medical image analysis according to claim 1, characterized in that: The analysis model is a convolutional neural network model or a Transformer model.

6. The participation-adaptive federated learning method for medical image analysis according to claim 1, characterized in that: The analysis model is an image classification model, an image segmentation model, or an image detection model.

7. A participation-adaptive federated learning system for medical image analysis, applied to a central server, characterized in that the system include: Model parameter sending module, used to send model parameters of analysis models to multiple medical institutions , so that medical institutions can update model parameters locally; Model parameter receiving module, used to receive the gradients sent back by multiple medical institutions after the model parameters are updated ; The model parameter update module is used to update the model parameters of the analysis model for the first time according to the following formula: Among them, K is the number of medical institutions participating in the training, is the gradient from the medical institution side k, η is the learning rate, is a scalar that controls the medical institution's participation in this round of training; Participation update module, which is used to update the model parameters to The loss function of the analysis model is constructed as Differentiable functions , and for this function Derivative calculation The gradient value ; Then use renew to get updated engagement ; The participation update module obtains the updated participation The process includes: The test data set is input into the model parameter The analysis model is built and the loss function of the analysis model is constructed; Set the above loss function to A differentiable function, denoted by ; According to the formula , calculated to obtain The gradient value ; right After normalization, Update and get ; The model parameter updating module is further configured to update the model parameters of the analysis model again according to the following formula based on the updated participation degree: ； The model parameter sending module is also used to send the updated model parameters of the analysis model to multiple medical institutions. , repeat the process of updating local model parameters, updating model parameters for the first time, updating participation, and updating model parameters again until the loss function of the analysis model converges.

8. The participation-adaptive federated learning system for medical image analysis according to claim 7, characterized in that: The analysis model is a convolutional neural network model or a Transformer model.

9. The participation-adaptive federated learning system for medical image analysis according to claim 7, characterized in that: The analysis model is an image classification model, an image segmentation model, or an image detection model.

Images