Federal learning-based MRI image segmentation model training method and related device

Abstract

The embodiment of the invention discloses an MRI (Magnetic Resonance Imaging) image segmentation model training method based on federated learning and a related device. Each client trains the initial segmentation model to obtain a model parameter difference value, the server aggregates the model parameter difference values sent by the plurality of clients according to the aggregation weight of each client to obtain an aggregation gradient, updates the initial segmentation model according to the aggregation gradient to obtain an intermediate-state segmentation model, and obtains the intermediate-state segmentation model; and taking the intermediate-state segmentation model as an initial segmentation model, carrying out multiple iterations again, and taking the intermediate-state segmentation model obtained by the last iteration as a target segmentation model. A federal learning multi-modal segmentation framework is adopted, each client uses a local multi-modal MRI image to train a segmentation model, local data is not leaked, the privacy of the local data is protected, a global model is obtained through interaction with a central server, the global model has higher generalization ability compared with a local model, and the data island phenomenon is broken technically.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and specifically to a federated learning-based MRI image segmentation model training method and related devices. Background Art

[0002] Brain tumors are a growing concern worldwide, with their incidence steadily increasing, posing a serious threat to human quality of life. In clinical treatment, multimodal magnetic resonance imaging (MR) is widely used because it provides high-definition images of internal structures and tissues. However, image segmentation of brain tumors relies heavily on manual labor from experts, which is not only time-consuming and labor-intensive, but also difficult to ensure accurate and consistent segmentation. Therefore, automated brain tumor segmentation based on artificial intelligence (AI) is an important area of ​​research in computer-assisted brain tumor surgery.

[0003] In the medical field, large amounts of diagnostic and treatment data covering different cases of similar diseases and different pathologies of the same patient are often distributed across different medical institutions. However, due to the sensitive nature of patient data, cross-institutional data collection and integration is difficult. On the one hand, integrating data across institutions for model training requires sharing medical data, which raises concerns about medical data privacy, making centralized processing and training using MR datasets unfeasible. On the other hand, if only medical data from a single medical institution is used for model training, the large amount of clinical data covering different cases of similar diseases and different conditions of the same patient is distributed across different institutions. Models trained using medical data from a single hospital or institution will not generalize well, creating the problem of data silos. Summary of the Invention

[0004] The embodiments of the present application provide a federated learning-based MRI image segmentation model training method and related devices. By using a federated learning multimodal segmentation framework, the global model has stronger generalization capabilities than the local model without leaking local data and protecting local data privacy, thereby realizing AI collaboration of medical big data.

[0005] In a first aspect, an embodiment of the present application provides a method for training an MRI image segmentation model based on federated learning, the method being applied to a server and comprising:

[0006] Sending the initial segmentation model to each client, so that the client trains the initial segmentation model based on a local multimodal MRI image training set, and obtains updated model parameters of the initial segmentation model based on the model training;

[0007] receiving a model parameter difference between the updated model parameter sent by each of the clients and the initial model parameter of the initial segmentation model;

[0008] Determine an aggregation weight for each of the clients, where the aggregation weight is used to represent the weight of the model parameter difference sent by the client in participating in model parameter aggregation;

[0009] Aggregating the model parameter differences sent by multiple clients according to the aggregation weight of each client to obtain an aggregated gradient;

[0010] updating the model parameters of the initial segmentation model according to the aggregated gradient to obtain an intermediate segmentation model;

[0011] The intermediate segmentation model is used as the initial segmentation model, and the initial segmentation model is sent to each client again until the model training meets the preset conditions to determine that the model converges, and the intermediate segmentation model obtained by the last model parameter update is used as the target segmentation model.

[0012] A second aspect of an embodiment of the present application provides a method for training an MRI image segmentation model based on federated learning, the method being applied to a client and comprising:

[0013] Receive the initial segmentation model sent by the server, train the initial segmentation model based on a local multimodal MRI image training set, and obtain updated model parameters of the initial segmentation model based on the model training;

[0014] Determining a model parameter difference between the updated model parameters and the initial model parameters of the initial segmentation model, and sending the model parameter difference to the server, so that the server aggregates the model parameter differences sent by multiple clients according to an aggregation weight of each client to obtain an aggregation gradient, where the aggregation weight is used to represent the weight of the model parameter differences sent by the clients participating in model parameter aggregation, and updating the model parameters of the initial segmentation model according to the aggregation gradient to obtain an intermediate segmentation model;

[0015] Receive the intermediate segmentation model sent by the server, use the intermediate segmentation model as the initial segmentation model, and return to execute the local multimodal MRI image training set to train the initial segmentation model.

[0016] A third aspect of an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method of the first aspect or the second aspect when executing the computer program.

[0017] A fourth aspect of an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method of the first aspect.

[0018] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0019] Each client trains the initial segmentation model to obtain the model parameter difference. The server aggregates the model parameter differences sent by multiple clients according to the aggregation weight of each client to obtain the aggregate gradient. The model parameters of the initial segmentation model are updated according to the aggregate gradient to obtain the intermediate segmentation model. The intermediate segmentation model is used as the initial segmentation model, and the execution returns to send the initial segmentation model to each client until the model training meets the preset conditions. The model converges and the intermediate segmentation model obtained from the last model parameter update is used as the target segmentation model. The federated learning multimodal segmentation framework is adopted. Each client can not only use local multimodal MRI images to train the segmentation model, but also interact with the central server to obtain a global model without leaking local data and protecting local data privacy. The global model has stronger generalization capabilities than the local model, technically breaking the data island phenomenon and realizing AI collaboration of medical big data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a method for training an MRI image segmentation model based on federated learning in an embodiment of the present application is provided;

[0021] Figure 2 This is an exemplary schematic diagram of the multi-round iterative training process of the model on the server side in an embodiment of the present application;

[0022] Figure 3 This is another flowchart of the MRI image segmentation model training method based on federated learning in an embodiment of the present application;

[0023] Figure 4 This is a schematic diagram of a scenario in which a client trains an initial segmentation model based on a multimodal MRI image training set in an embodiment of the present application;

[0024] Figure 5 This is an exemplary schematic diagram of the process of multiple rounds of iterative training of the local initial segmentation model by the client in an embodiment of the present application;

[0025] Figure 6 This is a structural diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The embodiments of the present application provide a federated learning-based MRI image segmentation model training method and related devices. By using a federated learning multimodal segmentation framework, the global model has stronger generalization capabilities than the local model without leaking local data and protecting local data privacy, thereby realizing AI collaboration of medical big data.

[0027] The following describes the MRI image segmentation model training method based on federated learning in the embodiment of the present application:

[0028] See also Figure 1 In the embodiment of the present application, an embodiment of the MRI image segmentation model training method based on federated learning includes:

[0029] 101. Sending an initial segmentation model to each client, so that the client trains the initial segmentation model based on a local multimodal MRI image training set, and obtains updated model parameters of the initial segmentation model based on the model training;

[0030] The method of this embodiment can be applied to a server. The server can send an initial segmentation model to each client. Each client then trains the initial segmentation model based on a local multimodal MRI image training set. The training process of the initial segmentation model is also a process of updating the model parameters. Therefore, the updated model parameters of the model can be obtained based on the training of the initial segmentation model. The updated model parameters are the model parameters updated by the initial segmentation model during model training.

[0031] The initial segmentation model is used to segment MRI images, for example, to segment tumor regions in MRI images. The tumor regions may be the entire tumor region, the tumor core region, the enhanced tumor region, or other regions. MRI images are magnetic resonance imaging (MRI). For example, a multimodal MRI image may be a multimodal MRI image of the brain. The multimodal MRI image may include MRI images of multiple modalities, such as FLAIR, T1, T1ce, and T2.

[0032] In this embodiment, the client can be a client of various medical institutions, such as hospitals, health institutions, etc. that can perform MRI imaging and obtain MRI images. These medical institutions can use the client to use the locally obtained MRI images as a training set to train the initial segmentation model.

[0033] 102. Receive a model parameter difference between the updated model parameter sent by each client and the initial model parameter of the initial segmentation model;

[0034] When each client completes a model update and obtains updated model parameters, it determines the model parameter difference between the updated model parameters and the initial model parameters of the initial segmentation model. The initial model parameters are the model parameters of the initial segmentation model before model training, while the updated model parameters are the model parameters obtained after the initial segmentation model training is completed. Therefore, the model parameter difference obtained by subtracting the updated model parameters from the initial model parameters can reflect the training status of the model.

[0035] The client may send the model parameter difference to the server, and the server receives the model parameter difference sent by each client.

[0036] 103. Determine an aggregation weight for each client, where the aggregation weight is used to represent the weight of the model parameter difference sent by the client in participating in model parameter aggregation;

[0037] The server can aggregate the model parameter differences sent by multiple clients to comprehensively summarize the model training results of multiple clients and update the model parameters of the initial segmentation model based on the aggregated results. Therefore, the server can determine an aggregation weight for each client. This aggregation weight is used to represent the weight of the model parameter differences sent by the client in the model parameter aggregation. In other words, this aggregation weight represents the degree of influence of the client's model training results on the update of the server's initial segmentation model. The larger the aggregation weight, the greater the influence of the client's model training results on the update of the server's initial segmentation model; conversely, the smaller the influence.

[0038] 104. Aggregate the model parameter differences sent by multiple clients according to the aggregation weight of each client to obtain an aggregated gradient;

[0039] The server can aggregate the model parameter differences sent by multiple clients according to the aggregation weight of each client to obtain an aggregate gradient, which represents the aggregate result of the model training results of multiple clients.

[0040] 105. Update the model parameters of the initial segmentation model according to the aggregated gradient to obtain an intermediate segmentation model;

[0041] After obtaining the aggregate gradient, the server can update the model parameters of the initial segmentation model on the server side according to the aggregate gradient. The segmentation model obtained by this operation can be called an intermediate segmentation model, which means that the intermediate state of the segmentation model is obtained by updating the model parameters of the initial segmentation model.

[0042] 106. Using the intermediate segmentation model as the initial segmentation model, returning to executing the sending of the initial segmentation model to each client until the model training meets a preset condition and determining that the model has converged, and using the intermediate segmentation model obtained by the last model parameter update as the target segmentation model;

[0043] To perform multiple iterative updates of the segmentation model, the intermediate segmentation model obtained can be used as the initial segmentation model, and the step of sending the initial segmentation model to each client is returned. Each client then trains the initial segmentation model again to obtain model parameter differences and returns the model parameter differences to the server. The server then executes steps 102 to 105 and again sends the intermediate segmentation model obtained by executing step 105 as the initial segmentation model to each client. The client then trains the initial segmentation model again to obtain model parameter differences... This cycle repeats, and the client and server perform multiple rounds of iterative updates on the initial segmentation model until the model training meets the preset conditions and determines that the model has converged. The server then uses the intermediate segmentation model obtained from the last model parameter update as the target segmentation model to complete model training. The target segmentation model can be applied to the segmentation of target objects in MRI images, such as the segmentation of tumor areas in MRI images.

[0044] Among them, the model training meets the preset conditions, which can be that the number of iterations of the model training reaches a preset number, or the loss value of the loss function of the model training reaches a preset threshold, or the loss value of the loss function of the model training tends to be stable, which is not limited here.

[0045] In this embodiment, each client trains the initial segmentation model to obtain the model parameter difference. The server aggregates the model parameter differences sent by multiple clients according to the aggregation weight of each client to obtain the aggregation gradient. The model parameters of the initial segmentation model are updated according to the aggregation gradient to obtain the intermediate segmentation model. The intermediate segmentation model is used as the initial segmentation model, and the execution returns to send the initial segmentation model to each client until the model training meets the preset conditions. The model convergence is determined, and the intermediate segmentation model obtained from the last model parameter update is used as the target segmentation model. The federated learning multimodal segmentation framework is adopted. Each client can not only use local multimodal MRI images to train the segmentation model, but also interact with the central server to obtain a global model without leaking local data and protecting local data privacy. The global model has stronger generalization capabilities than the local model, technically breaking the data island phenomenon and realizing AI collaboration of medical big data.

[0046] based on Figure 1In the illustrated embodiment, in a preferred implementation manner, in the process of aggregating the model parameter differences sent by multiple clients to obtain an aggregated gradient, the model parameter differences sent by multiple clients can be aggregated according to the aggregation weight of each client to obtain an initial aggregated gradient, and the server obtains a preset gradient clipping boundary and clips the initial aggregated gradient according to the gradient clipping boundary to obtain an aggregated gradient.

[0047] based on Figure 1 In another preferred implementation of the embodiment shown, when updating the model parameters of the initial segmentation model according to the aggregated gradient, Gaussian noise can be specifically obtained, and the model parameters of the initial segmentation model are updated according to the aggregated gradient and Gaussian noise to obtain an intermediate segmentation model.

[0048] Based on the above embodiments and their preferred implementations, when the model parameter differences sent by multiple clients are aggregated according to the aggregation weight of each client, a preset aggregation probability can be obtained, as well as the sum of the aggregation weights of multiple clients. The aggregation probability is used to represent the probability that the model parameter difference obtained by the client is sent to the server, and the model parameter differences sent by multiple clients can be aggregated based on the aggregation probability, the sum of the aggregation weights of multiple clients, and the aggregation weights of multiple clients.

[0049] When aggregating the model parameter differences sent by multiple clients based on the aggregation probability, the sum of the aggregation weights of the multiple clients, and the aggregation weights of the multiple clients, the product of the model parameter difference sent by each client and the aggregation weight of the client can be calculated, and the products of the multiple clients can be summed. The ratio of the summed value to the sum of the aggregation probability and the aggregation weight is then calculated to obtain the aggregate gradient.

[0050] For example, the server can receive the model training results sent by each client, that is, the model parameter difference, and perform operations such as aggregation, clipping, and adding noise on the model parameter differences sent by multiple clients to update the local global initial segmentation model to integrate the different knowledge obtained from the model training of multiple clients. The gradient clipping boundary C, the noise parameter σ, and the number of training samples n used by the Kth client can be pre-set. K , the upper limit of the number of samples used by each user

[0051] Among them, the number of training samples used by the K-th client is n KThe number of training samples used by each client can be determined based on the ratio of the training datasets of multiple clients. This ratio is the ratio of the MRI image training dataset used for model training to the total MRI image dataset. Specifically, the number of training samples used by each client can be calculated by multiplying the total MRI image dataset by this ratio. The training dataset ratio can be obtained by randomly sampling from a preset distribution sequence. For example, to simulate real-world conditions, a Dirichlet distribution can be used for random sampling, as it approximates a realistic distribution.

[0052] The gradient clipping boundary C can be a manually defined hyperparameter. The upper limit of the number of samples used by each user refers to the upper limit of the number of samples used by each user when each client can only use a certain number of MRI images of each patient for local model training.

[0053] After obtaining the above parameters, the model parameters of the initial segmentation model can be randomly initialized to obtain the global initial segmentation model Θ t . And, define the aggregation weight of the Kth client And define the sum of the aggregate weights of multiple clients W = ∑ k W K .

[0054] Initialize the model parameters to get the global initial segmentation model Θ t Afterwards, the local global initial segmentation model can be transmitted to each client, and the client receives the global model Θ from the central server. t As the local model θ K (θ K Refers to the model parameters of the client's local initial segmentation model). Each client performs model training to obtain the updated model parameters θ′ of the model K (θ′ K Refers to the model parameters updated after the initial segmentation model training) and calculates the model parameter difference Δ K =θ′ K -θ K , and transmit it to the central server, so that the server obtains the model parameter difference Δ sent by multiple clients K .

[0055] After collecting the local training results of multiple clients, that is, the model parameter difference Δ K Afterwards, the model parameter differences sent by multiple clients can be aggregated to obtain the initial aggregate gradient Δ t The calculation can be expressed as follows:

[0056]

[0057] Among them, ct Refers to all clients; q refers to the client's aggregate probability, that is, the client's local training result (that is, the model parameter difference Δ K ) will have a probability of q percent being selected and transmitted to the central server for aggregation to further protect the client's data privacy.

[0058] After obtaining the initial aggregation gradient Δ t Afterwards, the initial aggregation gradient can be clipped. Specifically, the calculation of the aggregation gradient obtained by clipping the initial aggregation gradient can be expressed as follows:

[0059]

[0060] After clipping the initial aggregate gradient to obtain the aggregate gradient, the global initial segmentation model on the server side can be updated based on the aggregate gradient. When updating the global parameters of the initial segmentation model, Gaussian noise can be added to further protect the client's data privacy. Specifically, the global model parameter update of the global initial segmentation model on the server side can be expressed as follows:

[0061] Θ t+1 =Θ t +Δ t +N(0,Iσ 2 );

[0062] Among them, N(0, Iσ 2 ) indicates that the variance is σ 2 The added Gaussian noise is sampled from a normal distribution. Therefore, by adding Gaussian noise when updating the global model parameters and updating the global model parameters based on the aggregated gradient and Gaussian noise, the model parameters for the current iteration can be updated. After the above operations, the tth global iteration is completed.

[0063] like Figure 2 As shown in the model iterative training flowchart, after completing the tth iteration to obtain the intermediate segmentation model, this intermediate segmentation model can be sent to the client as the initial segmentation model. The client trains the initial segmentation model again and returns the model parameter difference to the server. The server then performs the above operations based on the model parameter difference sent by each client to complete the t+1th global iteration... and so on, until the number of iterations reaches the preset T times, then the model iterative training is stopped to obtain the target segmentation model, thereby realizing multiple rounds of iterative updates of the initial segmentation model.

[0064] Therefore, this embodiment applies the federated learning mechanism to train the global segmentation model of the server. Federated learning technology can realize collaborative learning among multiple parties while ensuring that the original data is not output locally. Only model parameters are transmitted during the model training process, allowing medical institutions to use medical data shared by multiple parties to train models with strong generalization capabilities without exposing the original data, effectively solving the problems of data privacy protection and data silos.

[0065] The following will be Figure 1 Based on the embodiment shown, the embodiment of the present application is further described in detail. Figure 3 Another embodiment of the MRI image segmentation model training method based on federated learning in the embodiment of the present application includes:

[0066] 301. Receive an initial segmentation model sent by a server, train the initial segmentation model based on a local multimodal MRI image training set, and obtain updated model parameters of the initial segmentation model based on the model training;

[0067] 302. Determine a model parameter difference between the updated model parameters and the initial model parameters of the initial segmentation model, and send the model parameter difference to the server, so that the server aggregates the model parameter differences sent by multiple clients according to an aggregation weight of each client to obtain an aggregation gradient, where the aggregation weight is used to represent the weight of the model parameter differences sent by the clients in model parameter aggregation, and update the model parameters of the initial segmentation model according to the aggregation gradient to obtain an intermediate segmentation model.

[0068] 303. Receive the intermediate segmentation model sent by the server, use the intermediate segmentation model as the initial segmentation model, and return to execute the training of the initial segmentation model based on the local multimodal MRI image training set;

[0069] The method of this embodiment can be applied to the client, and the client can obtain a local multimodal MRI image training set. The training set can be a public brain tumor dataset BraTS2021 divided into multiple datasets according to Dirichlet distribution sampling, and some of the datasets are used as the local training set of the client.

[0070] Among them, the client trains the initial segmentation model based on the local multimodal MRI image training set. Specifically, the client's training data set ratio is obtained, and the training data set ratio is used to represent the ratio of the client's local MRI image training set used for model training to the total MRI image data set. The training data set ratio is obtained by random sampling from a preset distribution sequence, and then the local multimodal MRI image training set can be obtained according to the training data set ratio, and the initial segmentation model is trained based on the local multimodal MRI image training set.

[0071] Specifically, the client can divide the data set and build a federated learning training scenario. For example, suppose there are N medical institutions, X K represents the dataset of the kth institution, W K Represents X K The proportion of the total data set (i.e. the above-mentioned training data set proportion), and set the vector of the proportion of multiple training data sets of the clients corresponding to N medical institutions W is a set of vectors randomly sampled from a Dirichlet distribution, i.e., W~Dir(W|d). In order to ensure that the data set allocated to each institution is not too low, in this embodiment, α can be set to a value greater than or equal to 1, for example, to 2.

[0072] Taking the Kth client (medical institution) as an example, it receives the global model Θ from the central server t As the local model θ K , each client transmits the model parameter difference to the central server Δ K =θ′ K -θ K , where θ′ K are the model parameters updated after local model training.

[0073] like Figure 4 The figure shows a scenario in which the client trains an initial segmentation model based on a multimodal MRI image training set. As shown in the figure, the initial segmentation model can be composed of a U-NET network and a transformer. r Network composition: During model training, multimodal MRI brain tumor images are used as model input. The input images are extracted from the U-NET network encoder's multi-layer convolutional neural network to extract 5 levels of features. The highest level features are used for upsampling in the decoder. The transformer is used to fuse the features of the other 4 levels and sequentially concatenate them with the upsampled feature matrices of the four stages. Finally, two convolution operations and an argmax function are performed to obtain the segmentation mask. The loss function is constructed based on the segmentation mask and the groundtruth labels of the multimodal MRI image training set. For example, the loss function can be defined as:

[0074]

[0075] Among them, F(x) is the model output, Y(x) is the groundtruth label, and Ω represents the image pixel area, which refers to each pixel of the MRI image.

[0076] Of course, the above model training process is just an example. In fact, the client can also train the initial segmentation model based on other deep learning algorithms, and the initial segmentation model can also be other image segmentation network model structures, not limited to the above model structure, and is not specifically limited here.

[0077] based on Figure 3 In the embodiment shown, in a preferred implementation manner, the model parameter difference between the updated model parameters and the initial model parameters of the initial segmentation model is determined. The specific method can be to perform gradient descent on the initial model parameters of the initial segmentation model according to a preset learning rate and loss function to obtain gradient descent parameters, and perform gradient clipping on the gradient descent parameters according to a preset a priori upper bound to obtain gradient clipping parameters, and calculate the difference between the gradient clipping parameters and the initial model parameters to obtain the model parameter difference.

[0078] For example, the initial model parameters of the initial segmentation model are gradient-decreased according to the preset learning rate and loss function to obtain the gradient descent parameters, which can be expressed as follows:

[0079]

[0080] Among them, θ′ K is the parameter after gradient descent, θ K is the initial model parameter of the initial segmentation model, ηη is the pre-set learning rate, is the gradient (derivative), and L is the loss function.

[0081] After determining the gradient descent parameter, the gradient descent parameter can be further clipped according to the preset a priori upper bound to obtain the gradient clipping parameter, which can be expressed as follows:

[0082]

[0083] Where S is |Δ K The a priori upper bound of |, that is, |Δ determined based on human experience K The upper limit value of | is a hyperparameter.

[0084] To further protect privacy, the local training results of the client of the medical institution Δ KThere is a q% probability that the data will be selected and transmitted to the central server. In addition, in the actual deployment process, before training initialization, each client of the medical structure will have a q% probability of performing local training to save training resources.

[0085] Gradient descent is the method used in deep learning to calculate gradients. The model updates its parameters based on the calculated gradients. After the gradients are calculated, the parameters are not updated directly locally. Instead, the gradients are clipped before being transmitted to the server. Gradient clipping ensures that there are no extreme values ​​or outliers that could severely distort the results. Furthermore, clipping limits the maximum sensitivity of the model output relative to a single data point, allowing for the addition of less noise while still maintaining the desired level of data privacy.

[0086] For example, Figure 5 The figure shows the process of iterative training of a local initial segmentation model by a client. As shown in the figure, the client obtains local MRI images and performs data preprocessing, including data cleaning and data augmentation. It then uses the local MRI images to iteratively train the global initial segmentation model received from the server, performing gradient descent and gradient clipping on the model parameters to complete the jth iteration. This process continues until a total of J iterations have been completed, resulting in updated model parameters. The updated model parameters are then subtracted from the initial model parameters to obtain model parameter differences. These model parameter differences are then transmitted to the server, which aggregates the model parameter differences sent by multiple clients and updates the global initial segmentation model.

[0087] Therefore, an embodiment of the present application discloses a brain tumor segmentation framework based on multimodal MRI images and federated learning. First, the public brain tumor dataset is divided into several datasets as local training sets for the client according to Dirichlet distribution sampling. Secondly, each client uses the divided local dataset for training, and then calculates the gradient and trims it. Thirdly, after all clients complete local training, the federal server will randomly select the local models of several clients, collect the gradients and perform weighted aggregation. The aggregated gradients will be further trimmed and Gaussian noise will be added to obtain a global federal model. Then, the federal server distributes the aggregated model to each client, and allows each client to continue a new round of local training based on the aggregated model.

[0088] Through federated multimodal learning across multiple medical institutions, each institution gains knowledge about different cases and the varying circumstances of similar cases at other institutions without sharing local data, enabling the model to generalize more effectively. Federated learning's decentralized training approach and centralized differential privacy technology protect the data privacy of medical institutions.

[0089] Therefore, the main feature of the embodiment of the present application is that each client (medical institution) performs local model training using a local data set, communicates only with the federated server, and transmits model gradient differences rather than local raw data. The federated server performs weighted fusion and clipping on the collected model gradients and adds noise, thereby improving the generalization ability of the global model while protecting the privacy of patient data. The model takes advantage of federated learning and updates parameters by clipping parameters and adding noise, rather than directly exchanging raw data, thereby protecting data privacy.

[0090] The computer device in the embodiment of the present application is described below. Figure 6 In one embodiment of the present application, a computer device includes:

[0091] The computer device 600 may include one or more central processing units (CPUs) 601 and a memory 605 . The memory 605 stores one or more application programs or data.

[0092] Memory 605 may be volatile or persistent storage. The program stored in memory 605 may include one or more modules, each of which may include a series of instruction operations on the computer device. Furthermore, CPU 601 may be configured to communicate with memory 605 and execute the series of instruction operations in memory 605 on computer device 600.

[0093] The computer device 600 may also include one or more power supplies 602, one or more wired or wireless network interfaces 603, one or more input and output interfaces 604, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0094] The CPU 601 can execute the aforementioned Figure 1 or Figure 3 The operations performed by the server or client in the illustrated embodiment will not be described in detail here.

[0095] The present application also provides a computer storage medium, wherein one embodiment includes: the computer storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the aforementioned Figure 1 or Figure 3 Operations performed by the server or client in the illustrated embodiment.

[0096] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0098] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0099] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0100] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A method for training an MRI image segmentation model based on federated learning, characterized in that: The method is applied to a server, and the method comprises: Sending an initial segmentation model to each client, so that the client trains the initial segmentation model based on a local multimodal MRI image training set, and obtains updated model parameters of the initial segmentation model based on the model training; receiving a model parameter difference between the updated model parameter sent by each of the clients and the initial model parameter of the initial segmentation model; Determine an aggregation weight of each of the clients, where the aggregation weight is used to represent the weight of the model parameter difference sent by the client participating in the model parameter aggregation; Aggregating the model parameter differences sent by the multiple clients according to the aggregation weight of each client to obtain an aggregate gradient; updating the model parameters of the initial segmentation model according to the aggregated gradient to obtain an intermediate segmentation model; The intermediate segmentation model is used as the initial segmentation model, and the initial segmentation model is sent to each client again, until the model training meets the preset conditions and the model converges, and the intermediate segmentation model obtained by the last model parameter update is used as the target segmentation model.

2. The method according to claim 1, characterized in that The aggregating the model parameter differences sent by the plurality of clients according to the aggregation weight of each client to obtain an aggregate gradient includes: Aggregating the model parameter differences sent by the multiple clients according to the aggregation weight of each client to obtain an initial aggregation gradient; A preset gradient clipping boundary is obtained, and the initial aggregated gradient is clipped according to the gradient clipping boundary to obtain the aggregated gradient.

3. The method according to claim 1 or 2, characterized in that: The updating of the model parameters of the initial segmentation model according to the aggregated gradient to obtain an intermediate segmentation model includes: Gaussian noise is obtained, and model parameters of the initial segmentation model are updated according to the aggregated gradient and the Gaussian noise to obtain the intermediate segmentation model.

4. The method according to any one of claims 1 to 3, characterized in that: The aggregating the model parameter differences sent by the plurality of clients according to the aggregation weight of each client comprises: Obtaining a preset aggregation probability and obtaining the sum of the aggregation weights of the plurality of clients; the aggregation probability is used to indicate the probability that the model parameter difference obtained by the client is sent to the server; The model parameter differences sent by the multiple clients are aggregated according to the aggregation probability, the sum of the aggregation weights of the multiple clients, and the aggregation weights of the multiple clients.

5. The method according to claim 4, characterized in that The aggregating the model parameter differences sent by the multiple clients according to the aggregation probability, the sum of the aggregation weights of the multiple clients, and the aggregation weights of the multiple clients, comprises: Calculating, for each of the clients, the product of the model parameter difference sent by the client and the aggregation weight of the client, and summing the products of multiple clients; The ratio of the sum obtained by the summing to the sum of the aggregation probability and the aggregation weight is calculated to obtain the aggregation gradient.

6. A method for training an MRI image segmentation model based on federated learning, characterized in that: The method is applied to a client, and the method comprises: Receive the initial segmentation model sent by the server, train the initial segmentation model based on a local multimodal MRI image training set, and obtain updated model parameters of the initial segmentation model based on the model training; Determine a model parameter difference between the updated model parameters and the initial model parameters of the initial segmentation model, and send the model parameter difference to the server, so that the server aggregates the model parameter differences sent by multiple clients according to the aggregation weight of each client to obtain an aggregation gradient, wherein the aggregation weight is used to represent the weight of the model parameter differences sent by the client participating in the model parameter aggregation, and update the model parameters of the initial segmentation model according to the aggregation gradient to obtain an intermediate segmentation model; Receive the intermediate segmentation model sent by the server, use the intermediate segmentation model as the initial segmentation model, and return to execute the local multimodal MRI image training set to train the initial segmentation model.

7. The method according to claim 6, characterized in that The determining of a model parameter difference between the updated model parameter and the initial model parameter of the initial segmentation model comprises: Performing gradient descent on the initial model parameters of the initial segmentation model according to a preset learning rate and a loss function to obtain gradient descent parameters; The gradient descent parameter is gradient clipped according to a preset a priori upper bound to obtain a gradient clipping parameter, and the difference between the gradient clipping parameter and the initial model parameter is calculated to obtain the model parameter difference.

8. The method according to claim 6, characterized in that The initial segmentation model is trained based on the local multimodal MRI image training set, including: Obtaining the training data set ratio of the client, where the training data set ratio is used to indicate the ratio of the MRI image training set used for model training locally on the client to the total MRI image data set, and the training data set ratio is obtained by random sampling from a preset distribution sequence; A local multimodal MRI image training set is acquired according to the training data set ratio, and the initial segmentation model is trained based on the local multimodal MRI image training set.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

10. A computer storage medium, characterized in that: The computer storage medium stores instructions, and when the instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 8.

Images