Federated learning method and apparatus for medical image segmentation
By generating and aligning prototype vectors for medical image segmentation models using a federated learning method, the problem of incompatibility between image segmentation models from different hospitals is solved, thereby improving the universality of cross-hospital image segmentation and the generalization performance of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GENERAL HOSPITAL OF PLA
- Filing Date
- 2025-09-22
- Publication Date
- 2026-08-04
AI Technical Summary
Medical image segmentation cannot be universally applied across different hospitals, mainly due to data heterogeneity caused by differences in image modalities and scanning sequences.
A federated learning approach is adopted, which generates local prototype vectors through local model training, uploads them to a central server for aggregation to obtain global prototype vectors, and optimizes the global image segmentation model through prototype alignment loss and segmentation loss to achieve image segmentation across hospitals.
While maintaining data privacy, the model achieves universality in image segmentation across different hospitals, improves model generalization performance across modalities and doses, and reduces communication overhead and model heterogeneity issues.
Smart Images
Figure CN121504804B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image segmentation technology, and in particular to a federated learning method and apparatus for medical image segmentation. Background Technology
[0002] Medical imaging (such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging)) has been widely used in the auxiliary diagnosis of various diseases. Utilizing deep learning models for automatic segmentation of medical images holds the promise of significantly improving diagnostic efficiency and accuracy. However, image data acquired by different hospitals exhibits significant heterogeneity. For example, differences in image modalities (e.g., CT vs. MRI), and variations in different scanning sequences or doses within the same modality (e.g., T1-weighted vs. T2-weighted imaging in MRI), make image segmentation between different hospitals incompatible. Summary of the Invention
[0003] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a federated learning method and apparatus for medical image segmentation, which solves the technical problem that image segmentation cannot be universally applied between different hospitals.
[0004] To achieve the above objectives, the main technical solutions adopted by the present invention include:
[0005] The first aspect of this invention provides a federated learning method for medical image segmentation.
[0006] The federated learning method for medical image segmentation proposed in this invention includes:
[0007] Based on local medical image data from multiple clients, local model training is performed on each of their respective medical image segmentation models, and local prototype vectors of each organ category in the local medical images are output.
[0008] The local prototype vectors are uploaded to the central server for aggregation to obtain the global prototype vectors.
[0009] The local prototype vector is aligned with the global prototype vector based on the feature morphology of the organ segmentation region, and the prototype alignment loss is determined.
[0010] Using the total loss function as the optimization objective, the model parameters of each medical image segmentation model are updated through backpropagation to obtain a global image segmentation model; wherein, the total loss function is the sum of prototype alignment loss and segmentation loss.
[0011] In some instances, if the medical image segmentation model is a single-modality model, then the local medical image segmentation model based on local medical image data from multiple clients is trained locally, outputting local prototype vectors for each organ category in the local medical images, including:
[0012] Local medical image data is encoded into embedded features using a feature extraction network;
[0013] The embedded features are processed by a segmenter to obtain the predicted output;
[0014] The difference between the predicted output and the true label is calculated by using a segmentation loss function, and the model is trained to supervise the training based on the difference between the predicted output and the true label, so as to obtain the semantic feature vector for each organ category in the embedding space.
[0015] The local prototype vector is obtained by averaging the semantic feature vectors of the same organ category corresponding to different clients.
[0016] In some instances, if the medical image segmentation model is a multimodal model, then the local medical image segmentation model based on local medical image data from multiple clients is used to train the respective medical image segmentation model locally, outputting local prototype vectors for each organ category in the local medical images, including:
[0017] Local medical image data is encoded into embedded features using a feature extraction network;
[0018] The embedding features of the source modality are mapped to the embedding features of the target modality through a domain adaptation network;
[0019] The segmenter is used to process the embedded features of the mapped target modality to obtain the prediction output;
[0020] The difference between the predicted output and the true label is calculated by using a segmentation loss function, and the model is trained to supervise the training based on the difference between the predicted output and the true label, so as to obtain the semantic feature vector for each organ category in the embedding space.
[0021] The local prototype vector is obtained by averaging the semantic feature vectors of the same organ category corresponding to different clients.
[0022] In some instances, uploading the local prototype vector to a central server for local prototype vector aggregation to obtain a global prototype vector includes:
[0023] The global prototype vector is obtained by summing and averaging all local prototype vectors of the same organ category corresponding to multiple clients.
[0024] In some instances, the prototype alignment loss for:
[0025] in, The global prototype vector; C (j) This is the local prototype vector.
[0026] In some instances, segmentation loss for:
[0027] in, y i Let i be the true class of the i-th pixel;
[0028] Predict the probability that the i-th pixel belongs to class c for the model.
[0029] In some instances, the single-modal model is a single data source 0 model that processes CT or MRI images;
[0030] The multimodal model is a multi-data source model for processing CT and MRI images.
[0031] A second aspect of the present invention provides a federated learning device for medical image segmentation, comprising:
[0032] The local model training unit is used to train the local model of each medical image segmentation model based on the local medical image data of multiple clients, and output the local prototype vector of each organ category in the local medical image.
[0033] The local prototype vector aggregation unit is used to upload the local prototype vector to the central server for local prototype vector aggregation to obtain the global prototype vector.
[0034] The prototype alignment loss determination unit is used to align the local prototype vector with the global prototype vector according to the feature morphology of the organ segmentation region, and determine the prototype alignment loss.
[0035] The model parameter update unit is used to update the model parameters of each medical image segmentation model through backpropagation with the total loss function as the optimization objective, so as to obtain the global image segmentation model; wherein, the total loss function is the sum of prototype alignment loss and segmentation loss.
[0036] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the steps of the method described in the first aspect above.
[0037] A fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the method described in the first aspect above.
[0038] This invention discloses a federated learning method for medical image segmentation, comprising: training local models for each medical image segmentation model based on local medical image data from multiple clients, outputting local prototype vectors for each organ category in the local medical images; uploading the local prototype vectors to a central server for aggregation to obtain a global prototype vector; aligning the local prototype vectors with the global prototype vectors according to the feature morphology of organ characteristics to determine the prototype alignment loss; and updating the model parameters of each medical image segmentation model through backpropagation with the total loss function as the optimization objective to obtain a global image segmentation model; wherein the total loss function is the sum of the prototype alignment loss and the segmentation loss. In this application, a global image segmentation model incorporating knowledge from each client is constructed through a federated learning paradigm of local training + prototype sharing and alignment, thereby enabling image segmentation of medical images from different hospitals using the global image segmentation model. Attached Figure Description
[0039] Figure 1 A flowchart of a federated learning method for medical image segmentation provided in an embodiment of the present invention;
[0040] Figure 2 A schematic diagram of a federated learning model for medical image segmentation provided in an embodiment of the present invention;
[0041] Figure 3 This is a schematic diagram of a federated learning device for medical image segmentation provided in an embodiment of the present invention. Detailed Implementation
[0042] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0043] The federated learning method for medical image segmentation proposed in this invention addresses the problem of incompatibility in image segmentation between different hospitals. By employing a federated learning paradigm of local training and prototype sharing alignment, a global image segmentation model that incorporates knowledge from each client is constructed, thereby enabling image segmentation of medical images from different hospitals through this global model.
[0044] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.
[0045] Figure 1 This is a flowchart illustrating a federated learning method for medical image segmentation, provided as an embodiment of the present invention. Figure 1 As shown in the embodiments of the present invention, the federated learning method for medical image segmentation includes:
[0046] Step 100: Based on the local medical image data of multiple clients, perform local model training on each of their respective medical image segmentation models, and output the local prototype vectors of each organ category in the local medical images.
[0047] Step 110: Upload the local prototype vector to the central server, perform local prototype vector aggregation, and obtain the global prototype vector.
[0048] Step 120: Align the local prototype vector with the global prototype vector according to the feature morphology of the organ segmentation region, and determine the prototype alignment loss;
[0049] Step 130: Using the total loss function as the optimization objective, update the model parameters of each medical image segmentation model through backpropagation to obtain the global image segmentation model; wherein, the total loss function is the sum of prototype alignment loss and segmentation loss.
[0050] In this exemplary embodiment, local medical image data refers to the medical image data and corresponding annotation data that are privately owned by each hospital (client) participating in federated learning, stored on their local server, and cannot be directly shared. The local prototype vector is a feature vector of the local medical image style extracted from the local medical image data. The feature vector contains annotation information of the image style. The global prototype vector is the feature vector of the target medical image. For example, the local medical image is a CT image, and the target medical image is an MRI image.
[0051] In this exemplary embodiment, "organ category" refers to different organs, such as the abdomen and chest. The local prototype vector is aligned with the global prototype vector based on the feature morphology of organs such as the abdomen and chest, and a prototype alignment loss is determined.
[0052] In this application, a global image segmentation model with knowledge from various clients is constructed through a federated learning paradigm of local training and prototype sharing alignment, thereby enabling image segmentation of medical images from different hospitals through the global image segmentation model.
[0053] This application presents a prototype-aligned federated learning method for heterogeneous medical image segmentation tasks. In this method, participating hospitals do not need to share their model parameters or raw data; instead, they only send abstracted category prototype information to a central server. The category prototype refers to a representative vector obtained by averaging the feature vectors of each category sample in the model's embedding space. By exchanging and aligning these prototype representations within the federation, models from different hospitals can achieve knowledge collaboration while maintaining data privacy. This method combines the discriminative power of prototype learning with the privacy-preserving advantages of federated learning, and can adapt to diverse data modalities and model structures. Compared to traditional parameter aggregation-based federated learning methods, the prototype-based federated learning method proposed in this application is more effective in mitigating model heterogeneity and can improve the global model generalization performance in cross-modal and cross-dose domain transfer scenarios.
[0054] In some instances, if the medical image segmentation model is a single-modality model, then the local medical image segmentation model based on local medical image data from multiple clients is trained locally, outputting local prototype vectors for each organ category in the local medical images, including:
[0055] Local medical image data is encoded into embedded features using a feature extraction network;
[0056] The embedded features are processed by a segmenter to obtain the predicted output;
[0057] The difference between the predicted output and the true label is calculated by using a segmentation loss function, and the model is trained to supervise the training based on the difference between the predicted output and the true label, so as to obtain the semantic feature vector for each organ category in the embedding space.
[0058] The local prototype vector is obtained by averaging the semantic feature vectors of the same organ category corresponding to different clients.
[0059] In this exemplary embodiment, the single-modal client undergoes local training. The steps are as follows:
[0060] For hospital clients with only single-modality images (CT or MRI), the model consists of a feature extraction network and a segmenter. Given an input image x, it is first encoded into an embedded feature vector h = f(x) by a feature extraction module such as a convolutional neural network, and then the corresponding prediction is output by the segmenter g(h). The probability that the i-th pixel belongs to each category is:
[0061]
[0062] The model is trained locally using supervised learning, and cross-entropy loss is used to compare the predicted g(f(x)) with the true label y. i Alignment is performed within {1,2,...,C} to ensure the model's ability to discriminate against local data.
[0063] Its loss function is:
[0064]
[0065] Where N represents the number of pixels, y i ∈{1,...,C} is the true class of the i-th pixel. It is the probability that the model predicts the i-th pixel as belonging to the c-th class. This indicates that the real label y will be used. i Convert to a one-hot vector:
[0066]
[0067] Simultaneously, for each category in the embedding space, the client's local prototype vector C is computed. (j) At the end of each round of local training, the client sends its current class prototypes to the server. It's important to note that the parameters of the local model are not uploaded; only the high-level anonymized representation of the class prototypes is shared, thus maximizing the protection of patient privacy and institutional data confidentiality.
[0068] In some instances, if the medical image segmentation model is a multimodal model, then the local medical image segmentation model based on local medical image data from multiple clients is used to train the respective medical image segmentation model locally, outputting local prototype vectors for each organ category in the local medical images, including:
[0069] Local medical image data is encoded into embedded features using a feature extraction network;
[0070] The embedding features of the source modality are mapped to the embedding features of the target modality through a domain adaptation network;
[0071] The segmenter is used to process the embedded features of the mapped target modality to obtain the prediction output;
[0072] The difference between the predicted output and the true label is calculated by using a segmentation loss function, and the model is trained to supervise the training based on the difference between the predicted output and the true label, so as to obtain the semantic feature vector for each organ category in the embedding space.
[0073] The local prototype vector is obtained by averaging the semantic feature vectors of the same organ category corresponding to different clients.
[0074] In this exemplary embodiment, for hospital clients that simultaneously possess multiple image modalities or different scanning sequences (e.g., simultaneous CT and MRI, or simultaneous T1- and T2-weighted MRI), this method adds an image domain adaptation module to achieve cross-domain feature alignment. Specifically, given an input image x, it is first encoded into an embedded feature vector h = f(x) through a feature extraction module such as a convolutional neural network. Then, using the constructed domain adaptation network m = a(h), a source modality (e.g., CT) is converted into a target modality (e.g., MRI). The converted image is then analyzed by a segmenter g(m) to predict and calculate the supervised loss. Simultaneously, the client also calculates its local class prototype, following the same process as in the single-modality case.
[0075] Figure 2 This is a schematic diagram of a federated learning model for medical image segmentation provided in an embodiment of the present invention. Figure 2 As shown, "CT pointing to MRI" refers to the migration of CT image annotations to MRI images. "MRI pointing to CT" refers to the migration of MRI image annotations to CT images. t1 Pointing to MRI t2 This refers to MRI. t1 Image annotation migration to MRI t2 Images. Among them,
[0076] Domain adaptation refers to converting images from CT style to MR style or vice versa. Feature extraction refers to using a model to extract features from the original image to obtain image feature vectors. Classification refers to labeling the feature vectors, i.e., obtaining semantic categories from the vectors.
[0077] Where, x CT and x MR These represent CT and MRI images, respectively. CT and h MR These represent the feature vectors for the CT and MRI modalities, respectively. and The y represents the classification results of CT and MRI images, respectively. CT and y MR These represent the true labels for CT and MRI images, respectively. and These represent MRI1 and MRI2 images, respectively. and These represent the feature vectors of the MRI1 and MRI2 modes, respectively. and These represent the true labels for MRI1 and MRI2, respectively. and These represent the classification results for MRI1 and MRI2 images, respectively. and These represent the local prototype vectors corresponding to the organ categories in MRI1 and MRI2 images, respectively. and These represent the global prototype vectors corresponding to the organ categories in MRI1 and MRI2 images, respectively.
[0078] In this exemplary embodiment, the step of uploading the local prototype vector to the central server for local prototype vector aggregation to obtain the global prototype vector includes:
[0079] The global prototype vector is obtained by summing and averaging all local prototype vectors of the same organ category corresponding to multiple clients.
[0080] The local prototype vector is aligned with the global prototype vector based on the feature morphology of the organ segmentation region. The prototype alignment loss is determined, including:
[0081] The differences in organ feature morphology represented by the local prototype and the global prototype are directly compared using a morphological similarity measurement function.
[0082] Based on the differences in organ feature morphology represented by the local prototype and the global prototype, the prototype alignment loss is determined.
[0083] The morphological similarity measurement function can be a cosine similarity function, which makes the loss drive the two vectors to tend to be consistent in direction in the feature space, thereby aligning the morphological features of the organs they represent.
[0084] In some instances, the prototype alignment loss can also be
[0085] in, The global prototype vector; C (j) The local prototype vectors are used to make the loss drive the two vectors to align their directions in the feature space, thereby aligning the organ morphological features they represent.
[0086] In some instances, segmentation loss for:
[0087] in, y i Let i be the true class of the i-th pixel;
[0088] The model predicts the probability that the i-th pixel belongs to the c-th class; c is the number of organ classes.
[0089] In some instances, the single-modal model is a single data source 0 model that processes CT or MRI images;
[0090] The multimodal model is a multi-data source model for processing CT and MRI images.
[0091] In this exemplary embodiment, the learned embedding space may be biased regardless of whether the client is unimodal or multimodal. Therefore, a prototype alignment loss term is also introduced into the local loss to measure the difference between the local prototype and the global prototype.
[0092] After receiving local category prototypes from each client in each round, the central server aggregates the prototype vectors from different clients for each category to form a global prototype for that category. These global prototypes can be viewed as a summary of "collective knowledge" for that category across the federation. Alignment of image segmentation ranges, alignment, and alignment of organ segmentation ranges based on organ feature morphology.
[0093]
[0094] in, It is the global prototype vector corresponding to the j-th class. This represents a collection of clients with class j.
[0095] The server will then present a global prototype. This is then distributed to each participating client. During local training, each client uses the received global prototype as a reference to calculate its own local prototype vector C. (j) The distance between it and the prototype is calculated and incorporated into the prototype alignment loss.
[0096]
[0097] Therefore, the total loss of the local model is the sum of the segmentation loss and the prototype alignment loss:
[0098]
[0099] In this exemplary embodiment, the model parameters of each medical image segmentation model are updated through backpropagation, with the total loss function as the optimization objective, to obtain a global image segmentation model, including:
[0100] By using the chain rule, the partial derivatives of the loss function with respect to each parameter are calculated layer by layer from the output layer to the next, and the gradients of all parameters are obtained.
[0101] The gradient of the parameters propagates from the total loss to the prototype alignment loss, and then to the local prototype vectors, generating the feature extraction network parameters θf for these feature vectors.
[0102] After obtaining the gradients of all parameters, the model parameters are updated using an optimization algorithm.
[0103] The model parameters are updated using optimization algorithms, including:
[0104] Each client uploads its updated model parameters (or prototype, depending on the federated learning strategy) to the central server;
[0105] The server uses an aggregation algorithm to perform a weighted average of the model parameters from all clients, generating a new generation of global model parameters.
[0106] The server distributes the aggregated new global model parameters to all or some clients.
[0107] The client initializes its local model with the new global model and begins the next round of federated learning until the global model's performance converges. In this way, the parameters of each medical image segmentation model can be updated using the above method to obtain a global image segmentation model.
[0108] This process is akin to knowledge distillation: the global prototype carries data distribution information from other institutions, acting as "teacher" knowledge, while the local model corrects its own feature representation biases by aligning with these prototypes. As federated training progresses, the global prototype is continuously updated, and the feature spaces of each client gradually converge, improving the global model's adaptability to data from various domains.
[0109] Throughout the federated learning process, each client repeatedly cycles through "local training—prototype upload—prototype alignment" until the global model converges. This method only requires the transfer of small-scale prototype vectors between the client and server, without involving the exchange of original images or high-dimensional model parameters, thus resulting in low communication overhead and high security. Furthermore, since it does not force clients to adopt a uniform model structure, it allows each institution to customize its model based on its own data characteristics, solving the federated learning challenge caused by model heterogeneity. Through global prototype alignment, the discriminative boundaries of each client model are unified in the abstract feature space, enabling the training of a shared model with good generalization performance even when facing cross-modal or cross-dose distribution differences. In summary, the proposed prototype alignment federated learning framework fully combines the privacy-preserving collaborative advantages of federated learning with the discriminative power of prototype representation, providing an efficient and robust solution for heterogeneous medical image segmentation tasks.
[0110] This invention provides a federated learning device for medical image segmentation. Figure 3 This is a schematic diagram of a federated learning device for medical image segmentation provided in an embodiment of the present invention. Figure 3 As shown, the federated learning device for medical image segmentation includes:
[0111] The local model training unit 30 is used to train the local model of each medical image segmentation model based on the local medical image data of multiple clients, and output the local prototype vector of each organ category in the local medical image.
[0112] Local prototype vector aggregation unit 31 is used to upload the local prototype vector to the central server to perform local prototype vector aggregation and obtain a global prototype vector.
[0113] The prototype alignment loss determination unit 32 is used to align the local prototype vector with the global prototype vector according to the feature morphology of the organ segmentation region, and determine the prototype alignment loss.
[0114] The model parameter update unit 33 is used to update the model parameters of each medical image segmentation model through backpropagation with the total loss function as the optimization objective, so as to obtain the global image segmentation model; wherein, the total loss function is the sum of prototype alignment loss and segmentation loss.
[0115] In this exemplary embodiment, local medical image data refers to the medical image data and corresponding annotation data that are privately owned by each hospital (client) participating in federated learning, stored on their local server, and cannot be directly shared. The local prototype vector is a feature vector of the local medical image style extracted from the local medical image data. The feature vector contains annotation information of the image style. The global prototype vector is the feature vector of the target medical image. For example, the local medical image is a CT image, and the target medical image is an MRI image.
[0116] In this exemplary embodiment, "organ category" refers to different organs, such as the abdomen and chest. The local prototype vector is aligned with the global prototype vector based on the feature morphology of organs such as the abdomen and chest, and a prototype alignment loss is determined.
[0117] In this application, a global image segmentation model with knowledge from various clients is constructed through a federated learning paradigm of local training and prototype sharing alignment, thereby enabling image segmentation of medical images from different hospitals through the global image segmentation model.
[0118] This application presents a federated learning device for medical image segmentation, designed for heterogeneous medical image segmentation tasks. Participating hospitals do not need to share their model parameters or raw data; instead, they only send abstracted category prototype information to a central server. The category prototype refers to a representative vector obtained by averaging the feature vectors of each category sample in the model's embedding space. By exchanging and aligning these prototype representations within the federation, models from different hospitals can achieve knowledge collaboration while maintaining data privacy. This method combines the discriminative power of prototype learning with the privacy-preserving advantages of federated learning, and can adapt to diverse data modalities and model structures. Compared to traditional parameter aggregation-based federated learning methods, the prototype federated learning method proposed in this application is more effective in mitigating model heterogeneity and can improve the global model generalization performance in cross-modal and cross-dose domain transfer scenarios.
[0119] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.
[0120] This invention provides a computer-readable storage medium storing a computer program thereon, characterized in that the program, when executed by a processor, implements the steps of the methods described in the above embodiments.
[0121] This invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the methods described in the above embodiments.
[0122] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0123] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0124] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that they are in indirect contact through an intermediate medium. Furthermore, "above," "over," or "on top" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," or "beneath" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.
[0125] In the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0126] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make modifications, alterations, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A federated learning method for medical image segmentation, characterized in that, include: Based on local medical image data from multiple clients, local model training is performed on each of their respective medical image segmentation models, and local prototype vectors of each organ category in the local medical images are output; wherein, the medical image segmentation model is a single-modal model or a multi-modal model. Multiple clients can upload their local prototype vectors to a central server without sharing model parameters or raw data. By weighted averaging and aggregating the local prototype vectors, global prototype vectors corresponding to each organ category can be obtained, enabling information exchange and knowledge collaboration of local prototype vectors among multiple clients while protecting data privacy within the federation. By comparing the differences in organ feature morphology represented by the local prototype vector and the global prototype vector using a morphological similarity metric function, the local prototype vector and the global prototype vector are aligned, and the prototype alignment loss is determined. Using the total loss function as the optimization objective, the model parameters of each medical image segmentation model are updated through backpropagation to obtain a global image segmentation model; wherein, the total loss function is the sum of prototype alignment loss and segmentation loss.
2. The federated learning method for medical image segmentation of claim 1, wherein, If the medical image segmentation model is a single-modal model, then the local medical image segmentation model based on local medical image data from multiple clients is trained locally, outputting local prototype vectors for each organ category in the local medical images, including: Local medical image data is encoded into embedded features using a feature extraction network; The embedded features are processed by a segmenter to obtain the predicted output; The difference between the predicted output and the true label is calculated by using a segmentation loss function, and the model is trained to supervise the training based on the difference between the predicted output and the true label, so as to obtain the semantic feature vector for each organ category in the embedding space. The local prototype vector is obtained by averaging the semantic feature vectors of the same organ category corresponding to different clients.
3. The federated learning method for medical image segmentation of claim 1, wherein, If the medical image segmentation model is a multimodal model, then the local medical image segmentation model based on local medical image data from multiple clients is used to train the respective models locally, outputting local prototype vectors for each organ category in the local medical images, including: Local medical image data is encoded into embedded features using a feature extraction network; The embedding features of the source modality are mapped to the embedding features of the target modality through a domain adaptation network; The segmenter is used to process the embedded features of the mapped target modality to obtain the prediction output; The difference between the predicted output and the true label is calculated by using a segmentation loss function, and the model is trained to supervise the training based on the difference between the predicted output and the true label, so as to obtain the semantic feature vector for each organ category in the embedding space. The local prototype vector is obtained by averaging the semantic feature vectors of the same organ category corresponding to different clients.
4. The federated learning method for medical image segmentation of claim 1, wherein, The prototype alignment loss Is: ; wherein, is a global prototype vector; is a local prototype vector.
5. The federated learning method for medical image segmentation of claim 1, wherein, Segmentation loss is: ; wherein, ; is the true class of the th pixel; Predict the probability that a model classifies a pixel as a first class. second class.
6. The federated learning method for medical image segmentation of claim 1, wherein, The single-modal model is a single data source model for processing CT or MRI images; The multimodal model is a multi-data source model for processing CT and MRI images.
7. A federated learning device for medical image segmentation, characterized in that, include: The local model training unit is used to train the local model of each medical image segmentation model based on the local medical image data of multiple clients, and output the local prototype vector of each organ category in the local medical image. The local prototype vector aggregation unit is used by multiple clients to upload the local prototype vectors to the central server without sharing model parameters or raw data. By performing weighted average aggregation on the local prototype vectors, global prototype vectors corresponding to each organ category are obtained, realizing information exchange and knowledge collaboration of local prototype vectors of multiple clients while protecting data privacy within the federation. The prototype alignment loss determination unit is used to compare the differences in organ feature morphology represented by the local prototype vector and the global prototype vector through a morphological similarity measurement function, align the local prototype vector with the global prototype vector, and determine the prototype alignment loss. The model parameter update unit is used to update the model parameters of each medical image segmentation model through backpropagation with the total loss function as the optimization objective, so as to obtain the global image segmentation model; wherein, the total loss function is the sum of prototype alignment loss and segmentation loss.
8. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 6.