A partial labeling federated multi-organ segmentation method based on personalized probabilistic atlas
By introducing personalized probabilistic graphs and registration networks into federated learning, the problem of decreased multi-organ segmentation performance caused by client-side annotation heterogeneity was solved, achieving more efficient multi-organ segmentation of medical images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2026-03-11
- Publication Date
- 2026-06-05
AI Technical Summary
In federated learning, the client only labels some organs, resulting in inconsistent labeling categories, which affects the performance and generalization ability of the global model and makes it difficult to achieve effective segmentation of all target organs in medical images.
By introducing personalized probabilistic maps as a cross-client supervisory information carrier, prior guidance on location, size, and shape is provided, and spatial alignment is achieved using personalized registration networks, thereby enhancing the adaptability of the ground truth probabilistic maps and alleviating the labeling heterogeneity problem.
It improves the accuracy and generalization performance of multi-organ segmentation, enhances the segmentation effect of the global model under partially labeled conditions, and improves the cross-client collaborative training capability.
Smart Images
Figure CN122156633A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing technology, and specifically to a federated multi-organ segmentation method based on partially labeled datasets of personalized probabilistic graphs using federated learning technology. Background Technology
[0002] Multi-organ segmentation is a crucial task in medical image analysis, playing a vital role in automating clinical workflows across various anatomical regions, including the head and neck, chest, and abdominal and pelvic cavities. In recent years, federated learning, as a distributed machine learning paradigm, has garnered significant attention due to its ability to collaboratively train unified models across medical institutions without sharing raw data. This characteristic is particularly important in the field of medical imaging, where medical image data involves highly sensitive privacy information, necessitating stringent data protection requirements.
[0003] In practical clinical applications, fully annotating all organs in medical images is typically costly and time-consuming. Therefore, different medical institutions often only annotate organs relevant to their clinical tasks. For example, a urology hospital might only annotate the kidneys and bladder in an abdominal CT (computed tomography) image, treating other organs as background. While this selective annotation significantly reduces workload, it inevitably leads to cross-institutional organ annotation discrepancies, resulting in a heterogeneous partially annotated dataset. In such scenarios, inconsistent annotation categories across different clients lead to differences in local training objectives, consequently affecting the performance and generalization ability of the global model. Therefore, how to alleviate annotation heterogeneity and improve cross-institutional collaborative segmentation performance under partially annotated conditions is a pressing technical problem that needs to be solved.
[0004] Partially labeled datasets have gained widespread attention in weakly supervised learning in recent years due to their significant reduction in data acquisition costs, particularly in medical image analysis tasks. In centralized learning scenarios, existing research has proposed various solutions, including designing loss functions compatible with partial labels, constructing specific network structures that effectively utilize partial labels, and transforming partial supervision into full supervision through pseudo-label mechanisms.
[0005] Similar ideas have gradually extended to federated learning frameworks to address the labeling discrepancies caused by partially labeled datasets in distributed environments. Among existing methods, representative sub-network structures include FedMENU and FedIOD, both of which utilize partial labels by constructing organ-specific sub-networks. FedMENU's main encoder consists of multiple organ-specific sub-encoders. During local training, it only uses the output corresponding to locally labeled organs for loss calculation and introduces an auxiliary decoder to enhance feature discrimination. However, this method leads the client to overemphasize locally labeled organs, exacerbating parameter conflicts during the federated aggregation stage and limiting the performance of the global model in multi-organ segmentation tasks. In contrast, FedIOD argues that the encoder is more resistant to labeling heterogeneity than the decoder, thus proposing a symmetric structure of a single encoder and multiple decoders and introducing a self-attention mechanism to model inter-organ dependencies to alleviate parameter conflicts. However, as the number of organ categories increases, the number of parameters in such methods grows linearly, limiting their scalability and generalization ability in multi-class scenarios.
[0006] Another approach centers on knowledge distillation, with FLKD (Federated Learning Knowledge Distillation) being a representative work. In the local distillation phase, each client first trains an organ-specific model based on locally labeled organs and shares it with other clients, generating pseudo-labels for unlabeled organs. In the global distillation phase, distillation is achieved by aligning the predictions of the local and global models. While this method enriches the supervision signal by fusing multi-source knowledge, the local distillation process is highly dependent on the client data distribution. Given demographic, pathological, and imaging protocol differences among medical institutions, pseudo-labels may exhibit structural biases or spatial misalignments, affecting model stability. Furthermore, the local model needs to align both local and global distributions simultaneously, potentially leading to target conflicts and weakening the distillation effect. Summary of the Invention
[0007] Currently, when using federated learning techniques to perform multi-organ segmentation tasks in medical images, a problem arises: different clients only annotate a portion of the organs in the original image, leading to inconsistent organ categories and differences in local training targets. This, in turn, affects the performance and generalization ability of the global model, making it difficult to effectively segment all target organs in medical images. To address this issue, this invention proposes a partially annotated federated multi-organ segmentation method based on personalized probabilistic maps. The probabilistic map serves as a cross-client supervision information carrier, providing prior guidance on the location, size, and shape of organs lacking annotation on the client side. This alleviates the annotation heterogeneity problem caused by partially annotated datasets during federated training. Simultaneously, a personalized registration network is introduced to spatially align the shared ground truth probabilistic map with the client's actual local organ structure, thereby enhancing the cross-client adaptability of the ground truth probabilistic map and improving the efficiency of effective segmentation of all target organs in medical images.
[0008] The present invention provides a partially labeled federated multi-organ segmentation method based on personalized probabilistic graphs, comprising the following steps:
[0009] Step 1: Determine the clients in the application scenario and the organ categories labeled by each client;
[0010] Step 2: Each client preprocesses the acquired local medical images to unify the spatial resolution and normalize the grayscale values.
[0011] Step 3: Each client uses the pre-trained segmentation network to identify locally labeled organs from local medical images, constructs a ground truth probability map of locally labeled organs based on the recognition results of all local medical images, and uploads it to the server; the server aggregates the ground truth probability maps of locally labeled organs uploaded by all clients to obtain a ground truth probability map of all organs. And distribute it to each client;
[0012] Step 4: Perform the federated training process, including:
[0013] (1) Warm-up phase, which includes: server initialization of global segmentation network. The parameters are distributed to each client; each client initializes its own registration network; each client updates its local segmentation network using the global segmentation network parameters distributed by the server, and trains its local segmentation network using supervised loss based on image samples of locally labeled organs; on the other hand, it uses the global segmentation network to perform whole-organ segmentation prediction on local medical images, averages the prediction results of all images at the voxel level, and then extracts the prediction atlas corresponding to the locally labeled organs; each client uploads its local segmentation network parameters and prediction atlas to the server.
[0014] (2) In the formal stage, the execution includes: step a1, the server updates the global segmentation network parameters through federated aggregation, and splices the prediction maps uploaded by all clients along the channel dimension to obtain the whole organ prediction map. The server distributes the current global segmentation network parameters and whole-organ prediction maps to each client; in step a2, each client updates the registration network; the registration network is used to fit the spatial transformation from the whole-organ prediction map to the prediction results of the global segmentation network on the local medical image, obtaining a voxel-level deformation field; in step a3, each client uses the deformation field to generate a true probability map of the whole organs. A spatial transformation is performed to obtain a personalized ground truth probability map. The personalized ground truth probability map and the local organ real annotations are used together as supervision signals to train the local segmentation network of the client. In step a4, each client uses the current global segmentation network to predict the local training samples and generate a prediction map of the locally annotated organs. The client uploads the prediction map of the locally annotated organs and the local segmentation network parameters to the server. In step a5, steps a1 to a4 are repeated until the maximum number of iterations is reached or the global segmentation network converges. The iteration is then stopped, and the global segmentation network obtained at this time is deployed to each client and new clients.
[0015] Compared with the prior art, the present invention has the following beneficial effects:
[0016] (1) This invention proposes a federated partial annotation multi-organ segmentation method based on probabilistic graphs, which makes up for the shortcomings of existing algorithms that suffer from reduced segmentation performance due to heterogeneous supervision signals among different clients, expands new ideas for federated multi-organ segmentation, and achieves better generalization performance on federated extra-domain datasets.
[0017] (2) The method of this invention proposes a probabilistic graph supervision mechanism, which realizes the transfer of structural knowledge across clients, provides more comprehensive supervision signals for some labeled clients, and helps to unify the training objectives of all clients.
[0018] (3) The method of the present invention utilizes a personalized registration network to train and generate a personalized probability map for the client, thereby improving the spatial adaptability and segmentation accuracy of the probability map among different clients. Attached Figure Description
[0019] Figure 1 This is a flowchart illustrating the partially annotated federated multi-organ segmentation method of this invention on the client and server.
[0020] Figure 2 This is an example diagram of an application scenario of an embodiment of the present invention;
[0021] Figure 3 This is an implementation architecture diagram of the federated multi-organ segmentation method according to an embodiment of the present invention;
[0022] Figure 4 This is a schematic diagram illustrating the spatial mismatch phenomenon of different clients on the same organ probability map in an embodiment of the present invention;
[0023] Figure 5 This is a comparison of the visual effects of the method of this invention and existing methods on the AMOS dataset for multi-organ segmentation;
[0024] Figure 6 This is a comparison of the visual effects of the method of this invention and existing methods on the BTCV dataset for multi-organ segmentation;
[0025] Figure 7 This is a comparison chart of the similarity index between the personalized registration network in this invention and the ideal graphs of each client before and after registering the probability graph. Detailed Implementation
[0026] The technical solution of the present invention will now be clearly and completely described in conjunction with the accompanying drawings and embodiments.
[0027] like Figure 1 As shown, under partially labeled conditions, the partially labeled federated multi-organ segmentation method based on personalized probabilistic graphs proposed in this invention introduces organ probabilistic graphs into the federated learning framework, enabling cross-client collaborative training of multi-organ segmentation, thereby obtaining a multi-organ segmentation model with higher accuracy and stronger generalization ability. This invention's method enhances the consistency between local training objectives and global multi-organ segmentation objectives through the combination of probabilistic graph sharing and personalized registration, effectively improving the performance of federated training in multi-organ segmentation under partially labeled conditions.
[0028] like Figure 2 As shown, the application scenario of this invention is a federated learning environment among multiple institutions under partial annotation conditions. Since each client's dataset only contains partial organ annotations, and the types of annotated organs vary due to differences in research interests among different institutions, inconsistent supervision signals arise across clients. To address this, the partially annotated federated multi-organ segmentation method based on personalized probabilistic maps of this invention includes: ① First, each client generates a corresponding ground truth probability map based on its annotated organs; ② and uploads this probability map to the server; ③ The server aggregates the organ probability maps from different clients to obtain a complete global ground truth probability map of all organs; ④ Subsequently, the server distributes the global ground truth probability map of all organs to clients lacking corresponding organ annotations to guide their segmentation training on unannotated organs. Through this mechanism, clients without annotated organs can also obtain reliable supervision information, including the location, size, and shape of the organs, thereby achieving collaborative training of multi-organ segmentation models across clients.
[0029] like Figure 1 He Ru Figure 3 As shown, the partially labeled federated multi-organ segmentation method based on personalized probabilistic graphs in this embodiment of the invention includes the aggregation and distribution of a global model, the training of a client-side personalized registration network, and the training of a segmentation network guided by a personalized probabilistic graph. The main implementation steps of the method of this invention are described below.
[0030] Step 1: Determine the clients in the application scenario and the organ categories labeled by each client.
[0031] In a federated multi-organ segmentation scenario with partial annotation conditions, let the common... The client, the first Each client has a locally labeled dataset. ,in Indicates the size is 3D images, Image height, Image width, For image depth, For the corresponding voxel-level segmentation labels, each label At most contains The annotation of target organs, and ,in It is the collection of all target organs. It is the total number of all target organs. It is the first The amount of labeled data per client, Represents a real number.
[0032] For ease of explanation, this embodiment first considers a simplified scenario, in which each client only labels one organ. ,Right now Furthermore, the organs labeled by different clients do not overlap, therefore While this setup is simplified, the method of this invention is also applicable to more general scenarios where each client annotates multiple organs. The goal of this invention is to jointly train a global model while ensuring data privacy. To achieve Accurate segmentation of all organs. The global model is achieved by combining local models on each client. The parameters are obtained through federated aggregation.
[0033] Step 2: Each client acquires local medical images and performs preprocessing.
[0034] Based on the task definition, each client first collects local medical image data and preprocesses it. Specific preprocessing includes: removing invalid data; resampling CT images at voxel intervals according to a preset spatial resolution to ensure consistency in spatial resolution across different data sources; and windowing and normalizing image grayscale values—that is, windowing the image grayscale values and normalizing them to a specified range to reduce the impact of imaging differences. Through these preprocessing operations, the resolution and intensity range of data from different sources are unified, reducing cross-institutional imaging differences and providing a unified data foundation for subsequent probabilistic atlas construction.
[0035] Step 3: The client constructs the organ ground truth probability map and uploads it to the server. The server aggregates all the ground truth probability maps uploaded by the clients to obtain the whole organ ground truth probability map, and then sends it to the clients.
[0036] After data preprocessing, each client performs preliminary model training based on locally labeled organs to construct a probabilistic atlas. Specifically, each client only calculates the loss between the output of the local organ segmentation model on the locally labeled organs and the label, and uses this loss as the objective function to optimize the local organ segmentation model. Then, a few rounds of federated training are performed to obtain the global organ segmentation model. .
[0037] Each client performs initial training based on locally labeled organ image samples, with the loss function consisting of Dice loss and binary cross-entropy loss. The loss function for initial training of the local organ segmentation model by a client. The calculation is as follows:
[0038] ;
[0039] in, For the first The first client's Image samples, Representing the A client pair The label, in this embodiment, corresponds to an organ. ; Indicates the first A local organ segmentation model for each client; It is a function for calculating the supervised loss, expressed as: Dice loss and binary cross-entropy loss The calculations are as follows:
[0040] ;
[0041] ;
[0042] in, Indicates the labeling of the organ. Indicates the predicted probability. Voxel representation The binary truth label belongs to the labeled organ. voxels The predicted probability of belonging to the labeled organ. Represents the set of voxels in an image. A small constant introduced to avoid division by zero. Substituting into the loss function above, the loss value is calculated. .
[0043] Each client uploads the trained local organ segmentation model parameters to the server. The server aggregates the model parameters uploaded by all clients to obtain the parameters of the global segmentation model, and then distributes them to each client. Each client uses the received initial coarse global segmentation model to generate organ prediction results, uses the prediction results to crop the original image to initially unify the field of view, then performs voxel-level averaging on the local annotations, and obtains the ground truth probability map of the locally annotated organs through threshold filtering, which is then uploaded to the server.
[0044] In this embodiment of the invention, the first The client first uses a global coarse segmentation model to predict local data, generating coarse organ prediction results. Based on these prediction results, the Otsu thresholding method is used to extract the abdominal region, and the image and labels are uniformly cropped to a set size. , The height, width, and depth of the cropped image are defined to initially unify the field of view of different data, thereby establishing a roughly unified coordinate reference system in three-dimensional space, providing a foundation for the subsequent construction of ground truth probability maps. Labels are probability matrices identifying each voxel in the image as belonging to a labeled organ; the size of the probability matrix is the same as the size of the cropped image. Next, each client performs voxel-level averaging on all ground truth images of its locally labeled organs to construct the probability map for the corresponding labeled organ. To estimate which organ each voxel belongs to in three-dimensional space. The probability of [the outcome]. This embodiment sets a threshold value here. To reduce the interference of outlier samples on map construction, the probability of voxels with values below this threshold is set to zero, as shown below:
[0045] ;
[0046] in, Voxel representation Belongs to the label organ The probability of.
[0047] Subsequently, each client uploads the constructed probability graph to the server, which then performs the aggregation.
[0048] The client divides the preprocessed labeled data into training, validation, and test sets. Each medical image acquired by the client contains multiple organ structures, but the client only labels the ground truth values of a subset of organs according to its own research needs.
[0049] The server receives the labeled organ ground truth probability maps uploaded by each client, and stitches all probability maps along channels to generate a global all-organ ground truth probability map. This is then distributed to all clients. During the formal federated training phase, clients can utilize this probabilistic graph. As an auxiliary supervisory signal, it guides the local model to learn the structural information of unlabeled organs.
[0050] Step 4: Perform federal training.
[0051] Although the method of this invention standardizes the client data in the early stages, differences in demographic characteristics, pathological structures, and imaging protocols among different medical institutions may lead to spatial adaptability issues when directly sharing probability maps across clients, thus affecting the accuracy of assisted supervision. Figure 4 As shown. Figure 4 As shown, spatial mismatch occurs between different clients on the same organ probability map. The second row represents the organ probability map extracted by one client from partially labeled data, the first row represents the ideal probability map from another client lacking the organ label, and the third row is an overlay of the two, with the yellow area representing the overlap, i.e., the effective spatial guidance region. It can be observed that when the probability map is used directly across clients without adaptation, significant spatial mismatch may occur, thus reducing the supervision effect. This invention provides a personalized registration network for alignment, and the improved map is shown in the fourth row.
[0052] To address this spatial adaptability issue, this invention proposes a personalized registration network. This network can spatially transform the global probability map based on the data characteristics of each client, thereby generating a personalized probability map that better matches their local anatomical structure. Registration networks have been widely used in medical image analysis. Their core capability is to estimate the deformation field between image pairs, thereby achieving spatial mapping and alignment of labels or structures. In traditional centralized settings, registration networks typically receive fixed and moving images as input, outputting the deformation field from the fixed image to the moving image, and thereby transforming the label corresponding to the fixed image into a representation in the moving image space, achieving good application results. However, in federated learning scenarios, due to privacy considerations, clients cannot share the original images, making it difficult to directly transfer traditional registration strategies. To address this, this invention designs a method to model organ structural differences between clients without accessing the original images. Considering that the global model is an aggregation of knowledge from all clients, its prediction results have stronger structural representation capabilities and generalization, providing a stable and reliable structural reference for the registration process. Therefore, this invention proposes to use the prediction map of the global model and its soft prediction of local data to construct the registration input, and train a personalized registration network to achieve personalized structural adaptation across clients.
[0053] Step 41: Warm-up Phase. Initially, the server determines the global segmentation network. The server sets the network structure and randomly initializes the network parameters, then distributes the initialized global segmentation network parameters to each client. At the start of training, all clients receive the segmentation network parameters from the server as the initial state of their local segmentation networks. Simultaneously, each client independently holds and initializes its personalized registration network. Subsequently, each client, during the first round of training, used supervised loss... Perform backpropagation to train the local segmentation network After the local update is complete, the local segmentation network parameters are uploaded to the server. Each client uses the initial global segmentation network to segment organs in local medical image samples, then averages the prediction results of all samples at the voxel level and extracts the organs compared to the locally labeled organs. For the corresponding channels, construct a prediction atlas specific to locally labeled organs. The formal expression of this process is as follows:
[0054] ;
[0055] The formula above indicates that the first Each client utilizes the global segmentation network distributed by the server to... A preprocessed local medical image Perform organ segmentation prediction, then perform voxel-level averaging of all samples to extract locally labeled organs. probability map .
[0056] Ultimately, each client generates its own probability map. Uploaded to the server.
[0057] Step 42: Server aggregation and distribution.
[0058] The server receives all local segmentation network parameters uploaded by clients and obtains the updated global segmentation network model through federated aggregation. and all client uploads By stitching along the channel dimension, a global whole-organ prediction atlas is constructed. The formal expression of this process is shown in the following formula:
[0059] .
[0060] After the update is complete, the server will display the updated version. and Distributed to various clients for a new round of local training, such as Figure 3The process is shown in (1).
[0061] Step 43: The client trains its own registration network. This invention introduces a personalized registration network that takes the global model prediction and the whole-organ prediction atlas as input, estimates the deformation field between the two, learns the spatial changes between the global prediction atlas of the whole organ and the anatomical structure of the data itself, and simulates the difference between the global ground truth atlas of the whole organ and the local anatomical structure. The registration network adopts a U-Net structure and introduces a smoothing regularization term in the loss function to maintain the continuity and rationality of the deformation field.
[0062] Within the framework of this invention, the local segmentation model and personalized registration network Update using an alternating training method. Figure 3 The process (2) describes The training process. The personalized registration network is based on the U-Net structure, and its goal is to fit the whole organ prediction atlas. The prediction results of the global segmentation model on the current local data Spatial transformation is used to estimate the corresponding voxel-level deformation field. The deformation field and Together, they are input into the spatial transformation module to generate a deformed whole-organ segmentation prediction atlas. . training loss It consists of two parts:
[0063] ;
[0064] in, For cross-entropy loss, encourage the deformed graph and They should be as similar as possible, and It is a smoothing regularization term used to suppress deformation fields. Dramatic local changes in weights These two goals were balanced. The specific calculation method is as follows:
[0065] ;
[0066] in, Voxel representation The deformation field at the first The value of each component Indicates in Unit step size in direction, . The loss is applied to each of the three spatial dimensions, and the average value is taken to obtain the final regularization term. The deformation field is predicted at the resolution of the input volume, thus ensuring the local continuity of the predicted displacement.
[0067] Step 44: Training the segmentation network guided by personalized probabilistic graphs.
[0068] like Figure 3 As shown in process (3), during the local segmentation model training phase, the parameters of the personalized registration network are frozen and used only for inference. The personalized registration network receives... and As input, the output deformation field The deformation field will Personalized truth probability maps are obtained by performing spatial transformations. , This is a spatial transformation module. This personalized truth probability map... During the local training phase on the client side, it works together with real annotations as a supervisory signal to provide more accurate guidance for the segmentation of unannotated organs.
[0069] Finally, within the framework of this invention, the first... Training loss function for the local segmentation network of each client As shown below:
[0070] ;
[0071] Among them, segmentation loss Loss due to local supervision With map supervision loss The weighted structure is used to supervise labeled organs, and the weighted structure is used to supervise unlabeled organs. Loss function. Personalized probabilistic maps are used as supervision signals to guide the local model on the client side in segmenting unlabeled organs. Used for control The strength of the loss function. The BCE loss method is also used, as in step 3. The calculation formula will Substituting into the calculation formula, the loss value is obtained. . Indicates organs Personalized truth probability graphs; Indicates the local segmentation network in the image Predicted organs The segmentation result.
[0072] Step 45: Information Upload and Iteration.
[0073] like Figure 3As shown in process (4), after completing local training, each client calculates the current global segmentation network for organ-specific segments. Predicted map and the local segmentation model The parameters are uploaded to the server. Then, the process returns to step 42, where the server performs a new round of aggregation and distribution. This process iterates continuously until the preset number of training rounds is reached, ultimately resulting in a global organ segmentation model capable of achieving high-precision segmentation on all target organs.
[0074] Step 5: Evaluation of the segmentation performance of the global multi-organ segmentation network.
[0075] After training, the performance of the obtained global segmentation network on the multi-organ segmentation task is evaluated. Specifically, experiments are conducted on multiple datasets, using several commonly used segmentation performance metrics for measurement, and comparative analysis is performed with various existing methods to verify the effectiveness and superiority of the proposed method.
[0076] Example 1: The federated multi-organ segmentation method based on partially labeled datasets of personalized organ probability maps according to the present invention is illustrated in a specific scenario. The steps are as follows:
[0077] Step 1: Define the task scenario and objective. This embodiment uses four clients, each holding abdominal CT image data, but each client only annotates the liver, kidneys, spleen, or pancreas. The objective of this embodiment is to jointly train a global multi-organ segmentation network while ensuring data privacy, enabling it to simultaneously output the segmentation results of the four organs in a single forward propagation.
[0078] Step 2: Dataset Preparation and Preprocessing. This embodiment selects four publicly available datasets: LiTS, KiTS, MSD_Task09, and MSD_Task07, as partially annotated datasets for the liver, kidney, spleen, and pancreas, respectively, to construct local data sources for the four clients. LiTS, KiTS, MSD_Task09, and MSD_Task07 contain 131, 210, 41, and 281 CT images, respectively, and are divided into training, validation, and test sets in a 6:1:3 ratio. Additionally, the federated extra-domain datasets AMOS and BTCV, containing annotations for the four organs, are introduced to evaluate the model's generalization ability. Finally, the global model that performs best on the validation set is selected and tested on both the federated and extra-federated datasets. The Dice score (DSC), Jaccard index (JC), HD95, and ASSD are used as evaluation metrics for segmentation performance.
[0079] Before formal training, the data underwent standardized preprocessing. First, the spatial orientation of all data was unified to LPS (Standard Patient Anatomical Coordinate System), and the voxel spacing was resampled according to [2.5, 2.5, 5.0] mm. Subsequently, the body region was separated from the raw CT data by setting a threshold of -325 HU and extracting the largest connected region. Further, the image grayscale values were windowed, cropping the grayscale values of the CT images to the range of [-325, 325] and normalizing them to [-1, 1] to mitigate the impact of imaging differences.
[0080] Step 3: Constructing the ground truth probability map. After data preparation and preprocessing, each client computes the supervised loss only on locally labeled organs and performs 25 rounds of federated training to obtain an initial global segmentation model for coarse segmentation. Subsequently, this initial global segmentation model is used to generate organ prediction results on local data, and the data is uniformly cropped to a voxel size of [192, 192, 96]. After cropping, the four clients perform voxel-level averaging on the ground truth segmentation results of their locally labeled organs, setting a confidence threshold of 0.1 to generate ground truth probability maps for the liver, kidney, spleen, and pancreas. Each client uploads the obtained probability maps to the server, which concatenates the four maps along the channel dimension to generate a global organ ground truth probability map and distributes it to all clients.
[0081] Step 4, Federated Training. In the formal federated training phase, the backpropagation algorithm is used to optimize the model parameters until the loss converges or the maximum number of communication rounds is reached. In this embodiment, training is performed using the PyTorch framework, with a batch size of 2, and on eight NVIDIA V100 graphics cards. Considering the differences in data volume among clients, this embodiment uniformly specifies that the number of iterations for both the local segmentation network and the registration network is 80.
[0082] Step 41, Warm-up Phase. The segmentation network adopts a multi-head U-Net structure, with encoder channels of [32, 64, 128, 256] and 4 segmentation heads. Parameters are initialized using the Kaiming Uniform method, and the optimizer uses SGD with an initial learning rate of 1e-2. The registration network also adopts a U-Net structure, with encoder channels of [16, 32, 32, 32]. To reduce computational overhead, the original input is downsampled to half resolution before being fed into the registration network. The optimizer uses Adam with an initial learning rate of 1e-4, and the learning rate for both networks is updated using an exponential decay strategy. In this phase, all clients update their local segmentation networks based on local supervised loss and generate organ-specific prediction maps using the initialized global segmentation model. The prediction maps and local segmentation model parameters are then uploaded to the server. This phase involves only one round of training.
[0083] Step 42, Server Aggregation and Distribution. The server uses the FedAvg algorithm to aggregate the segmentation network parameters uploaded by each client, obtaining an updated global segmentation model. Simultaneously, the organ-specific prediction maps from each client are concatenated along the channel dimension to form a global prediction map containing all target organs. Subsequently, the server distributes the global model and global prediction map to all clients.
[0084] Step 43: Train the client-side personalized registration network. At this point, freeze the local segmentation network and train only the personalized registration network. The registration network takes the prediction results of the global model on local data and the global prediction map as input, learns the spatial transformation relationship between the two, and thus generates a voxel-level deformation field and smooths the weights of the regularization term. Set to 0.01.
[0085] Step 44: The client guides the training of the local segmentation network based on the personalized probability map. In this stage, the personalized registration network is frozen and used only for inference. The registration network receives the prediction results from the global segmentation network based on local data and the global prediction map as input, outputs the corresponding deformation field, and applies it to the global organ ground truth probability map generated in Step 3 to obtain the personalized probability map. The personalized map and local annotations together serve as supervision signals to train the local segmentation network to complete the segmentation task of unlabeled organs. Map supervision signal weights. Set it to 0.7.
[0086] Step 45, Information Upload and Iteration. After completing local training, each client uploads the updated segmentation model parameters and the organ-specific prediction atlas of the local image generated by the global segmentation network to the server. The process then returns to step 42, where the server performs a new round of aggregation and distribution. This process is repeated iteratively until 250 rounds of federated communication are completed, ultimately obtaining a global model capable of high-precision segmentation of all target organs. The global segmentation network obtained at this point is then deployed to each client and to new clients.
[0087] Step 5: Evaluate the segmentation performance of the global segmentation network.
[0088] After the model was trained, the performance of the global segmentation network was evaluated on datasets both within and outside the federated domain. The evaluation results are shown in Tables 1 and 2. DSC and JC are expressed as percentages (%), with higher values indicating better segmentation performance. HD95 and ASSD are expressed in millimeters (mm), with lower values indicating better performance in boundary accuracy and spatial consistency.
[0089] Table 1. Comparison of the Invention Method with Existing Methods within the Federal Domain
[0090]
[0091] Table 2 Comparison of the Invention Method and Existing Methods Outside the Federal Territory
[0092]
[0093] DSC measures the overlap between the predicted segmentation result and the ground truth annotation; a value closer to 1 indicates higher segmentation accuracy. JC evaluates the ratio of the intersection to the union between the predicted segmentation and the ground truth annotation; a larger value indicates better model segmentation performance. HD95, or 95% Hausdorff distance, measures the boundary deviation between the segmentation result and the ground truth labeled surface; a smaller HD95 value indicates better boundary consistency. ASSD, or Average Symmetric Surface Distance, calculates the bidirectional average surface distance between the predicted boundary and the ground truth boundary; a smaller value indicates more consistent overall boundaries.
[0094] Using the method of this invention, in tests within the federated domain, the Dice value is 90.02%, the JC index is 83.22%, the HD95 value is 8.56 mm, and the ASSD value is 2.09 mm. For tests outside the federated domain, on the AMOS dataset, the Dice value is 77.36%, the JC index is 67.71%, the HD95 value is 30.55 mm, and the ASSD value is 12.45 mm; on the BTCV dataset, the Dice value is 82.65%, the JC index is 73.90%, the HD95 value is 16.24 mm, and the ASSD value is 5.54 mm. It can be seen that compared with existing methods, the method of this invention achieves optimal results in the above indicators and exhibits better stability.
[0095] like Figure 5 and Figure 6 As shown, the visualization comparison results on the AMOS and BTCV datasets further demonstrate that the method of the present invention can achieve more accurate and consistent multi-organ segmentation results in the axial, sagittal and coronal directions, and its performance in complex boundary regions is significantly better than that of existing methods. Figure 5 The yellow dashed box area further indicates that the FedPOPA framework involved in this invention captures organ structure better than FLKD, and can more accurately maintain organ morphology and the boundaries between adjacent organs.
[0096] like Figure 7As shown, this embodiment also quantitatively compares the similarity index between the personalized registration network and the ideal map before and after personalization of the probabilistic map. The results show that the average Dice coefficient increased from 81.3% to 83.8%, indicating a significant improvement in overall performance. The improvements were most significant for the kidney and pancreas, with the Dice coefficient for the kidney increasing from 76.0% to 83.0% and for the pancreas from 84.8% to 90.3%. These results demonstrate that the personalized registration network can effectively alleviate cross-client spatial mismatch and improve the adaptability and effectiveness of the probabilistic map in multi-source data scenarios.
[0097] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.
Claims
1. A partially labeled federated multi-organ segmentation method based on personalized probabilistic graphs, characterized in that, include: Step 1: Determine the clients in the application scenario and the organ categories labeled by each client; Step 2: Each client preprocesses the acquired local medical images to unify the spatial resolution and normalize the grayscale values. Step 3: Each client uses the pre-trained segmentation network to identify locally labeled organs from local medical images, constructs a ground truth probability map of locally labeled organs based on the recognition results of all local medical images, and uploads it to the server; the server aggregates the ground truth probability maps of locally labeled organs uploaded by all clients to obtain a ground truth probability map of all organs. And distribute it to each client; Step 4: Perform the federated training process, including: (1) Warm-up phase, which includes: server initialization of global segmentation network. The parameters are distributed to each client; each client initializes its own registration network; each client updates its local segmentation network using the global segmentation network parameters distributed by the server, and trains its local segmentation network using supervised loss based on image samples of locally labeled organs; on the other hand, it uses the global segmentation network to perform whole-organ segmentation prediction on local medical images, averages the prediction results of all images at the voxel level, and then extracts the prediction atlas corresponding to the locally labeled organs; each client uploads its local segmentation network parameters and prediction atlas to the server. (2) In the formal stage, the execution includes: step a1, the server updates the global segmentation network parameters through federated aggregation, and splices the prediction maps uploaded by all clients along the channel dimension to obtain the whole organ prediction map. The server distributes the current global segmentation network parameters and whole-organ prediction maps to each client; in step a2, each client updates the registration network; the registration network is used to fit the spatial transformation from the whole-organ prediction map to the prediction results of the global segmentation network on the local medical image, obtaining a voxel-level deformation field; in step a3, each client uses the deformation field to generate a true probability map of the whole organs. A spatial transformation is performed to obtain a personalized ground truth probability map. The personalized ground truth probability map and the local organ real annotations are used together as supervision signals to train the local segmentation network of the client. In step a4, each client uses the current global segmentation network to predict the local training samples and generate a prediction map of the locally annotated organs. The client uploads the prediction map of the locally annotated organs and the local segmentation network parameters to the server. In step a5, steps a1 to a4 are repeated until the maximum number of iterations is reached or the global segmentation network converges. The iteration is then stopped, and the obtained global segmentation network parameters are distributed to each client and new clients.
2. The method according to claim 1, characterized in that, In step two, the client preprocesses the image by: resampling the voxel spacing according to a preset uniform spatial resolution, and then windowing and normalizing the image grayscale values.
3. The method according to claim 1, characterized in that, In step three, the method for constructing the local labeled organ ground truth probability map is as follows: Let the first... One client, collected A local medical image, labeled with organs. The Otsu thresholding method was used to analyze each labeled organ. The image of the abdominal region is extracted, and the image and labels are cropped to a set size. The labels identify the organ to which each voxel in the cropped image belongs. The probability matrix; then... Voxel-level averaging of the label matrices yields the labeled organs. The probability spectrum is further used to set thresholds. , to include those in the probability graph that are less than the threshold The voxel probabilities are set to zero to obtain the local labeled organ true probability map.
4. The method according to claim 1, characterized in that, The method for obtaining the pre-trained segmentation network in step three is as follows: perform federated training for a set number of rounds. In each round of training, each client optimizes its local segmentation network based on image samples of locally labeled organs using supervised loss, and then uploads the network parameters to the server. The server updates the global segmentation network through federated aggregation. After the training rounds are completed, the server distributes the obtained global segmentation network as the pre-trained segmentation network to each client. The supervised loss consists of Dice loss and binary cross-entropy loss.
5. The method according to claim 1, characterized in that, The registration network in step four is implemented using a U-Net architecture. The registration network for each client is represented as follows: Corresponding local medical images The deformation field is , Number the images; combine the deformation field and Input spatial transformation module to generate deformed whole-organ prediction atlas ; train Loss of time The calculation is as follows: ; in, For cross-entropy loss, This refers to using a global segmentation network to process images. Results of whole-organ segmentation prediction; Weights are used for balancing. and ; This is a smoothing regularization term used to suppress drastic local changes in the deformation field, calculated as follows: ; in, It is a set of voxels of an image. It is the number of voxels in the image; It is the input deformation field; It is a voxel The deformation field at the first The value of each component Indicates in Unit step size in direction, These correspond to the width, height, and depth directions of the image, respectively.
6. The method according to claim 1, characterized in that, In step four, the segmentation network is implemented using a multi-head U-Net structure. The local segmented network of each client is represented as: Corresponding local medical images The deformation field is , Number the images; assign the atlas Spatial transformation is represented as In step a3, train the local segmentation network. Loss function used as follows: ; in, It is an image Local organs The true label, It is a local segmentation network in the image Predicted organs The segmentation results; It is a supervised loss calculated using real annotations of local organs, consisting of Dice loss and binary cross-entropy loss; The loss is calculated using a personalized truth probability map as a supervisory signal, with weights... Used to control losses The strength; It was calculated using binary cross-entropy; Indicates organs Personalized truth probability graphs; Indicates the local segmentation network in the image Predicted organs The segmentation results; It is the set of all organs to be predicted.