A federated medical image segmentation method based on client consistency and adaptive perturbation

Abstract

The application relates to the technical field of medical image segmentation, and discloses a federated medical image segmentation method based on client consistency and adaptive disturbance, which comprises the following contents: constructing a supervised client image segmentation network, an unsupervised client image segmentation network and a server global network; constructing an interaction mechanism between clients to obtain a difference loss between the clients, reducing the model distance between different types of clients to reduce the model degradation problem during global model aggregation, and completing a high-quality aggregated global model; using the performance of the unsupervised client local model to dynamically adjust the noise intensity in the learning process of the unsupervised client, adapting to the learning difficulty of different client data sets, enhancing the learning quality of the unsupervised client and thus enhancing the performance of the global model. The method is helpful for more accurate medical image segmentation in a federated semi-supervised scenario and improves the segmentation capability of the global model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image segmentation, specifically a federated medical image segmentation method based on client consistency and adaptive perturbation. BACKGROUND

[0002] Federated semi-supervised medical image segmentation is an innovative application that combines federated learning frameworks, semi-supervised learning methods, and deep learning algorithms. These technologies enable the training of high-performance segmentation models using distributed data from multiple medical institutions while protecting data privacy. Semi-supervised learning effectively utilizes limited labeled medical image data to achieve automatic segmentation of anatomical structures and lesions in images. This technology aims to help doctors and medical professionals more accurately and efficiently locate and analyze key areas in medical images, reducing their workload and improving the speed and accuracy of diagnosis. In addition, this method ensures data security while providing strong support for addressing the problem of insufficient labeled data, which has important value for clinical decision-making and early diagnosis and treatment of patients.

[0003] Federated semi-supervised medical image segmentation technology combines federated learning, semi-supervised learning, and deep learning methods to address challenges such as data privacy protection, insufficient labeled data, and model generalization in medical image segmentation. Medical image segmentation is a key task in computer-aided diagnosis, mainly used to accurately separate organs, lesions, or other anatomical structures from the background in medical images, providing important evidence for clinical diagnosis, surgical planning, and disease detection. However, this task faces many technical difficulties, including data privacy restrictions, high data labeling costs, and inconsistent data distribution.

[0004] First, medical image data such as CT, MRI, and ultrasound images often involve patient privacy and cannot be freely shared between different hospitals and medical institutions, leading to serious data silo problems and affecting the training effect of segmentation models. Traditional centralized training methods require all data to be aggregated on a unified server, but in practical applications, this approach not only violates data privacy protection regulations such as GDPR and HIPAA, but also increases the risk of data leakage. Federated learning achieves cross-institutional collaborative training by training models locally at each data source and only transmitting model parameters rather than raw data, ensuring data privacy and security.

[0005] Secondly, the annotation process of medical image data relies on professional doctors, which is time-consuming and costly, leading to a lack of large-scale annotated data. Semi-supervised learning can utilize a large amount of unlabeled data to assist supervised training through pseudo-label generation, contrastive learning, and other methods, thereby improving segmentation performance. Under the framework of federated learning, semi-supervised learning effectively alleviates the problem of insufficient labeled data by combining limited labeled data and a large amount of unlabeled data, thereby improving the performance of the model on medical image segmentation tasks.

[0006] In addition, medical image segmentation faces the problem of inconsistent data distribution. Images obtained by different institutions may differ significantly due to differences in device models, scanning parameters, and patient individuals, leading to data distribution drift and affecting the generalization ability of the model. For example, the resolution, noise level, contrast, and texture features of the image may vary. To solve these problems, federated semi-supervised learning performs collaborative training by combining multiple distributed data sources, fuses the feature distribution of different data sources, and optimizes the model locally, thereby enhancing the robustness and generalization performance of the segmentation model.

[0007] However, federated semi-supervised medical image segmentation also faces many challenges. On the one hand, due to the uneven quality of unlabeled data, the generated pseudo-labels may contain noise, affecting the accuracy of model training. On the other hand, communication and synchronization of the model between different data sources may incur additional computational and bandwidth overhead, especially in high-resolution medical image segmentation tasks. In addition, the problem of data imbalance also exists, as the number of samples for certain organs or lesions is small, causing the model to preferentially recognize classes with larger sample sizes during training, ignoring accurate segmentation of rare classes. To address this, common solutions include weighting the loss function, introducing data augmentation strategies, and designing more robust feature extraction modules.

[0008] In summary, federated semi-supervised medical image segmentation technology provides an efficient and accurate solution for medical image segmentation by protecting data privacy, making full use of unlabeled data, and collaborative learning across institutions. The application of this technology is expected to play an important role in clinical diagnosis, lesion detection, and surgical planning, promoting the development of intelligent medicine and further improving the efficiency of patient diagnosis and treatment. SUMMARY

[0009] To overcome the deficiencies of the prior art, the present application provides a federated medical image segmentation method based on client consistency and adaptive perturbation to solve the problems in the background art.

[0010] To achieve the above-mentioned purpose, the present application provides the following technical solution: a federated medical image segmentation method based on client consistency and adaptive perturbation, which comprises the following contents:

[0011] Loading case data: obtaining a heart disease segmentation database, the database naming a folder for each patient ID, each folder storing the slice data of the visual 3D graph of the case, and the slice data of the visual 3D graph being the 2D slice cut from the CT graph along one dimension; the image data of all slices corresponding to a case are first normalized and standardized in gray value, and recorded as original images; then the correct segmentation image corresponding to the case ID is obtained as the training label, and recorded as the original image label;

[0012] A federated semi-supervised medical image global segmentation network based on inter-client consistency and adaptive perturbation is constructed and trained.

[0013] A federated semi-supervised medical image global segmentation network based on inter-client consistency and adaptive perturbation is constructed and trained.

[0014] The loaded original medical image of the unsupervised client will first be sent to the unsupervised client image segmentation network, and the original medical image will be subjected to a certain degree of Gaussian noise as perturbation in the client; the trained unsupervised network will be used as a regularization term of the supervised network together with the label of the supervised client to optimize the supervised client network; after a certain number of local training, the two kinds of clients are synchronized to the server network and weighted to aggregate into a global network; effectively utilizing the labeled data and unlabeled data, a global segmentation network with rich information and strong generalization is constructed;

[0015] An unsupervised client adaptive perturbation mechanism is constructed; a certain degree of Gaussian noise is applied to the local image data of the unsupervised client; the data before and after the perturbation are simultaneously input into the unsupervised client image segmentation network for capturing relative feature information in the medical image; the performance of the local verification model is used to increase or decrease the perturbation strength with a fixed step to adapt to the optimal perturbation strength of each client with different learning difficulties; a stronger unsupervised client segmentation network is obtained;

[0016] An inter-client interaction mechanism is constructed; the optimized local model parameters of the unsupervised client are used by the supervised client as a regularization term for local supervised learning to reduce the parameter distance between the supervised model and the unsupervised model; more similar local models are obtained, which can effectively reduce the model aggregation degradation problem caused by different optimization functions when the global model is aggregated.

[0017] Preferably, the specific implementation is as follows:

[0018] The global and local image segmentation network adopts a U-Net segmentation network, which aims to generate high-quality segmentation results, especially when dealing with complex structures and small sample data; through the symmetrical encoder-decoder structure, the context information in the image is effectively captured; the local network and the global network adopt the same network structure to realize model aggregation;

[0019] The inter-client consistency strategy synchronously optimizes the supervised client using the optimized unsupervised client network parameters and the supervised client local supervised data, effectively reduces the gap between the supervised client network and the unsupervised client network parameters, reduces the model degradation problem during global model aggregation, and increases the segmentation performance of the global model;

[0020] The adaptive perturbation strategy uses consistency regularization to make the unsupervised client generalize the local data, specifically, it allows the local data of the unsupervised client to apply a certain degree of perturbation and makes the output before and after the perturbation consistent as much as possible; after each round of aggregation, the unsupervised client evaluates the local model performance and increases or decreases the perturbation strength according to a fixed step, to adapt the best perturbation strength for different learning difficulty local data images.

[0021] Preferably, the supervised client model training loss function is:

[0022] L SC =(1-γ)L SUP +γL CONS

[0023] Wherein, L SC represents the total loss of the supervised client model training; γ represents the weight of the consistency loss; L SuP represents the supervised loss; L CONS represents the inter-client consistency loss. The unsupervised client model training loss function is:

[0024]

[0025] Wherein, L UC represents the total loss of the unsupervised client model training; N represents the total number of unsupervised client data samples; U c () represents the output of the client; X i represents the input original sample of this round, X i represents the perturbed sample.

[0026] Preferably, the federal semi-supervised medical image segmentation network structure is: a symmetrical encoder-decoder structure; the overall network structure is divided into an encoding path (down-sampling part) and a decoding path (up-sampling part), both of which present a U-shaped symmetrical structure. The encoding path is mainly used for extracting high-level semantic features of the image, and gradually reduces the spatial resolution through a plurality of [c3s1, P] structures, wherein [c3s1] represents two consecutive convolution layers with a kernel size of 3*3 and a step size of 1, and each convolution layer is followed by a batch normalization (BN) and a ReLU activation function, and [P] represents a maximum pooling layer with a pooling window size of 2*2 and a step size of 2, which gradually reduces the resolution of the feature map and doubles the number of channels. The decoding path gradually restores the image resolution through the up-sampling (U) operation, and at the same time, the feature maps of the corresponding layers in the encoding path are spliced with the up-sampled feature maps through the jump connection, so as to realize the fusion of the shallow detail features and the deep semantic features, and the spliced feature maps are further extracted through [c3s1]. In the decoding process, the resolution of the feature map is doubled layer by layer, and the number of channels is halved step by step, until the size of the input image is restored. Finally, the number of channels is mapped to the number of categories through a 1*1 convolution, and a Sigmoid or Softmax activation function is used to generate a pixel-level probability map. The entire network is a fully convolutional network (FCN) without a fully connected layer, and can process input images of any size.

[0027] Preferably, the medical image segmentation network structure is U-Net.

[0028] Preferably, a multi-modal image data fusion strategy is introduced, if there are medical image data of multiple modalities, a fusion module is designed to fuse these data and input them into the network; the multi-modal image data fusion strategy includes but is not limited to feature-level fusion, decision-level fusion or pixel-level fusion.

[0029] Preferably, an attention mechanism is introduced in the image segmentation network to enhance the network's ability to capture key features, and the attention mechanism includes but is not limited to a spatial attention module, a channel attention module or a self-attention mechanism.

[0030] Preferably, a dynamic weight distribution mechanism is constructed, and in the global model aggregation process, the weight of each client model is dynamically adjusted according to the performance and data quality of the client model, and the dynamic weight distribution mechanism adjusts the weight based on the validation accuracy, data quality index or other performance index of the client model.

[0031] Compared with the prior art, the present application has the following beneficial effects:

[0032] (1) A federated semi-supervised medical image global segmentation network based on inter-client consistency and adaptive perturbation. The present invention proposes a new medical image segmentation method, which combines supervised clients and unsupervised clients, effectively integrating labeled data and unlabeled data through federated learning. Specifically, supervised clients and unsupervised clients use different image segmentation networks for training, while the unsupervised client applies Gaussian noise perturbation to the medical image as a regularization term to optimize the segmentation network of the supervised client. This method can compensate for the challenge of insufficient labeled data, especially in the medical image field, where labeled data is often expensive and difficult to obtain. Through global aggregation and weighted averaging, the present invention can achieve high-quality medical image segmentation results with strong generalization ability, significantly improving the accuracy of segmentation and the robustness of the model. Compared with the prior art, the present invention can better utilize unlabeled data, reduce the dependence on manual labeling, and enable medical image analysis to achieve excellent performance even in the case of insufficient data.

[0033] (2) Construction of an unsupervised client adaptive perturbation mechanism. In order to improve the performance of the unsupervised client image segmentation network, the present invention applies appropriate Gaussian noise perturbation to capture the relative feature information in the medical image. These perturbations not only increase the network's ability to recognize complex and diverse image features, but also enhance the model's robustness, allowing it to adapt to different client image data distributions. Through local verification performance evaluation, the perturbation strength is adjusted using a fixed step size, allowing each client to automatically adjust the perturbation strength based on the learning difficulty of its local data. This adaptive mechanism optimizes the performance of the unsupervised client model, especially when faced with high-noise or data-imbalanced medical images, it can better capture potential disease characteristics. Therefore, compared with traditional methods, the present invention is more suitable for the diversity and complexity of medical image data, improving segmentation accuracy and the ability to recognize difficult lesions.

[0034] (3) Innovation of inter-client interaction mechanism. In the present invention, a close collaboration relationship is established between the supervised clients and the unsupervised clients. By using the optimized local model parameters of the unsupervised clients as the regularization term of the supervised clients, the training process of the supervised model is guided. This mechanism effectively reduces the parameter gap between the supervised and unsupervised models, thereby reducing the model aggregation degradation problem caused by different optimization functions. When performing global model aggregation, the model parameters of all clients are integrated through weighted averaging to form a more consistent and robust global model, effectively improving the performance of the global segmentation network. Compared with traditional federated learning methods, the present invention optimizes parameter sharing and collaborative learning through inter-client model interaction, ensuring the quality of global model aggregation and reducing the bias and error caused by different client data distributions.

[0035] (4) Stronger global segmentation model generalization ability and accuracy. Through the combination of multi-client collaborative optimization, perturbation enhancement and regularization strategy during training, the invention improves the performance of medical image segmentation model in various different data and tasks. Whether in the positioning of diseases, morphological feature analysis or automatic identification of lesions, the invention can provide more accurate and highly generalizable segmentation results. In addition, by utilizing the potential information provided by unsupervised clients, the model can mine more detailed disease characteristics, especially for pathological features that are difficult to identify by conventional methods, and can provide more accurate diagnostic support.

[0036] (5) Enhanced explainability of medical image reports. Compared with existing technologies, the invention not only improves segmentation accuracy, but also enhances the explainability of model decision-making. Through the combination of multi-view analysis and perturbation mechanism, the model can more accurately identify key areas in the image and provide more detailed diagnostic information for doctors. The image features of patients will be accurately positioned by the global network, and the generated medical image report will contain more details, so that doctors can make more accurate diagnosis and treatment decisions based on the report.

[0037] Other features and advantages of the present invention will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the invention. The objects and other advantages of the present invention can be realized and obtained by the structure indicated in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The invention is a federal medical image segmentation method based on client consistency and adaptive perturbation.

[0039] Figure 2 The invention is a federal medical image segmentation method based on client consistency and adaptive perturbation. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the invention will be described in detail below with reference to the drawings in the embodiments of the invention. Obviously, the described embodiments are only a part of the embodiments of the invention, not all embodiments. Based on the embodiments in the invention, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the invention.

[0041] The invention is a federal semi-supervised medical image segmentation method based on client consistency and adaptive perturbation, according to Figure 1 , comprising the following steps:

[0042] Step one: load case data and pre-process medical image data: obtain a multi-site database for a certain disease, and name the case database folder with the ID of each different medical site. Each folder stores a dataset with different feature distribution. Normalize and standardize the image data of all views corresponding to a certain case, and record it as the original image. Introduce a multi-modal image data fusion strategy. If there are multiple modal medical image data, design a fusion module to fuse these data and input them into the network. The multi-modal image data fusion strategy includes but is not limited to feature-level fusion, decision-level fusion or pixel-level fusion.

[0043] Step two: according to the data loaded in step one, distinguish the dataset with labels to supervise the client database and the unsupervised client database without labels, and apply Gaussian disturbance to the unsupervised data to construct a disturbed database corresponding to the unsupervised client database. The specific disturbance method is:

[0044] G(x, y) = μ + P·Z

[0045] Where G(x, y) represents the disturbance value applied at position (x, y); μ is the mean of the Gaussian distribution, set to 0; P is the standard deviation of the Gaussian distribution, which controls the strength of the disturbance; Z is a random variable from a standard normal distribution with mean 0 and variance 1.

[0046] Step three: construct the client image segmentation network. Construct a convolutional neural network and distinguish between supervised client networks and unsupervised client networks. When constructing these networks, an attention mechanism is introduced to enhance the network's ability to capture key features. The attention mechanism can choose a spatial attention module, a channel attention module or a self-attention mechanism. Then input the supervised dataset into the supervised client network and the unsupervised dataset and the corresponding disturbed dataset into the unsupervised client network. Calculate the image class activation map and get the corresponding loss, where the loss of the unsupervised client is defined as:

[0047]

[0048] Where L UC represents the total loss of the unsupervised client model training; N represents the total number of unsupervised client data samples; U C () represents the output of the client; X i represents the input original sample of this round, X i represents the disturbed sample;

[0049] Step four: Inter-client feature interaction. By transferring the model of the unsupervised client to the fully supervised client, we can utilize the knowledge learned by the unsupervised client through consistency regularization and incorporate these knowledge into the model of the fully supervised client. This helps the fully supervised client to learn more robust feature representations, thus improving the performance of the global model. By calculating the consistency loss, we can encourage the model of the fully supervised client to learn consistent representations according to the model of the unsupervised client. This helps to reduce the representation space difference between the models of the unsupervised client and the fully supervised client, making it easier to achieve model aggregation and fusion during the federated learning process. This knowledge distillation method helps to alleviate the representation space difference problem to some extent. In the early stage of federated learning, we use the fully supervised client to pretrain the global model, aiming to accelerate the convergence of the global model. The total loss of the supervised client is defined as:

[0050] L SC = (1 - γ) L SUP + γ L CONS

[0051] where L SC represents the total loss of the supervised client model training; γ represents the weight of the consistency loss; L SUP represents the supervision loss; L CONS represents the inter-client consistency loss.

[0052] Step five: Dynamic adaptive perturbation strength. There is a risk of directly incorporating unsupervised clients into global model aggregation, especially when using unsupervised clients trained with consistency regularization, which can easily lead to model collapse problems. Ensuring that unsupervised clients learn as much correct knowledge as possible is a problem that must be solved. Consistency regularization is used to generalize the local data of unsupervised clients. Specifically, by perturbing the local data of unsupervised clients, the outputs before and after perturbation are made as consistent as possible. However, the degree of perturbation applied has a great influence on the training effect. If the perturbation is too light, it may increase the risk of model overfitting. The model may learn the details of the training data too confidently, leading to overfitting on unseen data and reducing the robustness of the model, making it difficult to handle diversity in real-world scenarios. On the other hand, excessive perturbation can lead to training difficulties. Therefore, it is crucial to choose the appropriate perturbation for each data set. In the federated learning paradigm, one of the key issues to improve the performance of global model aggregation is the feature shift problem. This means that there are significant differences in complexity, details, and other information between different client data. In previous work, the same degree of perturbation was applied in the federated learning paradigm, which improved the performance of the global model. This may be due to the selection of a good set of perturbations based on the performance of the global model. However, for each local model's data set, the strength of the perturbation will affect the performance of the global model during the aggregation process, as the generalization ability of the local model is not complete. Therefore, it is necessary to adaptively adjust the perturbation strength for each unsupervised client. Specifically, in each federated training round, the unsupervised client evaluates whether the current perturbation strength is appropriate before and after training, observing its performance on the validation set. Initially, the perturbation is set to a low level and tested on the validation set in each federated round. During each communication process, assume that the perturbation is increased by default, and evaluate the performance of the perturbed model on the validation set. If the performance after perturbation deteriorates, it indicates that the current perturbation is too complex, and the relevant information of the increased perturbation will not be passed to the server, but the original perturbation will be maintained, and the best perturbation strength will be recorded.

[0053] In federated learning, there can be significant differences in the quality and quantity of data from different clients, which can lead to some clients' contributions being underestimated or overestimated during global model aggregation. It is necessary to dynamically adjust the weight of each unsupervised client based on its performance and data quality; specifically, at the end of each federated round, the validation accuracy, data quality indicators, or other performance indicators of each client model are evaluated, and a weight value is calculated for each client model based on the evaluation results. The weight value can be determined based on the relative ranking of the validation accuracy, the score of the data quality indicator, or the comprehensive consideration of other performance indicators; during the global model aggregation process, a weighted average is performed based on the weight value of each client model to obtain the updated global model.

[0054] Step six: build the global model. The supervised client network and the unsupervised client network are weighted aggregated, specifically all the parameters of the model are one-to-one corresponding, the weights are calculated locally according to the performance of local verification, and the sum is calculated according to the multiplication of the locally calculated weights and the numerical value of the parameters themselves;

[0055] Step seven: global network backpropagation updates the client network. After the global network is updated, the parameters are transmitted to all client network models for the next round of update;

[0056] Step eight: calculate the binary cross-entropy loss constraint training process. In the training process, all parameters of the model are optimized: the weights of the supervised client network and the unsupervised client network are adjusted to make the binary cross-entropy loss smaller, thus obtaining the trained weights of the supervised client network and the unsupervised client network.

[0057] Example 1

[0058] This example is for medical image segmentation of ACDC and Fundus dataset. The ACDC dataset contains 1902 heart region images and their corresponding true labels, which are divided into training set, validation set and test set in the ratio of 7:2:1. Specifically, 1331 images are allocated to the training set, 380 images are allocated to the validation set, and 191 images are allocated to the test set. The resolution of the ACDC image is fixed at 224x224 pixels. The ACDC dataset is divided into 4 parts, and each part is allocated to 4 virtual clients. The Fundus dataset is from retinal fundus images from 4 different clinical centers for optic disc and cup segmentation, simulating a real-world federated learning scenario. For preprocessing, an 800x800 disc region is cropped from these datasets, and the cropped region is adjusted to 384x384 pixels as network input. All data are preprocessed, and the ratio of training, validation and test sets is also 7:2:1. The data is normalized separately, so that the intensity value has zero mean and unit variance.

[0059] The details of each step and the model parameter settings are described in detail as follows:

[0060] Step one: original image preprocessing.

[0061] Since the data formats of various medical sites are not unified, first, the gray value is normalized to the range [0, 1], then the shape and size of each image in the ACDC database is adjusted to a single-channel medical image of [1, 224, 224], and the shape and size of each image in the Fundus database is adjusted to a single-channel medical image of [1, 384, 384].

[0062] Step two: dataset division.

[0063] The corresponding site patient data is randomly shuffled, the screened medical image description database is divided using the scikit-learn library, and it is ensured that there is no overlap between the parts: 70% of the data is divided into a training set for model training; 20% of the data is divided into a validation set for selecting the optimal model; 10% of the data is divided into a test set for testing the effect of the model.

[0064] Step three: load multi-site medical image data.

[0065] Use the os module in python to traverse all.png format files in the folder where the data is stored, save the path, and then get the corresponding medical image according to the file name. Take the case whose folder name is xray_0001 stored in the’ / Fundus / data / ’ path as an example. First, get the storage path of the case, i.e. ’ / Fundus / data / Fundus_0001’, and then find the.png format file under the folder according to this path.

[0066] Step four: build an image segmentation network.

[0067] ① Build an unsupervised image segmentation network. Build U-Net as an unsupervised client image feature extraction network. The ACDC dataset inputs [1, 224, 224] size of the original medical image data, the Fundus dataset inputs [1, 384, 384] size of the original medical image data, the ACDC dataset inputs [1, 224, 224] size of the disturbed medical image data, and the Fundus dataset inputs [1, 384, 384] size of the disturbed medical image data. Calculate the output difference to get the loss.

[0068] ② Build a supervised image segmentation network. Build U-Net as a supervised client image feature extraction network. The ACDC dataset inputs [1, 224, 224] size of the original medical image data, the Fundus dataset inputs [1, 384, 384] size of the original medical image data according to the data corresponding label get loss L1, and get unsupervised client segmentation network model input corresponding data into the network get output and local network output loss calculation get loss L2.

[0069] Step five: build an unsupervised client adaptive perturbation module.

[0070] After a round of learning, the unsupervised client adjusts the perturbation parameter of the Gaussian perturbation according to the performance of the local model. If the perturbation is too light, it may increase the risk of model overfitting. The model may learn the details of the training data too confidently, leading to overfitting on unseen data and reducing the robustness of the model, making it difficult to handle the diversity in real-world scenarios. On the other hand, excessive perturbation may lead to training difficulties. Therefore, it is crucial to select an appropriate perturbation for each dataset. By dynamically adjusting this parameter, a more appropriate perturbation strength can be adapted according to the learning difficulty of different datasets to achieve better learning results.

[0071] Step six: global model aggregation.

[0072] Finally, after each client learns, the model parameters are matched according to the local performance verification effect, and are then summed and aggregated into a global model one by one before being passed to each client at the start of the next round of learning.

[0073] The present embodiment respectively conducts experiments on Fundus and ACDC databases.

[0074] Four criteria are used to evaluate the model on the Fundus and ACDC datasets, including label-based DICE accuracy, 95% Hausdorff distance, average symmetric surface distance (ASSD), and positive predictive value (PPV).

[0075] The method proposed in the present application is compared with several classic methods, including LabelOnly, FedST, FedMatch, PDCFed, and IsoFed. The LabelOnly method represents the use of only supervised client data for training, representing the lower limit of performance in this federal process. Table 1, Table 2, Table 3, and Table 4 respectively list the accuracy comparison of the present application method and other methods under the 1:3 and 2:2 ratio of supervised and unsupervised clients on the Fundus and ACDC datasets, where ours is the method of the present application.

[0076] Table 1 is a comparison of the performance of the Fundus dataset model with a ratio of supervised and unsupervised clients of 2:2

[0077]

[0078] Table 2 is a comparison of the performance of the Fundus dataset model with a ratio of supervised and unsupervised clients of 1:3

[0079]

[0080]

[0081] Table 3 is a comparison of model performance of ACDC dataset with a supervised to unsupervised client ratio of 2:2

[0082]

[0083] Table 4 is a comparison of model performance of ACDC dataset with a supervised to unsupervised client ratio of 1:3

[0084]

[0085]

[0086] The Dice coefficient is a measure of the similarity between two sets, commonly used to evaluate the overlap between predicted segmentation and ground truth labels. Its value ranges from 0 to 1, with higher values indicating more overlap between the predicted results and the ground truth labels, and better segmentation performance.

[0087] HD95 is an index that measures the similarity of the segmentation boundary, specifically calculating the 95th percentile of the maximum distance between the predicted segmentation boundary and the true boundary. It reflects how far the predicted segmentation is from the true boundary in space. The smaller the HD95 value, the closer the segmentation result is to the true boundary. This index can reveal the local differences of the segmentation boundary, so it is very important for evaluating the accuracy of the boundary.

[0088] ASSD is an index that evaluates the average distance between the predicted segmentation surface and the true segmentation surface. It calculates the average distance between the predicted surface and the true surface, taking into account the symmetry between them. ASSD takes into account the accuracy of the segmentation boundary, with smaller values indicating that the segmentation surface is closer to the true surface. The smaller the ASSD value, the closer the predicted result is to the true surface, indicating that the model has high accuracy in the segmentation boundary.

[0089] PPV (Positive Predictive Value) is a measure of the proportion of samples that are actually positive (true lesions) among all samples that are predicted to be positive (predicted lesions) by the model. It reflects the accuracy of the model, especially the accuracy of the model in predicting positive classes. The higher the PPV value, the more accurate the model's prediction, and the fewer samples that are incorrectly classified as positive.

[0090] As can be seen from the table, in all proportions of all datasets, the Dice based on labels, the method of the present application achieves State-of-the-Art (SOTA) performance; in HD95, ASSD and PPV, the method of the present application also achieves State-of-the-Art (SOTA) performance on ACDC dataset and Fundus dataset, which shows that the method of the present application has good adaptability to different types of datasets.

[0091] The method of the present application improves the aggregation efficiency and model performance when aggregating the global model by using the consistency between clients. Secondly, the method of the present application optimizes the disturbance degree in the learning process of the unsupervised client, improves the generalization ability and performance stability of the model. The combination of these technologies makes the method of the present application achieve good results on different data sets, showing its universality and scalability.

[0092] Wherein FedST, FedMatch, PDCFed and IsoFed are well known in the art, and the LabelOnly algorithm also presents the lower limit of the algorithm in the art by using only the label client, enhancing the feasibility of the algorithm.

[0093] The present application provides a new solution in the field of federated semi-supervised medical image segmentation, by comprehensively considering the design of model aggregation and optimizing the learning process of unsupervised clients, multiple modules are organically combined to form an end-to-end federated semi-supervised medical image segmentation framework, successfully coping with the challenges faced by existing methods.

Claims

1. A federated medical image segmentation method based on client consistency and adaptive perturbation, characterized in that, The method comprises the following contents: Load case data: obtain a heart disease segmentation database, the database is named folder with each patient's ID, each folder stores the slice data of the visual 3D graph of the case, the slice data of the visual 3D graph is the 2D slice cut by the CT graph along one dimension; the image data of all slices corresponding to a case are first normalized and standardized as the original image; then the correct segmentation image corresponding to the case ID is obtained as the training label, which is recorded as the original picture label; Construct and train a federated semi-supervised medical image global segmentation network based on client consistency and adaptive perturbation; Construct a federated semi-supervised medical image global segmentation network based on client consistency and adaptive perturbation, including constructing a supervised client image segmentation network, an unsupervised client image segmentation network and a server global network; The loaded original medical image of the unsupervised client will first be sent to the unsupervised client image segmentation network, and the original medical image will be subjected to a certain degree of Gaussian noise as perturbation in the client; the trained unsupervised network will be used as a regularization term of the supervised network together with the label of the supervised client to optimize the supervised client network; after a certain number of local training, the two kinds of clients are synchronized to the server network and are aggregated into a global network by weighting; the label data and the unlabeled data are effectively utilized to construct a global segmentation network with rich information and strong generalization; An unsupervised client adaptive perturbation mechanism is constructed; a certain degree of Gaussian noise is applied to the local image data of the unsupervised client; the data before and after the perturbation are simultaneously input into the unsupervised client image segmentation network for capturing the relative feature information in the medical image; the performance of the local verification model is used to increase or decrease the perturbation strength by a fixed step to adapt to the optimal perturbation strength of each client with different learning difficulties; a stronger unsupervised client segmentation network is obtained; An interaction mechanism between clients is constructed; the optimized local model parameters of the unsupervised client are used as the regularization term of the local supervised learning of the supervised client to reduce the parameter distance between the supervised model and the unsupervised model; more similar local models are obtained, which can effectively reduce the model aggregation degradation problem caused by different optimization functions when the global model is aggregated.

2. The federated medical image segmentation method based on client consistency and adaptive perturbation according to claim 1, characterized in that, The specific implementation method is as follows: Global and local image segmentation network; the structure adopts a U-Net segmentation network, which aims to generate high-quality segmentation results when dealing with complex structures and small sample data; through the symmetrical encoder-decoder structure, the context information in the image is effectively captured; the local network and the global network adopt the same network structure to realize model aggregation; Client consistency strategy; The strategy uses the optimized unsupervised client network parameters and the local supervised data of the supervised client to optimize the supervised client synchronously, effectively reduces the difference between the supervised client network and the unsupervised client network parameters, reduces the model degradation problem in global model aggregation, and increases the segmentation performance of the global model. Adaptive perturbation strategy; a consistency regularization method is used to make the unsupervised client generalize the local data, specifically, the unsupervised client's local data is allowed to apply a certain degree of perturbation and the outputs before and after the perturbation are consistent; after each round of aggregation, the unsupervised client evaluates the performance of the local model and increases or decreases the perturbation strength according to the fixed step, and adapts the optimal perturbation strength for local data images of different learning difficulty.

3. The federated medical image segmentation method based on client consistency and adaptive perturbation according to claim 1, characterized in that, The loss function of the supervised client model training is: L SC = (1 - γ) L SUP + γ L CONS wherein, L SC denotes the total loss of supervising the client model training; γ denotes the weight of the consistency loss; L SUP denotes the supervised loss; L CONS denotes the consistency loss between clients, and the loss function of the unsupervised client model training is: wherein, L UC represents the total loss of unsupervised client model training; N represents the total number of unsupervised client data samples; U C represents the output of the client; X i represents the input original sample of this round, X i ' represents the sample after perturbation.

4. The federated medical image segmentation method based on client consistency and adaptive perturbation according to claim 2, characterized in that, The federal semi-supervised medical image segmentation network structure is: a symmetrical encoder-decoder structure; the overall structure of the network is divided into an encoding path and a decoding path, both of which exhibit a U-shaped symmetrical structure, the encoding path is used to extract high-level semantic features of the image, and the spatial resolution is gradually reduced through multiple [c3s1, P] structures, where [c3s1] represents two consecutive convolution layers with a kernel size of 3x3 and a step size of 1, and each convolution layer is followed by batch normalization (BN) and ReLU activation function, while [P] represents a maximum pooling layer with a pooling window size of 2x2 and a step size of 2, which gradually reduces the resolution of the feature map and doubles the number of channels, the decoding path gradually restores the image resolution through the up-sampling (U) operation, while the feature maps of the corresponding layers in the encoding path are spliced with the up-sampled feature maps through the jump connection, thereby realizing the fusion of shallow detail features and deep semantic features, and the spliced feature maps are further extracted through [c3s1], in the decoding process, the resolution of the feature map is doubled layer by layer, while the number of channels is gradually halved, until the size of the input image is restored, finally, through a 1x1 convolution, the number of channels is mapped to the number of categories, and a Sigmoid activation function is used to generate a pixel-level probability map, the entire network is a fully convolutional network (FCN) without fully connected layers, which can process input images of any size.

5. The federated medical image segmentation method based on client consistency and adaptive perturbation according to claim 2, characterized in that, The medical image segmentation network structure is U-Net.

6. The federated medical image segmentation method based on client consistency and adaptive perturbation according to claim 1, characterized in that, A multi-modal image data fusion strategy is introduced, if there are multiple modal medical image data, a fusion module is designed to fuse these data and input them into the network; the multi-modal image data fusion strategy includes but is not limited to feature-level fusion, decision-level fusion or pixel-level fusion.

7. The federated medical image segmentation method based on client consistency and adaptive perturbation according to claim 1, characterized in that, An attention mechanism is introduced into the image segmentation network to enhance the network's ability to capture key features, the attention mechanism includes but is not limited to spatial attention module, channel attention module or self-attention mechanism.

8. The federated medical image segmentation method based on client consistency and adaptive perturbation according to claim 1, characterized in that, A dynamic weight distribution mechanism is constructed, during the global model aggregation process, the weight of each client model is dynamically adjusted according to its performance and data quality, the dynamic weight distribution mechanism adjusts the weight based on the validation accuracy, data quality index or other performance indicators of the client model.

Images