Self-adaptive medical image segmentation method based on selective online self-training passive field

Abstract

The invention discloses a passive field adaptive medical image segmentation method based on selective online self-training. According to the technical scheme, the method comprises the following steps: constructing a strong enhancement module, a weak enhancement module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective updating module and a similar-prior-guided pseudo label screening module; the passive field adaptive medical image segmentation system based on selective on-line self-training is composed of a filtering module, a filtering module and a consistency loss function calculation module. And constructing a training set and a verification set. And training the segmentation system by using the source domain model and the training set to obtain a trained segmentation system. Verifying the trained segmentation system by using a verification set, and obtaining the trained segmentation system with the most excellent performance when the segmentation performance is not improved any more; and segmenting the image by using the trained segmentation system with the most excellent performance to obtain a segmentation result. According to the method, the negative optimization problem of online self-training in passive field self-adaption can be relieved, and the segmentation accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and medical image processing, and particularly to a passive domain adaptation medical image segmentation method based on online self-training. Background Art

[0002] Image segmentation refers to dividing each pixel in an image into specific categories (i.e., segmentation classes). For example, when segmenting an abdominal image, the regions of the spleen, right kidney, left kidney, and liver in the abdomen need to be segmented out (where 0 represents other background regions in the image, 1 represents the spleen region, 2 represents the right kidney region, 3 represents the left kidney region, and 4 represents the liver region). As a fundamental task in image processing, with the development of deep learning technology, deep learning-based image segmentation has made remarkable progress in many practical fields including medical analysis, remote sensing, and industrial applications. However, when there is a domain shift between the training data and the test data, the segmentation performance of the deep learning model will decrease significantly. Here, the training data is called the source domain, and the test data is called the target domain. The source domain and the target domain belong to the same type of task, but the data distributions are different. In medical images, the domain shift problem between the source domain and the target domain is more common because the data usually comes from instruments with different specifications or is obtained using different imaging modalities (such as CT and MRI). This has prompted the research on unsupervised domain adaptation (UDA) medical image segmentation methods. This method uses the source domain data and the target domain data for model training. The source domain represents a domain different from the samples to be segmented and generally has rich labels as supervision information; the target domain represents the domain where the samples to be segmented are located and often has no label information. The goal of unsupervised domain adaptation is to reduce the impact of domain shift and maintain the model segmentation performance in the target domain. The literature "Cheng Chen, et al. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation [J]. IEEE transactions on medical imaging, 2020, 39(7): 2494-2505. The paper by Cheng Chen et al.: Unsupervised Bidirectional Cross-Modality Adaptation for Medical Image Segmentation Based on Deeply Synergistic Image and Feature Alignment" (SIFA) introduces an unsupervised domain adaptation medical image segmentation method that transforms the source domain image into an image close to the target domain at the image level and then fine-tunes the network; at the feature level, adversarial learning is performed to map data with different domain distributions to the same hidden feature space, thereby improving the segmentation performance of the model in the target domain. However, this method requires obtaining both the source domain and the target domain data simultaneously. Due to privacy issues in the medical field, the training data of the source domain is usually not accessible during the domain adaptation stage, resulting in the non-generality of this method.

[0003] Therefore, the demand for domain adaptation in areas restricted by privacy and security is urgent, making source-free domain adaptation (SFDA) of medical image segmentation a hot but challenging research direction. In source-free domain adaptation, the source domain data owner provides the neural network weights of the source domain model. To protect data privacy, the source domain data is not provided to the adaptation process. The network weights of the source domain model are trained by the source domain data owner using deep learning methods on the labeled source domain data. Therefore, source-free domain adaptation methods cannot obtain source domain data, but can use the neural network weights of the source domain model and unlabeled target domain data for domain adaptation.

[0004] The literature "Mathilde Bateson, et al. Source-free domain adaptation for image segmentation[J]. Medical Image Analysis, 2022, 82: 102617. The paper by Mathilde Bateson et al., 'Source-free Domain Adaptation for Image Segmentation' (AdaMI) proposed a source-free domain adaptation image segmentation method based on entropy minimization and domain-invariant priors. By minimizing the entropy loss on the target domain data and integrating domain-invariant priors into the loss function, the segmentation results predicted by the model are made more reliable. However, this method requires image-level labels of the target domain data, increasing the data annotation burden. Since the annotation of medical images often relies on medical experts, especially radiologists or pathology experts, even for experienced doctors, annotating a large number of medical images is very time-consuming and laborious, which results in high annotation costs and slow speeds. The literature "Serban Stan, et al. Unsupervised model adaptation for source-free segmentation of medical images[J]. Medical Image Analysis, 2024, 95: 103179. The paper by Serban Stan et al., 'Unsupervised Model Adaptation for Source-free Segmentation of Medical Images' (SFS) proposed a source-free domain adaptation image segmentation method based on source latent features. By approximating the source latent features during adaptation and minimizing the distribution distance metric to create a joint source / target embedding space, the model is adapted to the target domain. However, this method requires using a Gaussian mixture model first to obtain the source latent features of the source domain data. To protect privacy, the owner of the source domain data is likely not to provide this source latent feature, limiting the generality of this method. The literature "Yan Wang, et al. Fvp: Fourier visual prompting for source-free unsupervised domain adaptation of medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2023. The paper by Yan Wang et al., 'Fvp: Fourier Visual Prompting for Source-free Unsupervised Domain Adaptation of Medical Image Segmentation' (FVP) adopted a source-free domain adaptation image segmentation method based on offline self-training.Offline self-training first uses the source domain segmentation model to initialize a teacher model, and then uses the teacher model to predict the unlabeled target domain data to obtain pseudo labels; the generated pseudo labels are combined with the target domain data, and the student model is trained on the combined data; after the student model is trained, it becomes the teacher model to train the next student model again, and the whole process is repeated n times until convergence is reached. However, offline self-training cannot update the teacher model in time because it has to update the teacher model after training the student model. The teacher model cannot co-evolve with the student model, which limits performance improvement.

[0005] At present, the online self-training method has lifted this limitation of the offline self-training method. The difference between it and offline self-training is that the teacher model is not updated after the student model training is completed, but the teacher model parameters and the pseudo-labels predicted by the teacher model are updated every time the student model updates its parameters during the training process. However, this update is based on the assumption that the student model will be updated in the correct direction every time during the pseudo-label learning process, and thus the teacher model will also be updated in the correct direction. But in fact, the update of the student model may not be correct every time. The rapid update of the teacher model will disrupt the learning direction of the student model, resulting in negative optimization, that is, the performance of the student model will deteriorate with the increase in the number of training times. At the same time, if the pseudo-labels predicted by the teacher model are of poor quality during the pseudo-label screening process and cannot provide accurate supervision to the student model, it will also lead to negative optimization problems. In summary, the negative optimization problem is mainly affected by two processes: the pseudo-label learning process and the pseudo-label screening process.

[0006] Therefore, how to improve the passive domain adaptive medical image segmentation performance remains a technical issue of great concern to those skilled in the art. Summary of the invention

[0007] The technical problem to be solved by the present invention is that the existing passive domain adaptive medical image segmentation method requires additional labels or source domain information, the performance improvement is limited when based on offline self-training, there is negative optimization when based on online self-training, and the image segmentation performance needs to be improved. A passive domain adaptive medical image segmentation method based on selective online self-training is proposed. A selective strategy is introduced for the pseudo-label learning process and the pseudo-label screening process, and the entropy evaluation model is used to protect data privacy without the need for additional labels or source domain information, alleviate the problem of negative optimization of online self-training, and improve image segmentation performance.

[0008] To solve the above technical problems, the technical solution of the present invention is: to construct a source-free domain adaptive medical image segmentation system based on selective online self-training. The system consists of a strong augmentation module, a weak augmentation module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module. Prepare and construct the target domain dataset required for the source-free domain adaptive medical image segmentation system, and divide the target domain dataset into a training set and a validation set. Use the source domain model and the training set provided by the user to train the student model image segmentation module and the teacher model image segmentation module in the source-free domain adaptive medical image segmentation system based on selective online self-training to obtain a more robust trained source-free domain adaptive medical image segmentation system. Use the validation set to validate the student model image segmentation module after iterative training β times (β = 100). When the segmentation performance of the student model no longer improves, obtain the trained source-free domain adaptive medical image segmentation system with the best performance; finally, use the trained source-free domain adaptive medical image segmentation system with the best performance to segment the image input by the user to obtain the segmentation result of the image.

[0009] The present invention includes the following steps:

[0010] In the first step, construct a source-free domain adaptive medical image segmentation system based on selective online self-training. As Figure 1 shown, the source-free domain adaptive medical image segmentation system consists of a strong augmentation module, a weak augmentation module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module.

[0011] The strong augmentation module is connected to the student model image segmentation module. During training, the strong augmentation module receives the training set, performs strong augmentation processing on the training set images, such as random cropping, rotation, random brightness, random contrast, and random gamma transformation, to obtain the strongly augmented images of the training set, and sends the strongly augmented images of the training set to the student model image segmentation module.

[0012] The weak augmentation module is connected to the teacher model image segmentation module. During training, the weak augmentation module receives the training set, performs weak augmentation processing on the training set images, such as random cropping, rotation, and scaling, to obtain the weakly augmented images of the training set, and sends the weakly augmented images of the training set to the teacher model image segmentation module.

[0013] The student model image segmentation module is connected to the strong augmentation module, the entropy-guided selective update module, and the consistency loss function calculation module. During training, the student model image segmentation module receives the strongly augmented images of the training set from the strong augmentation module, extracts features and performs segmentation on the strongly augmented images of the training set to obtain the first segmentation probability map, and sends the first segmentation probability map to the consistency loss function calculation module. At the same time, the entropy value is calculated based on the first segmentation probability map and sent to the entropy-guided selective update module. During validation, the student model image segmentation module receives the validation set, extracts features and performs segmentation on the validation set images to obtain the validation results, and determines whether to continue training based on the validation results. When segmenting the image input by the user, the student model image segmentation module receives the image to be segmented input by the user, extracts features and performs segmentation on the image to be segmented to obtain the segmentation results of the image to be segmented. The student model image segmentation module consists of a segmentation model (denoted as the student model), and its initial parameters adopt the parameters of the source domain segmentation model.The student model adopts the same neural network structure as the source domain model, the DeepLabv2 architecture (see the literature "Liang-Chieh Chen, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(4): 834-848." The paper by Liang-Chieh Chen et al.: DeepLab: Semantic Image Segmentation Based on Deep Convolutional Networks, Atrous Convolution, and Fully Connected CRFs; the DeepLabv2 architecture is an improved version of the previous DeepLabv1 architecture by the same authors (see the literature "Liang-Chieh Chen, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[J]. arXiv preprint arXiv:1412.7062, 2014." The paper by Liang-Chieh Chen et al.: Semantic Image Segmentation Using Deep Convolutional Networks and Fully Connected CRFs)), DeepLabv2 uses dilated convolution, as a powerful tool for dense prediction tasks, dilated convolution can explicitly control the resolution of computing feature responses within the deep convolutional neural network DCNN (Deep Convolutional Neural Network), which can effectively expand the receptive field and obtain more context information without increasing the number of parameters and computational complexity; at the same time, the atrous spatial pyramid pooling (ASPP) is proposed, and ASPP parallelly adopts dilated convolutional layers with multiple sampling rates for prediction, capturing objects and image context information with multiple scales to obtain a stronger segmentation result.The feature extractor of DeepLabv2 uses the ResNet network (see the ResNet network described in the paper "Kaiming He, et al. Deep Residual Learning for Image Recognition [J]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778" by Kaiming He et al., which is the residual network ResNet in the network architecture of deep residual learning for image recognition). The ResNet network is constructed by stacking a series of residual modules (Residual Block), and the residual module is composed of a convolutional layer and a normalization layer. In specific applications, variants of the ResNet network (such as a series of networks like ResNet50 and ResNet101) are often used according to the different network depths. The ResNet network has small parameters, short training time, and good feature extraction ability.

[0014] The entropy-guided selective update module is connected to the student model image segmentation module and the teacher model image segmentation module, and its function is to judge whether to update the parameters of the teacher model image segmentation module. During training, the entropy-guided selective update module receives the entropy value from the student model image segmentation module, saves the entropy value of each round of iteration, and judges whether to update the teacher model image segmentation module according to the change of the entropy value. If the entropy value becomes smaller, it is determined that the teacher model needs to be updated, and the student model parameters are sent to the teacher model image segmentation module.

[0015] The teacher model image segmentation module is connected to the weak augmentation module, the entropy-guided selective update module, and the class prior-guided pseudo-label screening module. During training, the teacher model image segmentation module receives the weakly augmented image data of the training set from the weak augmentation module, performs feature extraction and segmentation to obtain the second segmentation probability map, and sends the second segmentation probability map to the class prior-guided pseudo-label screening module. The teacher model image segmentation module is also composed of a segmentation model (denoted as the teacher model). The teacher model also adopts the same neural network structure, the DeepLabv2 architecture, as the source domain model, and its initial parameters also adopt the parameters of the source domain segmentation model. When receiving the student model parameters from the entropy-guided selective update module, the exponential moving average method EMA (Exponential Moving Average) is used for parameter update.

[0016] The class prior-guided pseudo-label screening module is connected to the teacher model image segmentation module and the consistency loss function calculation module. During training, the class prior-guided pseudo-label screening module receives the second segmentation probability map from the teacher model image segmentation module, and gives the confidence threshold for each segmentation class according to the second segmentation probability map, that is, gives the confidence thresholds corresponding to the five segmentation classes of background, spleen, right kidney, left kidney, and liver in the abdominal image. Then, the pseudo-label is obtained by screening the segmentation class of each pixel point in the second segmentation probability map according to the confidence threshold, and the pseudo-label is sent to the consistency loss function calculation module.

[0017] The consistency loss function calculation module is connected to the student model image segmentation module and the class prior-guided pseudo-label screening module. During training, the consistency loss function calculation module receives the first segmentation probability map from the student model image segmentation module, receives the pseudo-label from the class prior-guided pseudo-label screening module, makes a consistency comparison between the first segmentation probability map and the pseudo-label, obtains the gap between the first segmentation probability map and the pseudo-label, and uses the gap as the loss function to backpropagate and update the parameters of the student model image segmentation module.

[0018] In the passive domain adaptive medical image segmentation system based on selective online self-training, the strong enhancement module, the weak enhancement module, the teacher model image segmentation module, the entropy-guided selective update module, the class prior-guided pseudo-label screening module, and the consistency loss function calculation module all assist in the training of the student model image segmentation module, only work during training, do not participate in the work of segmenting the input image to be segmented by the user, and only the trained student model image segmentation module completes the segmentation of the input image to be segmented by the user.

[0019] The second step is to construct the training set and the validation set, and the method is as follows:

[0020] 2.1 Collect abdominal medical images as the target domain dataset for passive domain adaptive medical image segmentation, and the method is as follows:

[0021] Use the medical abdominal MRI three-dimensional image dataset CHAOS (see the paper "A. Emre Kavur, et al. CHAOS challenge - combined (CT - MR) healthy abdominal organ segmentation [J]. Medical Image Analysis, 2021, 69: 101950.") by A. Emre Kavur et al. as the target domain dataset for unsupervised domain - adaptive medical image segmentation. The dataset includes abdominal images and their corresponding ground truth labels. The labels use 1, 2, 3, and 4 to label the four regions of the spleen, right kidney, left kidney, and liver respectively, and use 0 to label other background regions in the image. CHAOS has a total of 20 three - dimensional abdominal images.

[0022] 2.2 De - identify the target domain dataset for unsupervised domain - adaptive medical image segmentation, and strip personal information such as the privacy and medical history of the subjects from the images. Remove the background slices from each three - dimensional abdominal image, and only retain the slices containing the segmented objects. Then, perform two - dimensional slicing on each three - dimensional abdominal image and its three - dimensional label to obtain a series of two - dimensional images and corresponding two - dimensional labels. Combine three consecutive two - dimensional slices in the slice order to form a three - channel image. The label corresponding to this three - channel image is the two - dimensional label of the second two - dimensional slice among them. Name the processed three - channel images and corresponding two - dimensional labels according to the three - dimensional image name plus the slice order. After the above processing, the target domain dataset for unsupervised domain - adaptive medical image segmentation has a total of 492 three - channel images and corresponding two - dimensional labels, denoted as the CHAOS' dataset;

[0023] 2.3 According to the commonly used dataset partitioning method by researchers, divide the CHAOS' dataset into a training set D train , and a validation set D val . The partitioning method for the CHAOS' dataset is: the total number of images in the training set is T = 395, and the total number of images in the validation set is V = 97;

[0024] In the third step, use the gradient backpropagation method to train the unsupervised domain - adaptive medical image segmentation system based on selective online self - training constructed in the first step, and verify the segmentation accuracy of the unsupervised domain - adaptive medical image segmentation system using the validation set after every β iterations. When the segmentation accuracy no longer improves, obtain the network weight parameters of the best student model image segmentation module. The method is as follows:

[0025] 3.1 Load the model network parameters of the source domain model provided by the user (generally, the user trains the source domain model using deep learning methods on the labeled source domain data (abdominal CT images)) into the teacher model in the teacher model image segmentation module and the student model in the student model image segmentation module;

[0026] 3.2 Set the network training configuration. The optimizer uses the Adam optimizer, and the initial learning rate is set to 1×10 (4 , the weight decay is 5×10 (4 , the maximum number of iterations T ma+ = 2000, and the initial number of iterations β = 100;

[0027] 3.3 Let the number of iterations t = 1. Train one batch of images in each iteration, and there are B = 4 images in each batch; Initialize the maximum update interval K = 30;

[0028] 3.4 The strong augmentation module and the weak augmentation module simultaneously read the B images of the t-th batch from the training set D train , denoted as the tensor-form image I train . The strong augmentation module performs strong augmentation on I train (that is, performs random cropping, rotation, random brightness, random contrast, and random gamma transformation on I train ) to obtain the strongly augmented tensor QI train , and send QI train to the student model image segmentation module; The weak augmentation module performs weak augmentation on I train (that is, performs random cropping, rotation, and scaling on I train ) to obtain the weakly augmented tensor RI train , and send RI train to the teacher model image segmentation module. I train contains B images of H×W×3. Where H represents the height of the input image, W represents the width of the input image, and "3" represents the three slices included in each image; B is the number of images in a batch;

[0029] 3.5 The student model image segmentation module receives QI from the strong augmentation module train , and executes steps 3.5.1 to 3.5.3; At the same time, the teacher model image segmentation module receives RI from the weak augmentation module train , and executes steps 3.5.4 to 3.5.5.

[0030] 3.5.1 The student model image segmentation module uses the first feature extraction and segmentation method to perform feature extraction and segmentation on QI train , and the method is:

[0031] 3.5.1.1 The student model image segmentation module performs operations on QI trainFeature extraction is performed to obtain feature F1;

[0032] 3.5.1.2 The student model image segmentation module segments feature F1 to obtain the first probability map of the t-th round It is a four-dimensional tensor of size B×C×H×W, where C represents the number of categories to be segmented, C = 5, namely five segmentation categories: background, spleen, right kidney, left kidney, and liver.

[0033] 3.5.2 The student model image segmentation module sends to the consistency loss function calculation module;

[0034] 3.5.3 The student model image segmentation module calculates the entropy value of the t-th round according to the first segmentation probability map The calculation method is as shown in formula (1):

[0035]

[0036] Send to the entropy-guided selective update module, go to 3.6;

[0037] 3.5.4 The teacher model image segmentation module uses the second feature extraction and segmentation method to perform feature extraction and segmentation on RI train The method is as follows:

[0038] 3.5.4.1 The teacher model image segmentation module performs feature extraction on RI train to obtain feature F2;

[0039] 3.5.4.2 The teacher model image segmentation module segments feature F2 to obtain the second probability map of the t-th round It is also a four-dimensional tensor of size B×C×H×W.

[0040] 3.5.5 Send to the class prior-guided pseudo-label screening module, go to 3.6;

[0041] 3.6 The entropy-guided selective update module saves Adopt the selective update strategy, and send and the entropy value of the segmentation result of the student model image segmentation module when updating the teacher model in the (t-k)-th round (k is the number of iterations since the last update) are compared, and according to the comparison result, it is judged whether to update the parameters of the teacher model. The method is as follows:

[0042] 3.6.1 If t = 1, it means that there is only one round of iteration result. Let k = 1, go to 3.7; if t>1, the value of k has been obtained in the previous round of iteration, go to 3.6.2;

[0043] 3.6.2 Compare with and if it indicates that the reliability of the current result is enhanced. At this time, the performance of the student model becomes better, indicating that the network weight parameters of the student model need to be used to update the network weight parameters of the teacher model. Send the network weight parameters of the student model to the image segmentation module of the teacher model, and go to 3.6.4. Otherwise, go to 3.6.3;

[0044] 3.6.3 Determine whether k is greater than or equal to the maximum update interval K. If k ≥ K, it indicates that the network weight parameters of the student model need to be used to update the network weight parameters of the teacher model. Send the network weight parameters of the student model to the image segmentation module of the teacher model and go to 3.6.4; if k < K, do not send the network weight parameters of the student model to the image segmentation module of the teacher model, let k = k + 1, and directly go to 3.7;

[0045] 3.6.4 The image segmentation module of the teacher model uses the network weight parameters of the student model for exponential moving average EMA update to obtain the image segmentation module of the teacher model with updated network weight parameters. The EMA update method is shown in formula (2):

[0046]

[0047] where represents the network weight parameters of the teacher model at the (t - k)-th iteration, θ t represents the network weight parameters of the student model at the t-th iteration, represents the network weight parameters of the teacher model at the t-th iteration, and μ is the EMA update weight, μ = 0.99.

[0048] 3.6.5 Let k = 1 and go to 3.7.

[0049] 3.7 The pseudo-label screening module guided by class priors receives the second probability map of the t-th round from the image segmentation module of the teacher model selects pseudo-labels using a confidence threshold and sends the pseudo-labels to the consistency loss function calculation module. The method is as follows:

[0050] 3.7.1 The received from the image segmentation module of the teacher model is a four-dimensional tensor of size B × C × H × W, and each value corresponding to the four-dimensional coordinates is a probability between 0 and 1. Use

[0051]

[0052] where represents the b-th element in the first dimension, where b = 1, …, B; is a three-dimensional tensor of size C×H×W, corresponding to the probability map of the b-th image among the B images read in the t-th round, represents the maximum value among all probabilities in this three-dimensional tensor;

[0053] 3.7.2 Let b = 1, representing the b-th in the batch size B;

[0054] 3.7.3 Initialize the class serial number c = 0;

[0055] 3.7.4 Calculate the local threshold χ of the class with class serial number c of b,c , as specifically shown in formula (4):

[0056]

[0057] represents the c-th element in the first dimension, which is a two-dimensional tensor of size H×W;

[0058] 3.7.5 Combine the global confidence threshold τ and the local threshold X b,c to obtain the initial confidence threshold of the class with class serial number c of and scale the initial confidence threshold by the upper bound factor γ to obtain b,c the confidence threshold τ of the class with class serial number c of

[0059]

[0060] where the upper bound factor γ = 0.8;

[0061] 3.7.6 Calculate the class prior α of the class with class serial number c of b,c , as specifically shown in formula (6):

[0062]

[0063] where h and w respectively represent the height and width of represent the probability value corresponding to

[0064] 3.7.7 Incorporate the class prior α b,c into the confidence threshold τ b,c to obtain The final confidence threshold τ′ for class serial number c b,c , as shown in formula (7):

[0065]

[0066] where the hyperparameter σ = 0.001;

[0067] 3.7.8 If c < C - 1, let c = c + 1, and go to 3.7.4; if c = C - 1, it means that the probability map for all classes has been obtained The final confidence thresholds for all classes are τ′ b,0 , τ′ b,1 , τ′ b,2 , τ′ b,3 , τ′ b,4 , and go to 3.7.9;

[0068] 3.7.9 Filter using the confidence threshold the b-th probability map in to obtain the pseudo-label. The method is as follows:

[0069] 3.7.9.1 Initialize height h = 1;

[0070] 3.7.9.2 Initialize width w = 1;

[0071] 3.7.9.3 Divide the pixel point with coordinates (h, w) in into the class cls with the highest probability, as shown in formula (8):

[0072]

[0073] is the probability map of size C × H × W The class probability vector when the second and third dimensions of the coordinates are (h, w) in is of size C, which is the C probabilities corresponding to classifying the pixel point with coordinates (h, w) into C classes; denotes the class corresponding to the maximum probability in, denoted as cls;

[0074] 3.7.9.4 If the probability confirms that the class of the pixel point with coordinates (h, w) is cls, go to 3.7.9.5; otherwise, the class of the pixel point with coordinates (h, w) is the background, which is 0, and go to 3.7.9.5;

[0075] 3.7.9.5 If w < W, let w = w + 1, and go to 3.7.9.3; if w ≥ W, it means that the segmentation is completed for height h, and go to 3.7.9.6;

[0076] 3.7.9.6 If h < H, let h = h + 1, and go to 3.7.9.2; if h ≥ H, it means the segmentation is completed, and obtain the pseudo-labels, and go to 3.7.10;

[0077] 3.7.10 If b < B, let b = b + 1, and go to 3.7.3; if b ≥ B, it means the loop is completed, and obtain the pseudo-labels of the images in the t-th round and go to 3.7.11;

[0078] 3.7.11 The class prior-guided pseudo-label screening module sends the pseudo-labels to the consistency loss function calculation module;

[0079] 3.8 The consistency loss function calculation module receives the first probability map from the student model image segmentation module and receives the pseudo-labels from the class prior-guided pseudo-label screening module uses the pseudo-labels as the supervision signal of the first probability map to calculate the cross-entropy (CE) loss and the Dice loss and add them together to obtain the consistency loss as shown in formula (9);

[0080]

[0081] represents the number of pixel points of represents the number of pixel points of represents and the number of pixel points with the same segmentation result;

[0082] 3.9 Use the Adam optimizer to backpropagate the consistency loss and update the student model parameters;

[0083] 3.10 If t cannot be divided evenly by β, let t = t + 1, and go to 3.4; if t is an integer multiple of β, go to 3.11;

[0084] 3.11 Use the validation set D val to verify the segmentation performance of the domain-free adaptive medical image segmentation system, and save the network weight parameters when the segmentation performance no longer improves. The method is:

[0085] 3.11.1 The student model image segmentation module reads the validation set D val , including the images, labels, and names in the validation set D val ;

[0086] 3.11.2 Let the batch sequence number \(l = 1\), representing the \(l\)-th batch. Each batch contains \(B'\) validation set images, \(B'=4\), and the total number of batches is

[0087] 3.11.3 Take out the \(B'\) images of the \(l\)-th batch of the validation set, and denote these \(B'\) images as tensor-form images \(I\) val , \(I\) val which contains \(B'\) images of \(H\times W\times3\);

[0088] 3.11.4 The student model image segmentation module receives the tensor-form images \(I\) in the \(B'\) validation sets \(D\) val and performs feature extraction and segmentation on \(I\) val to obtain the segmentation results \(I'\) of the \(B'\) images in the \(l\)-th batch of the validation set; val l ;

[0089] 3.11.5 Save the segmentation results \(I'\) of the \(B'\) images in the \(l\)-th batch of the validation set, the corresponding labels mentioned in the second step, and the names of their respective image files (3D image name + slice order); l

[0090] 3.11.6 If \(l < N\) l , let \(l=l + 1\), and go to 3.11.3; if \(l\geq N\) l , it means that all the data in the validation set has been segmented, and the segmentation results of \(V\) validation set images, the corresponding labels, and the names of their respective image files are obtained, then go to 3.11.7;

[0091] 3.11.7 Use the Dice coefficient to evaluate the segmentation performance of the student model and determine whether to continue training. The method is as follows:

[0092] 3.11.7.1 Arrange the segmentation results of the \(V\) validation set images according to the names of their image files, and combine those belonging to the same 3D image in the slice order. That is, the \(V\) segmentation results of size \(H\times W\) are combined into \(J\) 3D images, namely \(X_1,X_2,\cdots,X\) j ,\(\cdots,X\) J , where \(J\) is the number of 3D images in the validation set. On the CHAOS dataset, \(J = 4\), and \(X\) j has a size of \(H\times W\times S\) j , \(S\) j is the number of slices of the \(j\)-th 3D image in the validation set, \(\sum_{j = 1}^{J}S=V\), \(j = 1,\cdots,J\). The same operation is also performed on the ground truth labels corresponding to the segmentation results of the \(V\) validation set images to obtain \(J\) 3D image labels \(Y_1,Y_2,\cdots,Y\) j \(S\) j j ,\(\cdots,Y\)​​​J ;

[0093] 3.11.7.2 Let \(j = 1\), representing the \(j\)-th 3D image in the validation set;

[0094] 3.11.7.3 For the segmentation result \(X\) of the \(j\)-th 3D image in the validation set j and the corresponding ground truth label \(Y\) j , calculate the Dice coefficients for all segmentation classes, the method is:

[0095] 3.11.7.3.1 Initialize the class \(c = 1\);

[0096] 3.11.7.3.2 Calculate the Dice coefficient \(Dice\) of \(X\) j and \(Y\) j for class \(c\) according to formula (10): j,c :

[0097]

[0098] where \(X\) j,c represents the region in the segmentation result \(X\) j that belongs to class \(c\), \(Y\) j,c represents the region in the ground truth label \(U\) j that belongs to class \(c\), \(|X\) j,c ∩Y j,c | represents the number of pixel points where the two regions overlap, \(|X\) j,c | represents the number of pixel points in the region of the segmentation result \(X\) j that belongs to class \(c\), \(|Y\) j,c | represents the number of pixel points in the region of the ground truth label \(Y\) j that belongs to class \(c\);

[0099] 3.11.7.3.3 If \(c < C - 1\), let \(c = c + 1\), go to 3.11.7.3.2; if \(c = C - 1\), it means that the Dice coefficients for all segmentation classes of the \(j\)-th 3D image have been calculated, and the Dice coefficients for all segmentation classes of the \(j\)-th 3D image, namely \(Dice\) j,1 , \(Dice\) j,2 , \(Dice\) j,3 , \(Dice\) j,4 , go to 3.11.7.4;

[0100] 3.11.7.4 If \(j < J\), let \(j = j + 1\), go to 3.11.7.3; if \(j\geq J\), it means that the Dice coefficients for \(J\) 3D images have been calculated, and the Dice coefficients of all segmentation results in the validation set compared with the ground truth labels are obtained, namely \(Dice\) 1,1 , \(Dice\) 1,2 , \(Dice\) 1,3 , \(Dice\)1,4 ,…, Dice j,1 , Dice j,2 , Dice j,3 , Dice j,4 ,…, Dice J,1 , Dice J,2 , Dice J,3 , Dice J,4 , transfer to 3.11.7.5;

[0101] 3.11.7.5 calculates the average Dice coefficients for the four segmentation classes (spleen, right kidney, left kidney, liver) of the J three-dimensional images excluding the background, obtaining the average Dice coefficients ω1, ω2, ω3, ω4 for the 4 classes.

[0102] The average Dice coefficient ω1 of the first class = (Dice 1,1 +…+ Dice j,1 …+ Dice J,1 ) / J

[0103] The average Dice coefficient ω2 of the second class = (Dice 1,2 +…+ Dice j,2 …+ Dice J,2 ) / J

[0104] The average Dice coefficient ω3 of the third class = (Dice 1,3 +…+ Dice j,3 …+ Dice J,3 ) / J

[0105] The average Dice coefficient ω4 of the fourth class = (Dice 1,4 +…+ Dice j,4 …+ Dice J,4 ) / J

[0106] 3.11.7.6 calculates the average of ω1, ω2, ω3, ω4, obtaining the overall average Dice coefficient Ω for all images and all segmentation classes in the t-th round of the validation set. t , Ω t = (ω1 + ω2 + ω3 + ω4) / 4;

[0107] 3.11.7.7 At this time, t is an integer multiple of β (see 3.10). If t = β, only save Ω t and the network weight parameters of the student model with the iteration number t, then transfer to 3.3; if t > β, transfer to 3.11.7.8;

[0108] 3.11.7.8 If t < T ma+ , transfer to 3.11.7.9; if t ≥ T ma+, save the network weight parameters of the student model at iteration t, and go to the fourth step;

[0109] 3.11.7.9 Save the average Dice coefficient Ω t and the network weight parameters of the student model at iteration t, and compare the average Dice coefficient Ω in the t-th round t with the average Dice coefficient Ω in the (t - β)-th round t(β , if Ω t > Ω t(β and (Ω t - Ω t(β ) ≥ threshold η, 0 < η < 1, preferably η = 0.001, indicating that the student model is still being optimized during training and the current iteration number may not have reached the optimal number yet, go to 3.3; if Ω t > Ω t(β and (Ω t - Ω t(β ) < η, then there is no need to continue training the image segmentation module of the student model, go to the fourth step; if Ω t ≤ Ω t(β , it indicates overfitting in training and the training effect will become worse instead. At this time, the model effect is the best at iteration (t - β). Since the network weight parameters of the student model at iteration t are loaded in the fourth step, so let the iteration number t = t - β, and go to the fourth step.

[0110] Fourth step, load the network weight parameters of the student model at iteration t into the image segmentation module of the student model to obtain the trained passive domain adaptive medical image segmentation system based on selective online self-training with the best segmentation performance.

[0111] Fifth step, use the trained passive domain adaptive medical image segmentation system based on selective online self-training with the best segmentation performance to segment the image to be segmented input by the user to obtain the image segmentation result. The method is as follows:

[0112] 5.1 The image segmentation module of the student model receives a single image I to be segmented input by the user user ;

[0113] 5.2 Use the image segmentation module of the student model to segment I user to obtain the final segmentation result, and the image segmentation ends.

[0114] The following beneficial effects can be achieved by using the present invention:

[0115] The present invention proposes a passive domain adaptive medical image segmentation method based on selective online self-training. The present invention adopts selective online self-training and anti-model negative optimization to improve the model performance and achieve adaptive segmentation of medical images. The following effects can be obtained by using the present invention:

[0116] 1. The present invention constructs a source - free domain - adaptive medical image segmentation system that integrates a strong enhancement module, a weak enhancement module, a student model image segmentation module, a teacher model image segmentation module, an entropy - guided selective update module, a class - prior - guided pseudo - label screening module, and a consistency loss function calculation module. In source - free domain - adaptive medical image segmentation, through selective online self - training, the effect of source - free domain - adaptive medical image segmentation is improved. The training set divided from the abdominal MRI three - dimensional image dataset CHAOS is used to experiment on the present invention. The present invention is compared with the three source - free domain - adaptive medical image segmentation methods FVP, AdaMI, and SFS described in the background technology. The segmentation Dice coefficient of the present invention is greatly improved compared with the above three methods. At the same time, the present invention is compared with some classic unsupervised domain - adaptive methods SynSeg - Net, AdaOutput, CycleGAN, CyCADA, and SIFA. These methods use source - domain data and cannot protect the privacy of source - domain data. Although the present invention does not use source - domain data, its segmentation performance can still be comparable to these methods that use source - domain data and even exceed them in some cases.

[0117] 2. The present invention uses selective online self - training. An entropy - guided selective update module is introduced to perform selective parameter updates on the teacher model (see 3.6 for details). The teacher model is selectively updated through the output entropy value of the student model, effectively alleviating the negative optimization problem that occurs during the training process. A class - prior - guided pseudo - label screening module is also proposed to optimize the pseudo - labels (see 3.7 for details) to alleviate the class imbalance problem in the pseudo - labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] Figure 1 It is the logical structure diagram of the source - free domain - adaptive medical image segmentation system constructed in the first step of the present invention.

[0119] Figure 2 It is the overall flow chart of the present invention.

[0120] Figure 3 It is a qualitative comparison diagram of the segmentation results of the present invention, the segmentation results of a source - free domain - adaptive image segmentation method AdaMI based on entropy minimization and domain - independent prior in the background technology, and the ground truth labels. Both rows are the segmentation results when the target domain is an MRI image, and different colors represent different organs. The other two source - free domain - adaptive medical image segmentation methods FVP and SFS in the background technology do not provide source code, so qualitative comparison is not carried out. DETAILED DESCRIPTION OF THE INVENTION

[0121] The following is a description of specific examples of the present invention with reference to the accompanying drawings. As Figure 2 shown, the present invention includes the following steps:

[0122] First, construct a source-free domain adaptation medical image segmentation system based on selective online self-training. As Figure 1 shown, the source-free domain adaptation medical image segmentation system consists of a strong augmentation module, a weak augmentation module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module.

[0123] The strong augmentation module is connected to the student model image segmentation module. During training, the strong augmentation module receives the training set, performs strong augmentation on the training set images, such as random cropping, rotation, random brightness, random contrast, and random gamma transformation, to obtain the strongly augmented images of the training set, and sends the strongly augmented images of the training set to the student model image segmentation module.

[0124] The weak augmentation module is connected to the teacher model image segmentation module. During training, the weak augmentation module receives the training set, performs weak augmentation on the training set images, such as random cropping, rotation, and scaling, to obtain the weakly augmented images of the training set, and sends the weakly augmented images of the training set to the teacher model image segmentation module.

[0125] The student model image segmentation module is connected to the strong augmentation module, the entropy-guided selective update module, and the consistency loss function calculation module. During training, the student model image segmentation module receives the strongly augmented images of the training set from the strong augmentation module, extracts features and performs segmentation on the strongly augmented images of the training set to obtain the first segmentation probability map, and sends the first segmentation probability map to the consistency loss function calculation module. At the same time, the entropy value is calculated based on the first segmentation probability map and sent to the entropy-guided selective update module. During validation, the student model image segmentation module receives the validation set, extracts features and performs segmentation on the validation set images to obtain the validation result, and determines whether to continue training based on the validation result. When segmenting the user-input image, the student model image segmentation module receives the image to be segmented input by the user, extracts features and performs segmentation on the image to be segmented to obtain the segmentation result of the image to be segmented. The student model image segmentation module consists of a segmentation model (denoted as the student model), and its initial parameters adopt the parameters of the source domain segmentation model.The student model adopts the same neural network structure as the source domain model, the DeepLabv2 architecture (see the literature "Liang-Chieh Chen, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs [J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(4): 834-848." The paper by Liang-Chieh Chen et al.: DeepLab: Semantic Image Segmentation Based on Deep Convolutional Networks, Atrous Convolution, and Fully Connected CRFs; this is an improved version based on the previous DeepLabv1 architecture by the same authors (see the literature "Liang-Chieh Chen, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs [J]. arXiv preprint arXiv:1412.7062, 2014." The paper by Liang-Chieh Chen et al.: Semantic Image Segmentation Using Deep Convolutional Networks and Fully Connected CRFs)), DeepLabv2 uses atrous convolution, as a powerful tool for dense prediction tasks. Atrous convolution can explicitly control the resolution of computing feature responses within the deep convolutional neural network DCNN (Deep Convolutional Neural Network), which can effectively expand the receptive field and obtain more context information without increasing the number of parameters and computational complexity; at the same time, the atrous spatial pyramid pooling (ASPP) is proposed. ASPP parallelly adopts multiple atrous convolution layers with different sampling rates for prediction, captures objects and image context information with multiple ratios, and obtains a stronger segmentation result.The feature extractor of DeepLabv2 uses the ResNet network (see the ResNet network described in the paper "Kaiming He, et al. Deep Residual Learning for Image Recognition [J]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778" by Kaiming He et al., which is the residual network ResNet in the network architecture of deep residual learning for image recognition). The ResNet network is constructed by stacking a series of residual modules (Residual Block), and the residual module consists of a convolutional layer and a normalization layer. In specific applications, depending on the depth of the network, variants of the ResNet network (such as a series of networks like ResNet50 and ResNet101) are often used. The ResNet network has small parameters, short training time, and good feature extraction ability.

[0126] The entropy-guided selective update module is connected to the student model image segmentation module and the teacher model image segmentation module, and its function is to judge whether to update the parameters of the teacher model image segmentation module. During training, the entropy-guided selective update module receives the entropy value from the student model image segmentation module, saves the entropy value of each round of iteration, and judges whether to update the teacher model image segmentation module according to the change of the entropy value. If the entropy value becomes smaller, it is determined that the teacher model needs to be updated, and the student model parameters are sent to the teacher model image segmentation module.

[0127] The teacher model image segmentation module is connected to the weak augmentation module, the entropy-guided selective update module, and the class prior-guided pseudo-label screening module. During training, the teacher model image segmentation module receives the weakly augmented image data of the training set from the weak augmentation module, performs feature extraction and segmentation to obtain the second segmentation probability map, and sends the second segmentation probability map to the class prior-guided pseudo-label screening module. The teacher model image segmentation module also consists of a segmentation model (denoted as the teacher model). The teacher model also adopts the same neural network structure, the DeepLabv2 architecture, as the source domain model, and its initial parameters also adopt the parameters of the source domain segmentation model. When receiving the student model parameters from the entropy-guided selective update module, the exponential moving average method EMA (Exponential Moving Average) is used for parameter update.

[0128] The class prior-guided pseudo-label screening module is connected to the teacher model image segmentation module and the consistency loss function calculation module. During training, the class prior-guided pseudo-label screening module receives the second segmentation probability map from the teacher model image segmentation module, and gives the confidence threshold for each segmentation class according to the second segmentation probability map, that is, gives the confidence thresholds corresponding to the five segmentation classes of background, spleen, right kidney, left kidney, and liver in the abdominal image. Then, the pseudo-label is obtained by screening the segmentation class of each pixel point in the second segmentation probability map according to the confidence threshold, and the pseudo-label is sent to the consistency loss function calculation module.

[0129] The consistency loss function calculation module is connected to the student model image segmentation module and the class prior-guided pseudo-label screening module. During training, the consistency loss function calculation module receives the first segmentation probability map from the student model image segmentation module, receives the pseudo-label from the class prior-guided pseudo-label screening module, makes a consistency comparison between the first segmentation probability map and the pseudo-label, obtains the gap between the first segmentation probability map and the pseudo-label, and uses the gap as the loss function to backpropagate and update the parameters of the student model image segmentation module.

[0130] In the passive domain adaptive medical image segmentation system based on selective online self-training, the strong enhancement module, the weak enhancement module, the teacher model image segmentation module, the entropy-guided selective update module, the class prior-guided pseudo-label screening module, and the consistency loss function calculation module are all used to assist the training of the student model image segmentation module. They only work during training and do not participate in the segmentation of the input image to be segmented by the user. Only the trained student model image segmentation module completes the segmentation of the input image to be segmented by the user.

[0131] The second step is to construct the training set and the validation set. The method is as follows:

[0132] 2.1 Collect abdominal medical images as the target domain dataset for passive domain adaptive medical image segmentation. The method is as follows:

[0133] Use the medical abdominal MRI three-dimensional image dataset CHAOS (see the paper "A. Emre Kavur, et al. CHAOS challenge - combined (CT - MR) healthy abdominal organ segmentation [J]. Medical Image Analysis, 2021, 69: 101950.") by A. Emre Kavur et al. as the target domain dataset for unsupervised domain adaptation medical image segmentation. The dataset includes abdominal images and their corresponding ground truth labels. The labels use 1, 2, 3, and 4 to label the four regions of the spleen, right kidney, left kidney, and liver respectively, and 0 to label other background regions in the image. CHAOS has a total of 20 three-dimensional abdominal images.

[0134] 2.2 De-identify the target domain dataset for unsupervised domain adaptation medical image segmentation, and strip personal information such as the privacy and medical history of the subjects from the images. Remove the background slices from each three-dimensional abdominal image, and only retain the slices containing the segmented objects. Then, perform two-dimensional slicing on each three-dimensional abdominal image and its three-dimensional label to obtain a series of two-dimensional images and corresponding two-dimensional labels. Combine three consecutive two-dimensional slices in the slice order to form a three-channel image, and the label corresponding to this three-channel image is the two-dimensional label of the second two-dimensional slice among them. Name the processed three-channel images and corresponding two-dimensional labels according to the three-dimensional image name plus the slice order. After the above processing, the target domain dataset for unsupervised domain adaptation medical image segmentation has a total of 492 three-channel images and corresponding two-dimensional labels, denoted as the CHAOS' dataset;

[0135] 2.3 According to the commonly used dataset partitioning method by researchers, divide the CHAOS' dataset into a training set D train , and a validation set D val . The partitioning method for the CHAOS' dataset is: the total number of images in the training set is T = 395, and the total number of images in the validation set is V = 97;

[0136] In the third step, use the gradient backpropagation method to train the unsupervised domain adaptation medical image segmentation system based on selective online self-training constructed in the first step, and use the validation set to verify the segmentation accuracy of the unsupervised domain adaptation medical image segmentation system after every β iterations. When the segmentation accuracy no longer improves, obtain the network weight parameters of the best student model image segmentation module. The method is as follows:

[0137] 3.1 Load the model network parameters of the source domain model provided by the user (usually, the user trains the source domain model using deep learning methods on the labeled source domain data (abdominal CT images)) into the teacher model in the teacher model image segmentation module and the student model in the student model image segmentation module;

[0138] 3.2 Set the network training configuration. The optimizer uses the Adam optimizer, and the initial learning rate is set to 1×10 (4 , the weight decay is 5×10 (4 , the maximum number of iterations T ma+ = 2000, and the initial number of iterations β = 100;

[0139] 3.3 Let the number of iterations t = 1. Train one batch of images in each iteration, and there are B = 4 images in each batch; Initialize the maximum update interval K = 30;

[0140] 3.4 The strong augmentation module and the weak augmentation module simultaneously read the B images of the t-th batch from the training set D train , denoted as the tensor-form image I train , the strong augmentation module performs strong augmentation on I train (that is, performs random cropping, rotation, random brightness, random contrast, and random gamma transformation on I train ) to obtain the strongly augmented tensor QI train , and send QI train to the student model image segmentation module; The weak augmentation module performs weak augmentation on I train (that is, performs random cropping and rotation on I train ) to obtain the weakly augmented tensor RI train , and send RI train to the teacher model image segmentation module. I train contains B images of H×W×3. Where H represents the height of the input image, W represents the width of the input image, and "3" represents the three slices included in each image; B is the number of images in a batch;

[0141] 3.5 The student model image segmentation module receives QI train from the strong augmentation module and executes steps 3.5.1 to 3.5.3; At the same time, the teacher model image segmentation module receives RI train from the weak augmentation module and executes steps 3.5.4 to 3.5.5.

[0142] 3.5.1 The student model image segmentation module uses the first feature extraction and segmentation method to perform feature extraction and segmentation on QI train , and the method is:

[0143] 3.5.1.1 The student model image segmentation module performs operations on Q trainFeature extraction is performed to obtain feature F1;

[0144] 3.5.1.2 The student model image segmentation module segments feature F1 to obtain the first probability map in the t-th round It is a four-dimensional tensor of size B×C×H×W, where C represents the number of categories to be segmented, C = 5, namely five segmentation categories: background, spleen, right kidney, left kidney, and liver.

[0145] 3.5.2 The student model image segmentation module sends to the consistency loss function calculation module;

[0146] 3.5.3 The student model image segmentation module calculates the entropy value in the t-th round according to the first segmentation probability map The calculation method is as shown in formula (1): The calculation method is as shown in formula (1):

[0147]

[0148] Send to the entropy-guided selective update module, go to 3.6;

[0149] 3.5.4 The teacher model image segmentation module uses the second feature extraction and segmentation method to perform feature extraction and segmentation on RI train The method is as follows:

[0150] 3.5.4.1 The teacher model image segmentation module performs feature extraction on RI train to obtain feature F2;

[0151] 3.5.4.2 The teacher model image segmentation module segments feature F2 to obtain the second probability map in the t-th round It is also a four-dimensional tensor of size B×C×H×W.

[0152] 3.5.5 Send to the class prior-guided pseudo-label screening module, go to 3.6;

[0153] 3.6 The entropy-guided selective update module saves Adopts the selective update strategy and compares with the entropy value of the segmentation result of the student model image segmentation module when updating the teacher model in the (t - k)-th round (k is the number of iterations since the last update) According to the comparison result, it judges whether to update the parameters of the teacher model. The method is as follows:

[0154] 3.6.1 If t = 1, it means there is only one round of iteration result. Let k = 1, go to 3.7; if t > 1, the value of k has been obtained in the previous round of iteration, go to 3.6.2;

[0155] 3.6.2 Compare with If it indicates that the reliability of the current result is enhanced. At this time, the performance of the student model becomes better, which means that the network weight parameters of the student model need to be used to update the network weight parameters of the teacher model. Send the network weight parameters of the student model to the image segmentation module of the teacher model, and go to 3.6.4.

[0156] Otherwise, go to 3.6.3;

[0157] 3.6.3 Determine whether k is greater than or equal to the maximum update interval K. If k ≥ K, it means that the network weight parameters of the student model need to be used to update the network weight parameters of the teacher model. Send the network weight parameters of the student model to the image segmentation module of the teacher model, and go to 3.6.4; if k < K, do not send the network weight parameters of the student model to the image segmentation module of the teacher model, let k = k + 1, and directly go to 3.7;

[0158] 3.6.4 The image segmentation module of the teacher model uses the network weight parameters of the student model for exponential moving average (EMA) update to obtain the image segmentation module of the teacher model with updated network weight parameters. The EMA update method is shown in formula (2):

[0159]

[0160] where represents the network weight parameters of the teacher model at the (t - k)-th iteration, θ t represents the network weight parameters of the student model at the t-th iteration, represents the network weight parameters of the teacher model at the t-th iteration, and μ is the EMA update weight, μ = 0.99.

[0161] 3.6.5 Let k = 1, and go to 3.7.

[0162] 3.7 The class prior-guided pseudo-label screening module receives the second probability map of the t-th round from the image segmentation module of the teacher model Selects pseudo-labels using a confidence threshold and sends the pseudo-labels to the consistency loss function calculation module. The method is as follows:

[0163] 3.7.1 The received from the image segmentation module of the teacher model is a four-dimensional tensor of size B × C × H × W, and each value corresponding to the four-dimensional coordinates is a probability between 0 and 1. Calculate the global confidence threshold τ using

[0164]

[0165] Among them represents the b-th element in the first dimension, where b = 1, …, B; is a three-dimensional tensor of size C×H×W, corresponding to the probability map of the b-th image among the B images read in the t-th round, represents the maximum value among all probabilities in this three-dimensional tensor;

[0166] 3.7.2 Let b = 1, representing the b-th in the batch size B;

[0167] 3.7.3 Initialize the class serial number c = 0;

[0168] 3.7.4 Calculate the local threshold χ of class serial number c of b,c , as specifically shown in formula (4):

[0169]

[0170] represents the c-th element in the first dimension, which is a two-dimensional tensor of size H×W;

[0171] 3.7.5 Combine the global confidence threshold τ and the local threshold χ b,c to obtain the initial confidence threshold of class serial number c of and scale the initial confidence threshold by the upper bound factor γ to obtain b,c the confidence threshold τ of class serial number c of

[0172]

[0173] where the upper bound factor γ = 0.8;

[0174] 3.7.6 Calculate the class prior α of class serial number c of b,c , as specifically shown in formula (6):

[0175]

[0176] where h and w respectively represent the height and width of , h = 1, …, H, w = 1, …, W, represents

[0177] 3.7.7 Incorporate the class prior α b,c into the confidence threshold τ b,c to obtain The final confidence threshold τ′ of class serial number c b,c , as shown in formula (7):

[0178]

[0179] where the hyperparameter σ = 0.001;

[0180] 3.7.8 If c < C - 1, let c = c + 1, go to 3.7.4; if c = C - 1, it means the probability map of all classes has been obtained The final confidence thresholds of all classes are τ′ b,0 , τ′ b,1 , τ′ b,2 , τ′ b,3 , τ′ b,4 , go to 3.7.9;

[0181] 3.7.9 Filter using the confidence threshold the b-th probability map in to obtain the pseudo-label, the method is:

[0182] 3.7.9.1 Initialize height h = 1;

[0183] 3.7.9.2 Initialize width w = 1;

[0184] 3.7.9.3 Classify the pixel point with coordinates (h, w) in into the class cls with the highest probability, as shown in formula (8):

[0185]

[0186] is the probability map of size C×H×W the class probability vector when the second and third dimensions of the coordinates are (h, w), with size C, which are the C probabilities corresponding to classifying the pixel point with coordinates (h, w) into C classes; represents the class corresponding to the maximum probability in, denoted as cls;

[0187] 3.7.9.4 If the probability confirms that the class of the pixel point with coordinates (h, w) is cls, go to 3.7.9.5; otherwise, the class of the pixel point with coordinates (h, w) is the background, which is 0, go to 3.7.9.5;

[0188] 3.7.9.5 If w < W, let w = w + 1, go to 3.7.9.3; if w ≥ W, it means the segmentation is completed for height h, go to 3.7.9.6;

[0189] 3.7.9.6 If h < H, let h = h + 1, and go to 3.7.9.2; if h ≥ H, it means the segmentation is completed, and the pseudo-labels are obtained, then go to 3.7.10;

[0190] 3.7.10 If b < B, let b = b + 1, and go to 3.7.3; if b ≥ B, it means the loop is completed, and the pseudo-labels of the images in the t-th round are obtained then go to 3.7.11;

[0191] 3.7.11 The class prior-guided pseudo-label screening module sends the pseudo-labels to the consistency loss function calculation module;

[0192] 3.8 The consistency loss function calculation module receives the first probability map from the student model image segmentation module and receives the pseudo-labels from the class prior-guided pseudo-label screening module takes the pseudo-labels as the supervision signal of the first probability map to calculate the cross entropy (CE) loss and the Dice loss and add them together to get the consistency loss as shown in formula (9);

[0193]

[0194] denotes the number of pixel points of denotes the number of pixel points of denotes and the number of pixel points with the same segmentation result;

[0195] 3.9 Use the Adam optimizer to backpropagate the consistency loss to update the student model parameters;

[0196] 3.10 If t cannot be divided evenly by β, let t = t + 1, and go to 3.4; if t is an integer multiple of β, go to 3.11;

[0197] 3.11 Use the validation set D val to verify the segmentation performance of the unsupervised domain adaptation medical image segmentation system. When the segmentation performance no longer improves, save the network weight parameters. The method is:

[0198] 3.11.1 The student model image segmentation module reads the validation set D val , including the images, labels, and names in the validation set D val ;

[0199] 3.11.2 Let the batch sequence number \(l = 1\), representing the \(l\)th batch. Each batch contains \(B'\) validation set images, where \(B'=4\). The total number of batches is

[0200] 3.11.3 Take out the \(B'\) images of the \(l\)th batch of the validation set, and denote these \(B'\) images as tensor-form images \(I\) val , \(I\) val which contains \(B'\) images of \(H\times W\times3\);

[0201] 3.11.4 The student model image segmentation module receives the tensor-form images \(I\) in the \(B'\) validation sets \(D\) val and performs feature extraction and segmentation on \(I\) val to obtain the segmentation results \(I\) of the \(B'\) images in the \(l\)th batch of the validation set val ; l

[0202] 3.11.5 Save the segmentation results \(I\) of the \(B'\) images in the \(l\)th batch of the validation set, the corresponding labels mentioned in the second step, and the names of their respective image files (3D image name + slice order); l

[0203] 3.11.6 If \(l < N\) l , let \(l=l + 1\), and go to 3.11.3; if \(l\geq N\) l , it means that all the data in the validation set has been segmented. Obtain the segmentation results of \(V\) validation set images, the corresponding labels, and the names of their respective image files, and go to 3.11.7;

[0204] 3.11.7 Use the Dice coefficient to evaluate the segmentation performance of the student model and determine whether to continue training. The method is as follows:

[0205] 3.11.7.1 According to the names of their image files, combine the segmentation results of the \(V\) validation set images. Combine those belonging to the same 3D image in the slice order. That is, the \(V\) segmentation results of size \(H\times W\) are combined into \(J\) 3D images, namely \(x_1,x_2,\cdots,x\) j ,\(\cdots,x\) J , where \(J\) is the number of 3D images in the validation set. On the CHAOS dataset, \(J = 4\). \(X\) j is of size \(H\times W\times S\) j , \(S\) j is the number of slices of the \(j\)th 3D image in the validation set. \(\sum\) j \(S\) j \(=V\), \(j = 1,\cdots,J\). Perform the same operation on the ground truth labels corresponding to the segmentation results of the \(V\) validation set images to obtain \(J\) 3D image labels \(Y_1,Y_2,\cdots,Y\) j ,\(\cdots,Y\)J ;

[0206] 3.11.7.2 Let j = 1, representing the j-th 3D image in the validation set;

[0207] 3.11.7.3 For the segmentation result X j and the corresponding ground truth label Y j of the j-th 3D image in the validation set, calculate the Dice coefficients for all segmentation classes, the method is:

[0208] 3.11.7.3.1 Initialize the class c = 1;

[0209] 3.11.7.3.2 Calculate the Dice coefficient Dice j of X j and Y j,c for class c according to formula (10):

[0210]

[0211] where X j,c represents the region in the segmentation result X j that belongs to class c, Y j,c represents the region in the ground truth label Y j that belongs to class c, |X j,c ∩Y j,c | represents the number of pixel points where the two regions overlap, |X j,c | represents the number of pixel points in the region of the segmentation result X j that belongs to class c, |Y j,c | represents the number of pixel points in the region of the ground truth label Y j that belongs to class c;

[0212] 3.11.7.3.3 If c < C - 1, let c = c + 1, go to 3.11.7.3.2; if c = C - 1, it means that the Dice coefficients for all segmentation classes of the j-th 3D image have been calculated, and the Dice coefficients for all segmentation classes of the j-th 3D image are obtained, namely Dice j,1 , Dice j,2 , Dice j,3 , Dice j,4 , go to 3.11.7.4;

[0213] 3.11.7.4 If j < J, let j = j + 1, go to 3.11.7.3; if j ≥ J, it means that the Dice coefficients for J 3D images have been calculated, and the Dice coefficients for all segmentation results in the validation set compared with the ground truth labels are obtained, namely Dice 1,1 , Dice 1,2 , Dice 1,3 , Dice1,4 ,…, Dice j,1 , Dice j,2 , Dice j,3 , Dice j,4 ,…, Dice J,1 , Dice J,2 , Dice J,3 , Dice J,4 , transfer to 3.11.7.5;

[0214] For the four segmentation classes (spleen, right kidney, left kidney, liver) of the J three-dimensional images excluding the background, calculate the average Dice coefficient respectively to obtain the average Dice coefficients ω1, ω2, ω3, ω4 of the 4 classes.

[0215] The average Dice coefficient ω1 of the first class = (Dice 1,1 +…+ Dice j,1 …+ Dice J,1 ) / J

[0216] The average Dice coefficient ω2 of the second class = (Dice 1,2 +…+ Dice j,2 …+ Dice J,2 ) / J

[0217] The average Dice coefficient ω3 of the third class = (Dice 1,3 +…+ Dice j,3 …+ Dice J,3 ) / J

[0218] The average Dice coefficient ω4 of the fourth class = (Dice 1,4 +…+ Dice j,4 …+ Dice J,4 ) / J

[0219] For ω1, ω2, ω3, ω4, calculate the average to obtain the overall average Dice coefficient Ω of all images and all segmentation classes in the t-th round of the validation set. t , Ω t = (ω1 + ω2 + ω3 + ω4) / 4;

[0220] At this time, t is an integer multiple of β (see 3.10). If t = β, only save Ω t and the network weight parameters of the student model with the iteration number t, then transfer to 3.3; if t > β, transfer to 3.11.7.8;

[0221] 3.11.7.8 If t < T ma+ , transfer to 3.11.7.9; if t ≥ T ma+, save the network weight parameters of the student model at iteration t, and go to the fourth step;

[0222] 3.11.7.9 Save the average Dice coefficient Ω t and the network weight parameters of the student model at iteration t, compare the average Dice coefficient Ω in the t-th round t with the average Dice coefficient Ω in the (t - β)-th round t(β , if Ω t > Ω t(β and (Ω t - Ω t(β ) ≥ threshold η, 0 < η < 1, preferably η = 0.001, it indicates that the student model is still being optimized during training, and the current iteration number may not have reached the optimal number yet. Go to 3.3; if Ω t > Ω t(β and (Ω t - Ω t(β ) < η, then there is no need to continue training the image segmentation module of the student model. Go to the fourth step; if Ω t ≤ Ω t(β , it indicates that overfitting occurs during training, and the training effect will become worse instead. At this time, the model effect is the best at iteration (t - β). Since the network weight parameters of the student model at iteration t are loaded in the fourth step, so let the iteration number t = t - β, and go to the fourth step.

[0223] Fourth step, load the network weight parameters of the student model at iteration t for the image segmentation module of the student model, and obtain the trained passive domain adaptive medical image segmentation system based on selective online self-training with the best segmentation performance.

[0224] Fifth step, use the trained passive domain adaptive medical image segmentation system based on selective online self-training with the best segmentation performance to segment the image to be segmented input by the user, and obtain the image segmentation result. The method is as follows:

[0225] 5.1 The image segmentation module of the student model receives an image I to be segmented input by the user user ;

[0226] 5.2 Use the image segmentation module of the student model to segment I user to obtain the final segmentation result, and the image segmentation ends.

[0227] To verify the effectiveness of the present invention, the abdominal MRI three-dimensional image dataset CHAOS is selected to replace the user's input image, and qualitative and quantitative comparisons are made on the segmentation results of the present invention. Among them, the quantitative comparison uses the segmentation Dice coefficient and ASSD. The experiment is carried out on a server, and the experimental environment is Ubuntu 20.04 (a version of the Linux system), and an NVIDIA Quadro RTX 6000 GPU with 24GB of memory is used. The validation set of the MRI image dataset CHAOS' is input into the passive domain adaptive image segmentation system of the present invention, the data in the validation set is segmented, the segmentation Dice coefficient and ASSD are statistically calculated and compared with other methods to measure the effectiveness of the present invention.

[0228] First, a qualitative comparison is made between the segmentation result of the present invention and the segmentation result of AdaMI, a passive domain adaptive image segmentation method based on entropy minimization and domain-independent prior in the background art. The source codes of the other two passive domain adaptive medical image segmentation methods FVP and SFS in the background art are not provided, so qualitative comparison cannot be carried out. As Figure 3 shown, the first column of images is the user-input MRI abdominal image, the second column is the segmentation prediction result of the AdaMI method, the third column is the segmentation prediction result of the present invention, and the fourth column is the ground truth label. Different colors represent different organs. From Figure 3 it can be seen that the segmentation prediction result of the present invention is more complete and closer to the ground truth label than AdaMI, indicating that the present invention can well segment and predict abdominal image organs.

[0229] Then, a quantitative comparison is carried out. First, the performance evaluation indexes of the image segmentation algorithm are defined. This experiment adopts the medical image segmentation performance evaluation method, which has 2 specific indexes: Dice and ASSD. Dice represents the Dice coefficient, that is, twice the area of the correctly segmented region divided by the union of the prediction result and the label. The larger the Dice value, the better the segmentation effect; ASSD is the abbreviation of Average Symmetric Surface Distance, that is, the average symmetric surface distance. The smaller the ASSD value, the better the segmentation effect.

[0230] According to the experimental results of the present invention, the experimental results of the abdominal MRI three-dimensional image dataset CHAOS are analyzed. The source domain model is trained on the abdominal CT image dataset, so it is an adaptation from CT images to MRI images. The quantitative performance comparison of the present invention on the CHAOS dataset is shown in Table 1. "No adaptation" represents directly using the source domain CT data segmentation model to segment MRI data, and the segmentation performance at this time is very poor, representing the lower bound of the segmentation performance, indicating the necessity of domain adaptation; "Fully supervised" represents training a segmentation model on MRI data to segment MRI data in the case of having labels, representing the most ideal segmentation performance, that is, the upper bound of the segmentation performance; the five middle methods (SynSeg-Net, AdaOutput, CycleGAN, CyCADA, SIFA) are unsupervised domain adaptation methods, and they do not meet the source-free setting. The results in the table are from the paper "Cheng Chen, et al. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation [J]. IEEE transactions on medical imaging, 2020, 39(7): 2494-2505."; the bottom row shows the source-free domain adaptation method, which is the performance comparison between the present invention and the current SOTA (State-of-the-Arts, best performance) source-free domain adaptation medical image segmentation methods SFS, the previous source-free domain adaptation medical image segmentation methods AdaMI, and FVP. It can be seen from the experimental results that the present invention can perform image segmentation quickly and accurately, achieving a consistent performance improvement. The present invention is superior to all other SFDA methods on the right kidney, left kidney, and spleen, achieving the highest average Dice coefficient of 87.6% and the lowest average ASSD of 0.9.Among them, FVP and SFS do not provide code, and the results in the table are from the literature "Yan Wang, et al. Fvp: Fourier visual prompting for source-free unsupervised domain adaptation of medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2023." and the literature "Serban Stan, et al. Unsupervised model adaptation for source-free segmentation of medical images[J]. Medical Image Analysis, 2024, 95:103179."; AdaMI provides the source code, and the present invention uses the source code provided by it to conduct experiments on the CHAOS dataset.

[0231] Table 1

[0232]

[0233]

[0234] In summary, on the CHAOS dataset, the present invention has achieved a new SOTA (best performance) in the field of passive domain adaptive medical image segmentation. As can be seen from Table 1, in the comparison of the mean column of Dice, compared with 57.7% of "no adaptation", the Dice value of the present invention has been significantly improved, reaching 87.6%, indicating the necessity of domain adaptation; compared with the other passive domain adaptive methods FVP, AdaMI, and SFS at the bottom, the Dice value of the present invention has increased by 4.1% compared with the largest 83.5% among them, indicating that the present invention has a greater improvement and can achieve better segmentation results; compared with the five classic unsupervised domain adaptive methods SynSeg-Net, AdaOutput, CycleGAN, CyCADA, and SIFA in the middle, the Dice value of the present invention is also higher, indicating that the segmentation prediction result of the present invention is even better than that of the active domain method. In the comparison of the ASSD value, the ASSD value of the present invention is smaller than all the unsupervised domain adaptive methods in the middle and all the passive domain adaptive methods below, only 0.9, approaching the upper bound of "fully supervised" of 0.6. This indicates that the organs segmented by the present invention are closer to the organs of the true label in position and the boundary distances are similar. The Dice and ASSD values of the present invention have achieved the best results compared with other passive domain adaptive medical image segmentation methods, indicating that the present invention has achieved the best segmentation effect.

Claims

1. A passive domain adaptive medical image segmentation method based on selective online self-training, characterized in that The following steps are involved: The first step is to build a passive domain adaptive medical image segmentation system based on selective online self-training; The passive domain adaptive medical image segmentation system consists of a strong enhancement module, a weak enhancement module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module; The strong enhancement module is connected to the student model image segmentation module; during training, the strong enhancement module receives the training set, performs strong enhancement processing on the training set image, obtains the strongly enhanced image of the training set, and sends the strongly enhanced image of the training set to the student model image segmentation module; The weak enhancement module is connected to the teacher model image segmentation module; during training, the weak enhancement module receives the training set, performs weak enhancement processing on the training set image, obtains the weakly enhanced image of the training set, and sends the weakly enhanced image of the training set to the teacher model image segmentation module; The student model image segmentation module is connected to the strong enhancement module, the entropy-guided selective update module and the consistency loss function calculation module; during training, the student model image segmentation module receives the image of the training set after strong enhancement from the strong enhancement module, performs feature extraction and segmentation on the image of the training set after strong enhancement, obtains a first segmentation probability map, and sends the first segmentation probability map to the consistency loss function calculation module; at the same time, the entropy value is calculated according to the first segmentation probability map, and the entropy value is sent to the entropy-guided selective update module; during verification, the student model image segmentation module receives the verification set, performs feature extraction and segmentation on the verification set image, obtains the verification result, and determines whether to continue training according to the verification result; when segmenting the image input by the user, the student model image segmentation module receives the image to be segmented input by the user, performs feature extraction and segmentation on the image to be segmented, and obtains the segmentation result of the image to be segmented; the student model image segmentation module consists of a segmentation model, namely the student model, and its initial parameters adopt the parameters of the source domain segmentation model; the student model adopts the same neural network structure DeepLabv2 architecture as the source domain model; The entropy-guided selective update module is connected to the student model image segmentation module and the teacher model image segmentation module, and its function is to determine whether to update the parameters of the teacher model image segmentation module; during training, the entropy-guided selective update module receives the entropy value from the student model image segmentation module, and saves the entropy value of each round of iteration, and determines whether to update the teacher model image segmentation module according to the change of the entropy value; if the entropy value becomes smaller, it is determined that the teacher model needs to be updated, and the student model parameters are sent to the teacher model image segmentation module; The teacher model image segmentation module is connected to the weak enhancement module, the entropy-guided selective update module and the class prior-guided pseudo-label screening module; during training, the teacher model image segmentation module receives the weakly enhanced image data of the training set from the weak enhancement module, performs feature extraction and segmentation, obtains the second segmentation probability map, and sends the second segmentation probability map to the class prior-guided pseudo-label screening module; the teacher model image segmentation module is also composed of a segmentation model, namely the teacher model; the teacher model adopts the same neural network structure DeepLabv2 architecture as the source domain model, and its initial parameters adopt the parameters of the source domain segmentation model; when the student model parameters are received from the entropy-guided selective update module, the exponential moving average method EMA is used to update the parameters; The pseudo-label screening module guided by class priors is connected to the teacher model image segmentation module and the consistency loss function calculation module; during training, the pseudo-label screening module guided by class priors receives the second segmentation probability map from the teacher model image segmentation module, and gives the confidence thresholds corresponding to the five segmentation classes of background, spleen, right kidney, left kidney and liver in the abdominal image according to the second segmentation probability map; then, the segmentation class of each pixel point in the second segmentation probability map is screened according to the confidence threshold to obtain the pseudo-label, and the pseudo-label is sent to the consistency loss function calculation module; The consistency loss function calculation module is connected to the student model image segmentation module and the pseudo-label screening module guided by class priors; during training, the consistency loss function calculation module receives the first segmentation probability map from the student model image segmentation module, receives the pseudo-label from the pseudo-label screening module guided by class priors, performs consistency comparison on the first segmentation probability map and the pseudo-label, obtains the gap between the first segmentation probability map and the pseudo-label, and uses the gap as the loss function to back-propagate and update the parameters of the student model image segmentation module; In the passive domain adaptive medical image segmentation system based on selective online self-training, the strong enhancement module, weak enhancement module, teacher model image segmentation module, entropy-guided selective update module, class prior-guided pseudo-label screening module, and consistency loss function calculation module are all used to assist the student model image segmentation module in training. They only work during training and do not participate in the segmentation of the image to be segmented input by the user. Only the trained student model image segmentation module completes the segmentation of the image to be segmented input by the user. The second step is to construct the training set and the validation set by: 2.1 Collect abdominal medical images as the target domain dataset for passive domain adaptive medical image segmentation. The method is: use the medical abdominal MRI three-dimensional image dataset CHAOS as the target domain dataset for passive domain adaptive medical image segmentation; the dataset includes abdominal images and their corresponding true labels. The labels are marked with 1, 2, 3, and 4 for the spleen, right kidney, left kidney, and liver, respectively, and 0 for other background areas in the image; CHAOS contains a total of 20 three-dimensional abdominal images; 2.2 Anonymize the target domain dataset for passive domain adaptive medical image segmentation, stripping personal information such as the privacy and medical history of the subjects from the images; remove the background slices from each three-dimensional abdominal image, only retaining the slices containing the segmentation object, and then slice each three-dimensional abdominal image and its three-dimensional label into two-dimensional slices to obtain a series of two-dimensional images and corresponding two-dimensional labels; compose three consecutive two-dimensional slices in the slice order into a three-channel image, and the label corresponding to this three-channel image is the two-dimensional label of the second two-dimensional slice among them; name the processed three-channel images and corresponding two-dimensional labels according to the three-dimensional image name plus the slice order; after the above processing, the target domain dataset for passive domain adaptive medical image segmentation totals 492 three-channel images and corresponding two-dimensional labels, denoted as the CHAOS’ dataset; 2.3 According to the dataset division method commonly used by researchers, the CHAOS dataset is divided into training set D train , validation set D val , the total number of images in the training set is T, the total number of images in the validation set is V, T and V are positive integers and T+V=492; In the third step, use the gradient backpropagation method to train the passive domain adaptive medical image segmentation system based on selective online self-training constructed in the first step, and verify the segmentation accuracy of the passive domain adaptive medical image segmentation system using the validation set after every β iterations. When the segmentation accuracy no longer improves, obtain the network weight parameters of the best student model image segmentation module. The method is as follows: 3.1 Load the model network parameters of the source domain model provided by the user into the teacher model in the teacher model image segmentation module and the student model in the student model image segmentation module; 3.2 Set the network training configuration, including selecting the optimizer, initializing the learning rate, weight decay, and the maximum number of iterations T (ax , number of iterations β, T (ax and β are both positive integers greater than 1; 3.3 Let the iteration number t = 1. Train one batch of images in each iteration. Each batch has a total of B images. Initialize the maximum update interval K. Both B and K are positive integers; 3.4 Strong enhancement module and weak enhancement module are simultaneously extracted from the training set D train Read the B images of the tth batch in the image, recorded as tensor form image I train , strong enhancement module for I train Perform strong enhancement to obtain the strongly enhanced tensor QI train , QI train Send to the student model image segmentation module; weak enhancement module for I train Perform weak enhancement to obtain the weakly enhanced tensor RI train , change RI train Send to the teacher model image segmentation module; I train contains B H×W×3 images, where H represents the height of the input image, W represents the width of the input image, and "3" represents the three slices contained in each image; 3.5 Student model image segmentation module receives QI from the strong enhancement module train , execute steps 3.5.1 to 3.5.3; at the same time, the teacher model image segmentation module receives RI from the weak enhancement module train , execute steps 3.5.4 to 3.5.5; 3.5.1 Student model image segmentation module uses the first feature extraction and segmentation method to QI train Perform feature extraction and segmentation by: 3.5.1.1 Student Model Image Segmentation Module for QI train Perform feature extraction to obtain feature F1; 3.5.1.2 The student model image segmentation module segments feature F1 and obtains the first probability map of round t is a four-dimensional tensor of size B×C×H×W, where C represents the number of categories to be segmented, C=5, i.e., background, spleen, right kidney, left kidney, and liver. 3.5.2 Student model image segmentation module Send to the consistency loss function calculation module; 3.5.3 Student model image segmentation module based on the first segmentation probability map Calculate the entropy value of the tth round The calculation method is shown in formula (1): Will Send to the entropy-guided selective update module, go to 3.6; 3.5.4 Teacher model image segmentation module uses the second feature extraction and segmentation method to RI train Perform feature extraction and segmentation by: 3.5.4.1 Teacher Model Image Segmentation Module for RI train Perform feature extraction to obtain feature F2; 3.5.4.2 The image segmentation module of the teacher model segments feature F2 to obtain the second probability map of the tth round It is also a four-dimensional tensor of size B×C×H×W; 3.5.5 Send to the pseudo-label screening module guided by class priors, go to 3.6; 3.6 Entropy-guided Selective Update Module Preservation Adopting a selective update strategy, And the entropy value of the segmentation result of the student model image segmentation module when updating the teacher model in the tkth round Compare and determine whether to update the parameters of the teacher model based on the comparison results. k is the number of iterations since the last update. The method is: 3.6.1 If t = 1, it means there is only one round of iteration result. Let k = 1 and go to 3.7; if t > 1, the value of k has been obtained in the previous round of iteration, go to 3.6.2; 3.6.2 and For comparison, if This indicates that the reliability of the current result has been enhanced. At this time, the effect of the student model has improved, indicating that the network weight parameters of the student model need to be used to update the network weight parameters of the teacher model. The network weight parameters of the student model are sent to the image segmentation module of the teacher model. Go to 3.6.

4. Otherwise, go to 3.6.

3. 3.6.3 If k ≥ K, it means that the network weight parameters of the teacher model need to be updated using the network weight parameters of the student model. Send the network weight parameters of the student model to the teacher model image segmentation module and go to 3.6.4; if k < K, let k = k + 1 and directly go to 3.7; 3.6.4 The teacher model image segmentation module performs exponential moving average EMA update using the network weight parameters of the student model to obtain the teacher model image segmentation module with updated network weight parameters; 3.6.5 Let k = 1 and go to 3.7; 3.7 Class Prior-Guided Pseudo-Label Filtering Module Receives the Second Probability Map of Round t from the Teacher Model Image Segmentation Module Use the confidence threshold to select pseudo labels and send the pseudo labels to the consistency loss function calculation module as follows: 3.7.1 Received from the Teacher Model Image Segmentation Module It is a four-dimensional tensor of size B×C×H×W. Each value corresponding to the four-dimensional coordinate is a probability between 0 and 1. Calculate the global confidence threshold τ, as shown in formula (3): in express The b-th element in the first dimension of , b = 1,…,B; is a three-dimensional tensor of size C×H×W, corresponding to the probability map of the bth image among the B images read in the tth round, Represents the maximum value of all probabilities in this three-dimensional tensor; 3.7.2 Let b = 1, representing the b-th in the batch size B; 3.7.3 Initialize the class serial number c = 0; 3.7.4 Calculation The local threshold χ of the category number c b,c , as shown in formula (4): express The cth element in the first dimension is a two-dimensional tensor of size H×W; 3.7.5 Set the global confidence threshold τ and the local threshold X b,c Combined, we get The category number is the initial confidence threshold of c, and the initial confidence threshold is scaled by the upper bound factor γ to obtain The confidence threshold τ of the category number c b,c , as shown in formula (5): where the upper bound factor γ = 0.8; 3.7.6 Calculation The class prior α of the class number c b,c , as shown in formula (6): Where h and w represent The height and width of h=1,…,H,w=1,…,W, represent The probability value corresponding to the coordinates in the four dimensions (b, c, h, w); 3.7.7 Class Prior α b,c Incorporate into the confidence threshold τ b,c In, get The final confidence threshold τ′ of the category number c b,c , as shown in formula (7): where the hyperparameter σ = 0.001; 3.7.8 If c < C - 1, let c = c + 1, and go to 3.7.4; if c = C - 1, it means the probability graph has been obtained. The final confidence threshold for all categories is τ'. b,0 , τ' b,1 , τ' b,1 , τ' b,3 , τ' b,4 , go to 3.7.9; 3.7.9 Filtering using confidence threshold The bth probability graph in get Pseudo labels, go to 3.7.10; 3.7.10 If b < B, set b = b + 1 and go to 3.7.3; if b ≥ B, it means the loop is completed and the pseudo-label of the image in the t-th round is obtained Go to 3.7.11; 3.7.11 Class Prior-Guided Pseudo-Label Filtering Module Send to the consistency loss function calculation module; 3.8 The consistency loss function calculation module receives the first probability map from the student model image segmentation module Receive pseudo labels from the pseudo label filtering module guided by class priors The pseudo label As the first probability map The supervision signal calculates the cross entropy CE loss and Dice loss After adding them together, we get the consistency loss As shown in formula (9); express The number of pixels, express The number of pixels, express and The number of pixels with the same segmentation results; 3.9 Backpropagating Consistency Loss Using the Optimizer Update student model parameters; 3.10 If t is not divisible by β, let t = t + 1 and go to 3.4; if t is an integer multiple of β, go to 3.11; 3.11 Using the validation set D val Verify the segmentation performance of the passive domain adaptive medical image segmentation system and save the network weight parameters when the segmentation performance no longer improves. The method is: 3.11.1 Student model image segmentation module reads the verification set D val , including the validation set D val images, labels and names in ; 3.11.2 Let batch number l = 1, which means the lth batch. Each batch contains B′ validation set images, where B′ is a positive integer. The total number of batches is 3.11.3 Take out the B′ validation set images of the lth batch and record these B′ images as tensor form images I val , I val contains B′ H×W×3 images; 3.11.4 The student model image segmentation module receives B′ validation set D val The tensor form image I in val , to I val Perform feature extraction and segmentation to obtain the segmentation results I of the images of the B′ validation set in the lth batch l ; 3.11.5 The segmentation results of the images of the B′ validation set in the lth batch I l and save the corresponding labels and the names of the respective image files mentioned in the second step, wherein the names of the image files are the three-dimensional image name + the slice order; 3.11.6 If l <N l , let l=l+1, go to 3.11.3; if l≥N l , indicating that all the data of the validation set have been segmented, and the segmentation results, corresponding labels, and names of the respective image files of V validation set images are obtained. Go to 3.11.7; 3.11.7 Evaluate the segmentation performance of the student model using the Dice coefficient to determine whether to continue training. The method is as follows: 3.11.7.1 The segmentation results of the V validation set images are combined according to the names of their image files, and the images belonging to the same 3D image are combined in the order of slices, that is, V segmentation results of size H×W are combined into J 3D images, namely X1, X2, …, X j ,…,X J , where J is the number of 3D images in the validation set, J = 4 on the CHAOS dataset, X j Size: H×W×S j ,S j is the number of slices of the jth 3D image in the validation set, ∑ j S j =V, j = 1,…,J; the same operation is performed on the true labels corresponding to the segmentation results of the V validation set images to obtain J three-dimensional image labels Y1, Y2,…, Y j ,…,Y J ; 3.11.7.2 Let j = 1, representing the j-th three-dimensional image in the validation set; 3.11.7.3 Segmentation result X for the jth 3D image in the validation set j and the corresponding true label Y j , calculate the Dice coefficient of all segmentation categories, and get the Dice coefficient of all segmentation categories of the j-th three-dimensional image, that is, Dice j,1 ,Dice j,2 ,Dice j,3 ,Dice j,4 ; 3.11.7.4 If j < J, let j = j + 1 and go to 3.11.7.3; if j ≥ J, it means that the Dice coefficients of J three-dimensional images have been calculated, and the Dice coefficients of all segmentation results in the validation set compared with the ground truth labels are obtained, namely Dice 1,1 ,Dice 1,2 , Dice 1,3 ,Dice 1,4 ,…,Dice j,1 ,Dice j,2 ,Dice j,3 ,Dice j, x ,…,Dice J,1 ,Dice J, x ,Dice J,3 ,Dice J,4 , go to 3.11.7.5; 3.11.7.5 Calculate the average Dice coefficient for the four segmentation classes except the background, namely the spleen, right kidney, left kidney, and liver, for J three-dimensional images, to obtain the average Dice coefficients ω1, ω2, ω3, ω4 for the 4 classes, The first type average Dice coefficient ω1=(Dice 1,1 +…+Dice j,1 …+Dice J,1 ) / J The second type average Dice coefficient ω2=(Dice 1,2 +…+Dice j,2 …+Dice J,2 ) / J The third type average Dice coefficient ω3=(Dice 1,3 +…+Dice j,3 …+Dice J,3 ) / J The fourth type average Dice coefficient ω4 = (Dice 1,4 +…+Dice j,4 …+Dice J,4 ) / J 3.11.7.6 Take the average of ω1, ω2, ω3, and ω4 to get the average Dice coefficient Ω for all images and all segmentation classes in the tth round of validation set t ,Ω t =(ω1+ω2+ω3+ω4) / 4; 3.11.7.7 At this time, t is an integer multiple of β. If t = β, save Ω t and the network weight parameters of the student model with iteration number t, go to 3.3; if t is greater than β, go to 3.11.7.8; 3.11.7.8 If t <T (ax , go to 3.11.7.9; if t ≥ T (ax , save the network weight parameters of the student model when the number of iterations is t, and go to step 4; 3.11.7.9 Save Ω t and the network weight parameter of the student model with the number of iterations t, if Ω t >Ω t-β And (Ω t -Ω t-β )≥threshold η, 0<η<1, indicating that the student model is still being optimized during the training process, and the current number of iterations may not have reached the optimal number, go to 3.3; if Ω t >Ω t-β And (Ω t -Ω t-β )<η, go to step 4; if Ω t ≤Ω t-β , indicating that the training is overfitting, and the training effect becomes worse. Let the number of iterations t = t-β and go to step 4; In the fourth step, load the network weight parameters of the student model at the iteration number t into the student model image segmentation module to obtain the trained domain - adaptive medical image segmentation system based on selective online self - training with the best segmentation performance; In the fifth step, use the trained domain - adaptive medical image segmentation system based on selective online self - training with the best segmentation performance to segment the image to be segmented input by the user, and obtain the image segmentation result. The method is as follows: 5.1 The student model image segmentation module receives an image to be segmented I input by the user user ; 5.2 Using the Student Model Image Segmentation Module for I user Perform segmentation to obtain the final segmentation result, and the image segmentation is completed.

2. The method for adaptive medical image segmentation based on selective online self-training passive domain as claimed in claim 1, characterized in that The strong augmentation processing includes random cropping, rotation, random brightness, random contrast, and random gamma transformation, and the weak augmentation processing includes random cropping, rotation, and scaling.

3. The method for adaptive medical image segmentation based on selective online self-training passive domain as claimed in claim 1, characterized in that The partitioning method of the CHAOS’ dataset in step 2.3 is as follows: The total number of images in the training set is T = 395, and the total number of images in the validation set is V = 97.

4. The method for adaptive medical image segmentation based on selective online self-training passive domain as claimed in claim 1, characterized in that The method for setting the network training configuration in step 3.2 is: the optimizer uses the Adam optimizer, and the initial learning rate is set to 1×10 -4 , weight decay is 5×10 -4 , the maximum number of iterations T (ax =2000, the number of initialization iterations β=100; B=4 and K=30 in step 3.

3.

5. The method for adaptive medical image segmentation based on selective online self-training passive domain as claimed in claim 1, characterized in that The EMA update method in step 3.6.4 is shown in formula (2): in represents the network weight parameter of the teacher model at the tkth iteration, θ t represents the network weight parameter of the student model at the tth iteration, represents the network weight parameter of the teacher model at the tth iteration, μ is the EMA update weight, μ = 0.

99.

6. The method for adaptive medical image segmentation based on selective online self-training passive domain as claimed in claim 1, characterized in that 3.7.9 Filter using confidence threshold The bth probability graph in get The pseudo-labeling method is: 3.7.9.1 Initialize the height h = 1; 3.7.9.2 Initialize the width w = 1; 3.7.9.3 The pixel point with coordinates (h, w) in is classified into the category cls with the highest probability, as shown in formula (8): is a probability map of size C×H×W The category probability vector when the second and third dimension coordinates are (h, w) is of size C, that is, the C probabilities corresponding to classifying the pixel point with coordinates (h, w) into C categories; express The category corresponding to the maximum probability is denoted as cls; 3.7.9.4 If the probability Confirm that the category of the pixel with coordinates (h, w) is cls, go to 3.7.9.5; otherwise, the category of the pixel with coordinates (h, w) is background, that is, 0, go to 3.7.9.5; 3.7.9.5 If w < W, let w = w + 1, and go to 3.7.9.3; if w ≥ W, it means that the segmentation is completed when the height is h, and go to 3.7.9.6; 3.7.9.6 If h < H, let h = h + 1, and go to 3.7.9.2; if h ≥ H, it means the segmentation is completed, and the pseudo-labels are obtained, and end.

7. The method for medical image segmentation based on selective online self-training passive domain adaptation as claimed in claim 1, characterized in that B′ = 4 in step 3.11.2; η = 0.001 in step 3.11.7.

8.

8. The method for adaptive medical image segmentation based on selective online self-training passive domain as claimed in claim 1, characterized in that 3.11.7.3 The segmentation result X of the j-th 3D image in the validation set j and the corresponding true label Y j , the method to calculate the Dice coefficient of all segmentation categories is: 3.11.7.3.1 Initialize the category c = 1; 3.11.7.3.2 Calculate X according to formula (10) j and Y j Dice coefficient for category c j,c : Where X j,c Represents the segmentation result X j The area belonging to category c in Y j,c Represents the true label Y j The area belonging to category c in |X j,c ∩Y j,c | represents the number of pixels that overlap between two regions, |X j,c | represents the segmentation result X j The number of pixels in the region belonging to category c, |Y j,c | represents the true label Y j The number of pixels in the region belonging to category c; 3.11.7.3.3 If c < C - 1, let c = c + 1, and go to 3.11.7.3.2; if c = C - 1, it means that the Dice coefficients of all segmentation categories of the j-th 3D image have been calculated, and the Dice coefficients of all segmentation categories of the j-th 3D image, namely Dice j,1 , Dice j,2 , Dice j,3 , Dice j,4 .

Images