Adaptive medical image segmentation method based on selective online self-training passive domain

Abstract

The present invention discloses a passive domain adaptive medical image segmentation method based on selective online self-training. The technical solution is: construct a passive domain adaptive medical image segmentation system based on selective online self-training, which is composed of a strong enhancement module, a weak enhancement module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module. Construct a training set and a validation set. Use the source domain model and the training set to train the segmentation system to obtain a trained segmentation system. Use the validation set to verify the trained segmentation system, and when the segmentation performance no longer improves, obtain the trained segmentation system with the best performance; use the trained segmentation system with the best performance to segment the image and obtain a segmentation result. The present invention can alleviate the negative optimization problem of online self-training in passive domain adaptation and improve segmentation accuracy.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and medical image processing, and in particular to a passive domain adaptive medical image segmentation method based on online self-training. Background Art

[0002] Image segmentation involves assigning each pixel in an image to a specific category (i.e., a segmentation class). For example, when segmenting an abdominal image, the spleen, right kidney, left kidney, and liver regions within the abdomen must be segmented (where 0 represents the rest of the image background, 1 represents the spleen, 2 represents the right kidney, 3 represents the left kidney, and 4 represents the liver). As a fundamental task in image processing, with the development of deep learning technology, deep learning-based image segmentation has made significant progress in a variety of practical fields, including medical analysis, remote sensing, and industrial applications. However, when there is a domain shift between training and test data, the segmentation performance of deep learning models can significantly degrade. Here, the training data is referred to as the source domain, and the test data is referred to as the target domain. The source and target domains belong to the same task class, but have different data distributions. Domain shift between the source and target domains is particularly common in medical images, as the data often originate from instruments with different specifications or are acquired using different imaging modalities (e.g., CT and MRI). This has prompted research into unsupervised domain adaptation (UDA) methods for medical image segmentation. This method uses both source and target domain data for model training. The source domain represents a domain different from the samples to be segmented and typically has rich labels as supervisory information. The target domain represents the domain in which the samples to be segmented reside and often lacks label information. The goal of unsupervised domain adaptation is to mitigate the effects of domain shift while maintaining model segmentation performance in the target domain. The paper "Cheng Chen, et al. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation [J]. IEEE transactions on medical imaging, 2020, 39 (7): 2494-2505. Cheng Chen et al.'s paper: Unsupervised bidirectional cross-modality adaptive medical image segmentation based on deep collaborative image and feature alignment" (SIFA) introduces an unsupervised domain adaptive medical image segmentation method, which converts the source domain image into an image close to the target domain at the image level, and then fine-tunes the network; adversarial learning is performed at the feature level to map data distributed in different domains to the same latent feature space, thereby improving the segmentation performance of the model in the target domain. However, this method requires simultaneous acquisition of source and target domain data. Due to privacy issues in the medical field, the training data of the source domain is usually not accessible during the domain adaptation stage, making this method not universal.

[0003] Therefore, the urgent need for domain adaptation within privacy and security constraints has made source-free domain adaptation (SFDA) for medical image segmentation a popular but challenging research direction. In SFDA, the source data owner provides the neural network weights of the source model. To protect data privacy, the source data owner does not provide the source data to the adaptation process. The source model's network weights are trained using deep learning methods on labeled source data. Therefore, SFDA methods cannot access source data, but can use the source model's neural network weights and unlabeled target domain data for domain adaptation.

[0004] The paper "Mathilde Bateson, et al. Source-free domain adaptation for imagesegmentation [J]. Medical Image Analysis, 2022, 82: 102617. Mathilde Bateson et al.'s paper: Image segmentation based on passive domain adaptation" (AdaMI) proposes a passive domain adaptive image segmentation method based on entropy minimization and domain-independent prior. By minimizing the entropy loss on the target domain data and integrating the domain-independent prior into the loss function, the segmentation results predicted by the model are more reliable. However, this method requires image-level labels for the target domain data, which increases the burden of data annotation. Because the annotation of medical images often relies on medical experts, especially radiologists or pathologists, even for experienced doctors, it is very time-consuming and laborious to annotate a large number of medical images, which leads to high annotation costs and slow speeds. The paper "Serban Stan, et al. Unsupervised model adaptation for source-free segmentation of medical images [J]. Medical Image Analysis, 2024, 95: 103179. Serban Stan et al.'s paper: Unsupervised model adaptation for passive segmentation of medical images" (SFS) proposes a passive domain adaptive image segmentation method based on source latent features. By approximating the source latent features during adaptation and minimizing the distribution distance metric to create a joint source / target embedding space, the model is adapted to the target domain. However, this method requires the use of a Gaussian mixture model to obtain the source latent features of the source domain data. To protect privacy, the owner of the source domain data is likely not to provide this source latent feature, which limits the versatility of this method. The paper "Yan Wang, et al. Fvp: Fourier visual prompting for source-free unsupervised domain adaptation of medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2023. Yan Wang et al.'s paper: Fvp: Passive unsupervised domain medical image segmentation based on Fourier visual prompting" (FVP) adopts a passive domain adaptive image segmentation method based on offline self-training.Offline self-training first initializes a teacher model using the source domain segmentation model. The teacher model is then used to predict unlabeled target domain data to obtain pseudo-labels. The generated pseudo-labels are combined with the target domain data, and a student model is trained on this combined data. After the student model is trained, it becomes the teacher model to train the next student model. This process is repeated n times until convergence is achieved. However, offline self-training cannot update the teacher model in a timely manner because it only updates the student model after training the student model. This prevents the teacher model from co-evolving with the student model, thus limiting performance improvements.

[0005] Current online self-training methods overcome this limitation of offline self-training methods. Unlike offline self-training, online self-training methods do not update the teacher model until after the student model is trained. Instead, the teacher model parameters and the pseudo-labels predicted by the teacher model are updated with each parameter update during the training process. However, this update assumes that each update to the student model during the pseudo-label learning process will be in the correct direction, and thus the teacher model will also be updated in the correct direction. However, in reality, student model updates may not always be correct. Rapid updates to the teacher model can disrupt the student model's learning direction, leading to negative optimization, where the student model's performance deteriorates with increasing training cycles. Furthermore, during the pseudo-label screening process, if the pseudo-labels predicted by the teacher model are of poor quality and cannot provide accurate supervision to the student model, this can also lead to negative optimization. In summary, the negative optimization problem is primarily influenced by two processes: the pseudo-label learning process and the pseudo-label screening process.

[0006] Therefore, how to improve the passive domain adaptive medical image segmentation performance remains a technical issue of great concern to those skilled in the art. Summary of the Invention

[0007] The technical problem to be solved by this invention is to address the problems of existing passive domain adaptive medical image segmentation methods, which require additional labels or source domain information, have limited performance improvements when based on offline self-training, suffer from negative optimization when based on online self-training, and have room for improvement in image segmentation performance. This invention proposes a passive domain adaptive medical image segmentation method based on selective online self-training. This method introduces a selective strategy for the pseudo-label learning and pseudo-label screening processes, utilizes entropy to evaluate the model, and protects data privacy without requiring additional labels or source domain information, alleviating the negative optimization problem of online self-training and improving image segmentation performance.

[0008] To solve the above technical problems, the technical solution of the present invention is to construct a passive domain adaptive medical image segmentation system based on selective online self-training. The system consists of a strong enhancement module, a weak enhancement module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module. The target domain dataset required for the passive domain adaptive medical image segmentation system is prepared and constructed, and the target domain dataset is divided into a training set and a validation set. The student model image segmentation module and the teacher model image segmentation module in the passive domain adaptive medical image segmentation system based on selective online self-training are trained using the source domain model and training set provided by the user, thereby obtaining a more robust trained passive domain adaptive medical image segmentation system. The validation set is used to verify the student model image segmentation module after iterative training β times (β=100). When the segmentation performance of the student model no longer improves, the best-performing trained passive domain adaptive medical image segmentation system based on selective online self-training is obtained; finally, the best-performing trained passive domain adaptive medical image segmentation system based on selective online self-training is used to segment the image input by the user to obtain the image segmentation result.

[0009] The present invention comprises the following steps:

[0010] The first step is to build a passive domain adaptive medical image segmentation system based on selective online self-training. Figure 1 As shown in the figure, the passive domain adaptive medical image segmentation system consists of a strong enhancement module, a weak enhancement module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module.

[0011] The strong enhancement module is connected to the student model image segmentation module. During training, the strong enhancement module receives the training set and performs strong enhancement processing on the training set images, such as random cropping, rotation, random brightness, random contrast, and random gamma transformation, to obtain the strongly enhanced images of the training set. The strong enhancement images of the training set are then sent to the student model image segmentation module.

[0012] The weak enhancement module is connected to the teacher model image segmentation module. During training, the weak enhancement module receives the training set and performs weak enhancement processing on the training set images, such as random cropping, rotation, and scaling, to obtain the weakly enhanced images of the training set. The weakly enhanced images of the training set are then sent to the teacher model image segmentation module.

[0013] The student model image segmentation module is connected to the strong enhancement module, the entropy-guided selective update module and the consistency loss function calculation module. During training, the student model image segmentation module receives the strongly enhanced images of the training set from the strong enhancement module, performs feature extraction and segmentation on the strongly enhanced images of the training set, obtains a first segmentation probability map, and sends the first segmentation probability map to the consistency loss function calculation module. At the same time, the entropy value is calculated based on the first segmentation probability map, and the entropy value is sent to the entropy-guided selective update module. During verification, the student model image segmentation module receives the verification set, performs feature extraction and segmentation on the verification set images, obtains the verification results, and determines whether to continue training based on the verification results. When segmenting the image input by the user, the student model image segmentation module receives the image to be segmented input by the user, performs feature extraction and segmentation on the image to be segmented, and obtains the segmentation result of the image to be segmented. The student model image segmentation module consists of a segmentation model (let it be the student model), whose initial parameters adopt the parameters of the source domain segmentation model.The student model uses the same neural network structure DeepLabv2 architecture as the source domain model (see the literature "Liang-Chieh Chen, et al. Deeplab: Semantic images segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(4): 834-848." Liang-Chieh Chen et al.'s paper: DeepLab: Semantic image segmentation based on deep convolutional networks, atrous convolution and fully connected CRF; DeepLabv2 architecture is the same author's previous DeepLabv1 architecture (see the literature "Liang-Chieh Chen, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[J]. arXiv preprint arXiv:1412.7062, 2014." Liang-Chieh Chen et al.'s paper: "Semantic Image Segmentation Using Deep Convolutional Networks and Fully Connected CRFs" is an improved version of DeepLabv2. DeepLabv2 uses dilated convolutions as a powerful tool for dense prediction tasks. Dilated convolutions can explicitly control the resolution of feature responses calculated within the deep convolutional neural network (DCNN), effectively expanding the receptive field while obtaining more contextual information without increasing the number of parameters and computational complexity. At the same time, the atrous spatial convolutional pooling pyramid (ASPP) is proposed. ASPP uses dilated convolution layers of multiple sampling rates in parallel for prediction, capturing object and image context information at multiple scales to obtain more robust segmentation results.DeepLabv2's feature extractor uses the ResNet network (see the paper "Kaiming He, et al. Deep Residual Learning for Image Recognition [J]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778" by Kaiming He et al.: The Residual Network ResNet described in the network architecture of Deep Residual Learning for Image Recognition). The ResNet network is constructed by stacking a series of residual modules (ResidualBlocks). The residual module consists of convolutional layers and normalization layers. Depending on the depth of the network, variants of the ResNet network (such as ResNet50, ResNet101, and other networks) are often used in specific applications. The ResNet network has small parameters, short training time, and good feature extraction capabilities.

[0014] The entropy-guided selective update module is connected to the student model image segmentation module and the teacher model image segmentation module. Its function is to determine whether to update the parameters of the teacher model image segmentation module. During training, the entropy-guided selective update module receives entropy values ​​from the student model image segmentation module and saves the entropy values ​​of each iteration. Based on the changes in entropy values, it determines whether to update the teacher model image segmentation module. If the entropy value decreases, it determines that the teacher model needs to be updated, and sends the student model parameters to the teacher model image segmentation module.

[0015] The teacher model image segmentation module is connected to the weak enhancement module, the entropy-guided selective update module and the class prior-guided pseudo-label screening module. During training, the teacher model image segmentation module receives the weakly enhanced image data of the training set from the weak enhancement module, performs feature extraction and segmentation, obtains the second segmentation probability map, and sends the second segmentation probability map to the class prior-guided pseudo-label screening module. The teacher model image segmentation module is also composed of a segmentation model (let it be the teacher model). The teacher model also uses the same neural network structure DeepLabv2 architecture as the source domain model, and its initial parameters also use the parameters of the source domain segmentation model. When the student model parameters are received from the entropy-guided selective update module, the exponential moving average method EMA (Exponential Moving Average) is used to update the parameters.

[0016] The class prior-guided pseudo-label screening module is connected to the teacher model image segmentation module and the consistency loss function calculation module. During training, the class prior-guided pseudo-label screening module receives the second segmentation probability map from the teacher model image segmentation module and assigns confidence thresholds to each segmentation class based on the second segmentation probability map. Specifically, confidence thresholds are assigned to the five segmentation classes in the abdominal image: background, spleen, right kidney, left kidney, and liver. The pseudo-label is then generated by filtering the segmentation class of each pixel in the second segmentation probability map based on the confidence thresholds and sent to the consistency loss function calculation module.

[0017] The consistency loss function calculation module is connected to the student model image segmentation module and the class prior-guided pseudo-label screening module. During training, the consistency loss function calculation module receives the first segmentation probability map from the student model image segmentation module and the pseudo-label from the class prior-guided pseudo-label screening module. It compares the first segmentation probability map and the pseudo-label for consistency, and uses the difference between the first segmentation probability map and the pseudo-label as the loss function to backpropagate and update the parameters of the student model image segmentation module.

[0018] In the passive domain adaptive medical image segmentation system based on selective online self-training, the strong enhancement module, weak enhancement module, teacher model image segmentation module, entropy-guided selective update module, class prior-guided pseudo-label screening module, and consistency loss function calculation module are all auxiliary student model image segmentation module training. They only work during training and do not participate in the segmentation of the image to be segmented input by the user. Only the trained student model image segmentation module completes the segmentation of the image to be segmented input by the user.

[0019] The second step is to construct the training set and validation set by:

[0020] 2.1 Collect abdominal medical images as the target domain dataset for passive domain adaptive medical image segmentation by:

[0021] The CHAOS dataset, a 3D abdominal MRI medical image dataset (see "A. Emre Kavur, et al. CHAOS challenge-combined (CT-MR) healthy abdominal organ segmentation [J]. Medical Image Analysis, 2021, 69:101950," by A. Emre Kavur et al.), was used as the target domain dataset for passive domain adaptive medical image segmentation. The dataset includes abdominal images and their corresponding true labels. The labels are 1, 2, 3, and 4 for the spleen, right kidney, left kidney, and liver, respectively, and 0 for other background regions. CHAOS contains a total of 20 3D abdominal images.

[0022] 2.2 De-identify the passive domain adaptive medical image segmentation target domain dataset and strip the subject's privacy, medical history and other personal information from the image. Remove the background slices from each three-dimensional abdominal image, retain only the slices containing the segmented object, and then perform two-dimensional slicing on each three-dimensional abdominal image and its three-dimensional label to obtain a series of two-dimensional images and corresponding two-dimensional labels. Group three consecutive two-dimensional slices into a three-channel image in the slicing order, and the label corresponding to this three-channel image is the two-dimensional label of the second two-dimensional slice. Name the processed three-channel image and the corresponding two-dimensional label according to the three-dimensional image name plus the slicing order. After the above processing, the passive domain adaptive medical image segmentation target domain dataset has a total of 492 three-channel images and corresponding two-dimensional labels, which are called the CHAOS' dataset;

[0023] 2.3 According to the dataset division method commonly used by researchers, the CHAOS dataset is divided into training set D train , validation set D val The CHAOS' dataset is divided into the following ways: the total number of images in the training set is T = 395, and the total number of images in the validation set is V = 97;

[0024] In the third step, the passive domain adaptive medical image segmentation system based on selective online self-training constructed in the first step is trained using the gradient backpropagation method. The segmentation accuracy of the passive domain adaptive medical image segmentation system is verified using the validation set after each β iterations. When the segmentation accuracy no longer improves, the network weight parameters of the optimal student model image segmentation module are obtained. The method is:

[0025] 3.1 Load the model network parameters of the source domain model provided by the user (generally, the user will use deep learning methods to train the source domain model on labeled source domain data (abdominal CT images)) into the teacher model in the teacher model image segmentation module and the student model in the student model image segmentation module;

[0026] 3.2 Set the network training configuration, use the Adam optimizer, and set the initial learning rate to 1×10 (4 , weight decay is 5×10 (4 , the maximum number of iterations T ma+ =2000, initialization iteration number β=100;

[0027] 3.3 Let the number of iterations t = 1, and train one batch of images in each iteration, with a total of B = 4 images in each batch; initialize the maximum update interval K = 30;

[0028] 3.4 Strong enhancement module and weak enhancement module are simultaneously trained from the training set D train Read the B images of the tth batch in the image, recorded as tensor form image I train , strong enhancement module for I train Perform strong enhancement (i.e., train Perform random cropping, rotation, random brightness, random contrast, random gamma transformation) to obtain the strongly enhanced tensor QI train , QI train Send to the student model image segmentation module; weak enhancement module to I train Perform weak enhancement (i.e., train Perform random cropping, rotation, and scaling) to obtain the weakly enhanced tensor RI train , change RI train Sent to the teacher model image segmentation module. train Contains B H×W×3 images. Where H represents the height of the input image, W represents the width of the input image, and "3" represents the three slices contained in each image; B is the number of images in a batch;

[0029] 3.5 Student model image segmentation module receives QI from strong enhancement module train , execute steps 3.5.1 to 3.5.3; at the same time, the teacher model image segmentation module receives RI from the weak enhancement module train , execute steps 3.5.4 to 3.5.5.

[0030] 3.5.1 Student model image segmentation module uses the first feature extraction and segmentation method to QI train Perform feature extraction and segmentation by:

[0031] 3.5.1.1 Student Model Image Segmentation Module for QI trainPerform feature extraction to obtain feature F1;

[0032] 3.5.1.2 The student model image segmentation module segments feature F1 and obtains the first probability map of round t It is a four-dimensional tensor of size B×C×H×W, where C represents the number of categories to be segmented, C=5, i.e., five segmentation categories: background, spleen, right kidney, left kidney, and liver.

[0033] 3.5.2 Student model image segmentation module will Send to the consistency loss function calculation module;

[0034] 3.5.3 Student model image segmentation module based on the first segmentation probability map Calculate the entropy value of round t The calculation method is shown in formula (1):

[0035]

[0036] Will Send to the entropy-guided selective update module, go to 3.6;

[0037] 3.5.4 The teacher model image segmentation module uses the second feature extraction and segmentation method to RI train Perform feature extraction and segmentation by:

[0038] 3.5.4.1 Teacher Model Image Segmentation Module for RI train Perform feature extraction to obtain feature F2;

[0039] 3.5.4.2 The teacher model image segmentation module segments feature F2 to obtain the second probability map of round t It is also a four-dimensional tensor of size B×C×H×W.

[0040] 3.5.5 Send to the pseudo-label screening module guided by class priors, go to 3.6;

[0041] 3.6 Entropy-guided Selective Update Module Preservation Adopting a selective update strategy, And the entropy value of the image segmentation module segmentation result of the student model when updating the teacher model in the tkth round (k is the number of iterations from the last update) Compare and determine whether to update the parameters of the teacher model based on the comparison results. The method is:

[0042] 3.6.1 If t = 1, there is only one iteration left. Set k = 1 and go to 3.7. If t > 1, the value of k has been obtained in the previous iteration. Go to 3.6.2.

[0043] 3.6.2 Compare with If it indicates that the reliability of the current result is enhanced. At this time, the performance of the student model becomes better, which means that the network weight parameters of the teacher model need to be updated using the network weight parameters of the student model. Send the network weight parameters of the student model to the image segmentation module of the teacher model, and go to 3.6.4. Otherwise, go to 3.6.3;

[0044] 3.6.3 Determine whether k is greater than or equal to the maximum update interval K. If k ≥ K, it means that the network weight parameters of the teacher model need to be updated using the network weight parameters of the student model. Send the network weight parameters of the student model to the image segmentation module of the teacher model, and go to 3.6.4; if k < K, do not send the network weight parameters of the student model to the image segmentation module of the teacher model, let k = k + 1, and directly go to 3.7;

[0045] 3.6.4 The image segmentation module of the teacher model uses the network weight parameters of the student model for exponential moving average EMA update to obtain the image segmentation module of the teacher model with updated network weight parameters. The EMA update method is shown in formula (2):

[0046]

[0047] where represents the network weight parameters of the teacher model at the (t - k)-th iteration, θ t represents the network weight parameters of the student model at the t-th iteration, represents the network weight parameters of the teacher model at the t-th iteration, and μ is the EMA update weight, μ = 0.99.

[0048] 3.6.5 Let k = 1, and go to 3.7.

[0049] 3.7 The pseudo-label screening module guided by class priors receives the second probability map of the t-th round from the image segmentation module of the teacher model Selects pseudo-labels using the confidence threshold and sends the pseudo-labels to the consistency loss function calculation module. The method is as follows:

[0050] 3.7.1 The received from the image segmentation module of the teacher model is a four-dimensional tensor of size B × C × H × W, and each value corresponding to the four-dimensional coordinates is a probability between 0 and 1. Calculate the global confidence threshold τ using

[0051] <�000365>

[0052] where represents The b-th element in the first dimension of , b = 1,…,B; is a three-dimensional tensor of size C×H×W, corresponding to the probability map of the bth image among the B images read in the tth round, Represents the maximum value of all probabilities in this three-dimensional tensor;

[0053] 3.7.2 Let b = 1, representing the bth item in a batch of size B;

[0054] 3.7.3 Initialize category number c = 0;

[0055] 3.7.4 Calculation The local threshold χ of the category number c b,c , as shown in formula (4):

[0056]

[0057] express The cth element in the first dimension is a two-dimensional tensor of size H×W;

[0058] 3.7.5 Set the global confidence threshold τ and the local threshold X b,c Combined, we get The category number is c, and the initial confidence threshold is scaled by the upper bound factor γ to obtain The confidence threshold τ of the category number c b,c , as shown in formula (5):

[0059]

[0060] The upper bound factor γ = 0.8;

[0061] 3.7.6 Calculation The class prior α of the class number c b,c , as shown in formula (6):

[0062]

[0063] Where h and w represent The height and width of h=1,…,H,w=1,…,W, represent The probability value corresponding to the four-dimensional coordinates (b, c, h, w);

[0064] 3.7.7 Class Prior α b,c Incorporate into the confidence threshold τ b,c In, get The final confidence threshold τ′ for class serial number c b,c , as shown in formula (7):

[0065]

[0066] where the hyperparameter σ = 0.001;

[0067] 3.7.8 If c < C - 1, set c = c + 1 and go to 3.7.4; if c = C - 1, it means the probability map for all classes has been obtained The final confidence thresholds for all classes are τ′ b,0 , τ′ b,1 , τ′ b,2 , τ′ b,3 , τ′ b,4 , go to 3.7.9;

[0068] 3.7.9 Filter using the confidence threshold the b-th probability map in to obtain the pseudo labels, the method is:

[0069] 3.7.9.1 Initialize height h = 1;

[0070] 3.7.9.2 Initialize width w = 1;

[0071] 3.7.9.3 Divide the pixel point with coordinates (h, w) in into the class cls with the highest probability, as shown in formula (8):

[0072]

[0073] is the probability map of size C × H × W The class probability vector when the second and third dimensions of the coordinates are (h, w) in, with size C, is the C probabilities corresponding to classifying the pixel point with coordinates (h, w) into C classes; denotes the class corresponding to the maximum probability in, denoted as cls;

[0074] 3.7.9.4 If the probability confirms that the class of the pixel point with coordinates (h, w) is cls, go to 3.7.9.5; otherwise, the class of the pixel point with coordinates (h, w) is the background, which is 0, go to 3.7.9.5;

[0075] 3.7.9.5 If w < W, set w = w + 1 and go to 3.7.9.3; if w ≥ W, it means the segmentation for height h is completed, go to 3.7.9.6;

[0076] 3.7.9.6 If h < H, let h = h + 1, and go to 3.7.9.2; if h ≥ H, it means The segmentation is completed, and the pseudo labels are obtained, and go to 3.7.10;

[0077] 3.7.10 If b < B, let b = b + 1, and go to 3.7.3; if b ≥ B, it means the loop is completed, and the pseudo labels of the images in the t-th round are obtained and go to 3.7.11;

[0078] 3.7.11 The class prior-guided pseudo label screening module sends the pseudo labels to the consistency loss function calculation module;

[0079] 3.8 The consistency loss function calculation module receives the first probability map from the student model image segmentation module and receives the pseudo labels from the class prior-guided pseudo label screening module uses the pseudo labels as the supervision signal of the first probability map to calculate the cross entropy (CE) loss and the Dice loss and add them together to obtain the consistency loss as shown in formula (9);

[0080]

[0081] represents the number of pixel points, represents the number of pixel points, represents and the number of pixel points with the same segmentation result;

[0082] 3.9 Use the Adam optimizer to backpropagate the consistency loss to update the parameters of the student model;

[0083] 3.10 If t cannot be divided evenly by β, let t = t + 1, and go to 3.4; if t is an integer multiple of β, go to 3.11; <00​​​​​​​​​​

[0086] 3.11.2 Let batch number l = 1, which represents the lth batch. Each batch contains B′ validation set images, B′ = 4, and the total number of batches is

[0087] 3.11.3 Take out the B′ validation set images from the lth batch and record these B′ images as tensor form images I val , I val contains B′ H×W×3 images;

[0088] 3.11.4 The student model image segmentation module receives B′ validation set D val The tensor form image I in val , to I val Perform feature extraction and segmentation to obtain the segmentation results I of the B′ validation set images in the lth batch l ;

[0089] 3.11.5 The segmentation results of the B′ validation set images in the lth batch I l Save the corresponding labels and the names of the respective image files (3D image name + slice order) mentioned in the second step;

[0090] 3.11.6 If l <N l , let l=l+1, go to 3.11.3; if l≥N l , indicating that all validation set data has been segmented, and the segmentation results, corresponding labels, and names of the respective image files for V validation set images have been obtained. Go to 3.11.7.

[0091] 3.11.7 Use the Dice coefficient to evaluate the segmentation performance of the student model and determine whether to continue training. The method is:

[0092] 3.11.7.1 The segmentation results of the V validation set images are grouped together in slice order according to the names of their image files. That is, V segmentation results of size H×W are combined into J 3D images, namely X1, X2, …, X j ,…,X J , where J is the number of 3D images in the validation set, J = 4 on the CHAOS dataset, X j Size: H×W×S j ,S j is the number of slices of the jth 3D image in the validation set, ∑ j S j =V, j = 1,…, J. The same operation is performed on the true labels corresponding to the segmentation results of V validation set images to obtain J three-dimensional image labels Y1, Y2,…, Y j ,…,YJ ;

[0093] 3.11.7.2 Let j = 1, representing the j-th 3D image in the validation set;

[0094] 3.11.7.3 For the segmentation result X of the j-th 3D image in the validation set j and the corresponding ground truth label Y j , calculate the Dice coefficients for all segmentation classes as follows:

[0095] 3.11.7.3.1 Initialize class c = 1;

[0096] 3.11.7.3.2 Calculate the Dice coefficient Dice for class c of X j and Y j using formula (10): j,c :

[0097]

[0098] where X j,c represents the region of the segmentation result X j that belongs to class c, Y j,c represents the region of the ground truth label U j that belongs to class c, |X j,c ∩Y j,c | represents the number of pixel points where the two regions overlap, |X j,c | represents the number of pixel points in the region of the segmentation result X j that belongs to class c, |Y j,c | represents the number of pixel points in the region of the ground truth label Y j that belongs to class c;

[0099] 3.11.7.3.3 If c < C - 1, let c = c + 1 and go to 3.11.7.3.2; if c = C - 1, it means the Dice coefficients for all segmentation classes of the j-th 3D image have been calculated, obtaining the Dice coefficients for all segmentation classes of the j-th 3D image, i.e., Dice j,1 , Dice j,2 , Dice j,3 , Dice j,4 , and go to 3.11.7.4;

[0100] 3.11.7.4 If j < J, let j = j + 1 and go to 3.11.7.3; if j ≥ J, it means the Dice coefficients for J 3D images have been calculated, obtaining the Dice coefficients for all segmentation results in the validation set compared with the ground truth labels, i.e., Dice 1,1 , Dice 1,2 , Dice 1,3 , Dice1,4 ,…,Dice j,1 ,Dice j,2 ,Dice j,3 ,Dice j,4 ,…,Dice J,1 ,Dice J,2 ,Dice J,3 ,Dice J,4 , go to 3.11.7.5;

[0101] 3.11.7.5 Calculate the average Dice coefficients of the four segmentation classes (spleen, right kidney, left kidney, and liver) excluding the background in J 3D images, and obtain the average Dice coefficients ω1, ω2, ω3, and ω4 for the four classes.

[0102] The first type average Dice coefficient ω1=(Dice 1,1 +…+Dice j,1 …+Dice J,1 ) / J

[0103] The second type of average Dice coefficient ω2=(Dice 1,2 +…+Dice j,2 …+Dice J,2 ) / J

[0104] The third type average Dice coefficient ω3=(Dice 1,3 +…+Dice j,3 …+Dice J,3 ) / J

[0105] The fourth type average Dice coefficient ω4=(Dice 1,4 +…+Dice j,4 …+Dice J,4 ) / J

[0106] 3.11.7.6 Take the average of ω1, ω2, ω3, and ω4 to get the average Dice coefficient Ω for all images and all segmentation classes in the validation set in round t. t ,Ω t =(ω1+ω2+ω3+ω4) / 4;

[0107] 3.11.7.7 In this case, t is an integer multiple of β (see 3.10). If t = β, only Ω is saved. t and the network weight parameters of the student model with iteration number t, go to 3.3; if t is greater than β, go to 3.11.7.8;

[0108] 3.11.7.8 If t <T ma+ , go to 3.11.7.9; if t ≥ T ma+, save the network weight parameters of the student model when the number of iterations is t, and go to step 4;

[0109] 3.11.7.9 Saving the Average Dice Coefficient Ω t and the network weight parameters of the student model with the number of iterations t, and compare the average Dice coefficient Ω in the tth round t and the average Dice coefficient Ω in the t-β round t(β , if Ω t >Ω t(β And (Ω t -Ω t(β )≥threshold η, 0<η<1, preferably η=0.001, indicating that the student model is still being optimized during the training process, and the current number of iterations may not have reached the optimal number, go to 3.3; if Ω t >Ω t(β And (Ω t -Ω t(β )<η, there is no need to continue training the student model image segmentation module, and go to step 4; if Ω t ≤Ω t(β , indicating that the training is overfitting, and the effect of further training becomes worse. At this time, the model effect is best when the number of iterations is t-β. Since the number of iterations loaded in the fourth step is the network weight parameter of the student model at t, let the number of iterations t = t-β and go to the fourth step.

[0110] The fourth step is to load the network weight parameters of the student model when the number of iterations is t into the student model image segmentation module to obtain the passive domain adaptive medical image segmentation system based on selective online self-training with the best segmentation performance after training.

[0111] In the fifth step, the image to be segmented input by the user is segmented using the passive domain adaptive medical image segmentation system based on selective online self-training with the best segmentation performance, and the image segmentation result is obtained by:

[0112] 5.1 The student model image segmentation module receives an image to be segmented I input by the user user ;

[0113] 5.2 Using the Student Model Image Segmentation Module to I user Perform segmentation to obtain the final segmentation result, and the image segmentation is completed.

[0114] The following beneficial effects can be achieved by adopting the present invention:

[0115] This paper proposes a passive domain adaptive medical image segmentation method based on selective online self-training. This method uses selective online self-training to combat model negative optimization, improve model performance, and achieve adaptive segmentation of medical images. The present invention can achieve the following effects:

[0116] 1. The present invention constructs a passive domain adaptive medical image segmentation system that integrates a strong enhancement module, a weak enhancement module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module. The system improves the performance of passive domain adaptive medical image segmentation through selective online self-training. Experiments were conducted using a training set of a CHAOS-partitioned abdominal MRI three-dimensional image dataset. The present invention was compared with the three passive domain adaptive medical image segmentation methods described in the background art: FVP, AdaMI, and SFS. The segmentation Dice coefficient of the present invention was significantly improved compared to the above three methods. Furthermore, the present invention was compared with several classic unsupervised domain adaptive methods: SynSeg-Net, AdaOutput, CycleGAN, CyCADA, and SIFA. These methods use source domain data and are unable to protect the privacy of the source domain data. Although the present invention does not use source domain data, its segmentation performance is comparable to, and in some cases even surpasses, those of these methods that do use source domain data.

[0117] 2. This invention uses selective online self-training. It introduces an entropy-guided selective update module to selectively update the parameters of the teacher model (see 3.6 for details). This module selectively updates the teacher model based on the output entropy of the student model, effectively alleviating the negative optimization problem that occurs during training. It also proposes a pseudo-label screening module guided by class priors to optimize pseudo-labels (see 3.7 for details) to alleviate the class imbalance problem in pseudo-labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] Figure 1 This is the logical structure diagram of the passive domain adaptive medical image segmentation system constructed in the first step of the present invention.

[0119] Figure 2 It is the overall flow chart of the present invention.

[0120] Figure 3 This figure shows a qualitative comparison of the segmentation results of the present invention with those of AdaMI, a passive domain-adaptive image segmentation method based on entropy minimization and domain-independent priors, and the ground truth labels. Both rows show the segmentation results when the target domain is an MRI image, with different colors representing different organs. Since source code for two other passive domain-adaptive medical image segmentation methods in the background art, FVP and SFS, is not provided, a qualitative comparison was not performed. DETAILED DESCRIPTION

[0121] The following describes specific examples of the present invention in conjunction with the accompanying drawings. Figure 2 As shown, the present invention includes the following steps:

[0122] The first step is to build a passive domain adaptive medical image segmentation system based on selective online self-training. Figure 1 As shown in the figure, the passive domain adaptive medical image segmentation system consists of a strong enhancement module, a weak enhancement module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module.

[0123] The strong enhancement module is connected to the student model image segmentation module. During training, the strong enhancement module receives the training set and performs strong enhancement processing on the training set images, such as random cropping, rotation, random brightness, random contrast, and random gamma transformation, to obtain the strongly enhanced images of the training set. The strong enhancement images of the training set are then sent to the student model image segmentation module.

[0124] The weak enhancement module is connected to the teacher model image segmentation module. During training, the weak enhancement module receives the training set and performs weak enhancement processing on the training set images, such as random cropping, rotation, and scaling, to obtain the weakly enhanced images of the training set. The weakly enhanced images of the training set are then sent to the teacher model image segmentation module.

[0125] The student model image segmentation module is connected to the strong enhancement module, the entropy-guided selective update module and the consistency loss function calculation module. During training, the student model image segmentation module receives the strongly enhanced images of the training set from the strong enhancement module, performs feature extraction and segmentation on the strongly enhanced images of the training set, obtains a first segmentation probability map, and sends the first segmentation probability map to the consistency loss function calculation module. At the same time, the entropy value is calculated based on the first segmentation probability map, and the entropy value is sent to the entropy-guided selective update module. During verification, the student model image segmentation module receives the verification set, performs feature extraction and segmentation on the verification set images, obtains the verification results, and determines whether to continue training based on the verification results. When segmenting the image input by the user, the student model image segmentation module receives the image to be segmented input by the user, performs feature extraction and segmentation on the image to be segmented, and obtains the segmentation result of the image to be segmented. The student model image segmentation module consists of a segmentation model (let it be the student model), whose initial parameters adopt the parameters of the source domain segmentation model.The student model uses the same neural network structure DeepLabv2 architecture as the source domain model (see the literature "Liang-Chieh Chen, et al. Deeplab: Semantic images segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(4): 834-848." Liang-Chieh Chen et al.'s paper: DeepLab: Semantic image segmentation based on deep convolutional networks, atrous convolution and fully connected CRF; this is the same author's previous DeepLabv1 architecture (see the literature "Liang-Chieh Chen, et al. Semantic image segmentation with deepconvolutional nets and fully connected CRFs[J]. arXiv preprint arXiv:1412.7062, 2014." Liang-Chieh Chen et al.'s paper: "Semantic Image Segmentation Using Deep Convolutional Networks and Fully Connected CRFs" is an improved version of DeepLabv2. DeepLabv2 uses dilated convolutions as a powerful tool for dense prediction tasks. Dilated convolutions can explicitly control the resolution of feature responses calculated within the deep convolutional neural network (DCNN), effectively expanding the receptive field while obtaining more contextual information without increasing the number of parameters and computational complexity. At the same time, the atrous spatial convolutional pooling pyramid (ASPP) is proposed. ASPP uses dilated convolution layers of multiple sampling rates in parallel for prediction, capturing object and image context information at multiple scales to obtain more robust segmentation results.DeepLabv2's feature extractor uses the ResNet network (see the paper "Kaiming He, et al. Deep Residual Learning for Image Recognition [J]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778" by Kaiming He et al.: The Residual Network ResNet described in the network architecture of Deep Residual Learning for Image Recognition). The ResNet network is constructed by stacking a series of residual modules (ResidualBlocks). The residual module consists of convolutional layers and normalization layers. Depending on the depth of the network, variants of the ResNet network (such as ResNet50, ResNet101, and other networks) are often used in specific applications. The ResNet network has small parameters, short training time, and good feature extraction capabilities.

[0126] The entropy-guided selective update module is connected to the student model image segmentation module and the teacher model image segmentation module. Its function is to determine whether to update the parameters of the teacher model image segmentation module. During training, the entropy-guided selective update module receives entropy values ​​from the student model image segmentation module and saves the entropy values ​​of each iteration. Based on the changes in entropy values, it determines whether to update the teacher model image segmentation module. If the entropy value decreases, it determines that the teacher model needs to be updated, and sends the student model parameters to the teacher model image segmentation module.

[0127] The teacher model image segmentation module is connected to the weak enhancement module, the entropy-guided selective update module and the class prior-guided pseudo-label screening module. During training, the teacher model image segmentation module receives the weakly enhanced image data of the training set from the weak enhancement module, performs feature extraction and segmentation, obtains the second segmentation probability map, and sends the second segmentation probability map to the class prior-guided pseudo-label screening module. The teacher model image segmentation module is also composed of a segmentation model (let it be the teacher model). The teacher model also uses the same neural network structure DeepLabv2 architecture as the source domain model, and its initial parameters also use the parameters of the source domain segmentation model. When the student model parameters are received from the entropy-guided selective update module, the exponential moving average method EMA (Exponential Moving Average) is used to update the parameters.

[0128] The class prior-guided pseudo-label screening module is connected to the teacher model image segmentation module and the consistency loss function calculation module. During training, the class prior-guided pseudo-label screening module receives the second segmentation probability map from the teacher model image segmentation module and assigns confidence thresholds to each segmentation class based on the second segmentation probability map. Specifically, confidence thresholds are assigned to the five segmentation classes in the abdominal image: background, spleen, right kidney, left kidney, and liver. The pseudo-label is then generated by filtering the segmentation class of each pixel in the second segmentation probability map based on the confidence thresholds and sent to the consistency loss function calculation module.

[0129] The consistency loss function calculation module is connected to the student model image segmentation module and the class prior-guided pseudo-label screening module. During training, the consistency loss function calculation module receives the first segmentation probability map from the student model image segmentation module and the pseudo-label from the class prior-guided pseudo-label screening module. It compares the first segmentation probability map and the pseudo-label for consistency, and uses the difference between the first segmentation probability map and the pseudo-label as the loss function to backpropagate and update the parameters of the student model image segmentation module.

[0130] In the passive domain adaptive medical image segmentation system based on selective online self-training, the strong enhancement module, weak enhancement module, teacher model image segmentation module, entropy-guided selective update module, class prior-guided pseudo-label screening module, and consistency loss function calculation module are all auxiliary student model image segmentation module training. They only work during training and do not participate in the segmentation of the image to be segmented input by the user. Only the trained student model image segmentation module completes the segmentation of the image to be segmented input by the user.

[0131] The second step is to construct the training set and validation set by:

[0132] 2.1 Collect abdominal medical images as the target domain dataset for passive domain adaptive medical image segmentation by:

[0133] The CHAOS dataset, a 3D abdominal MRI medical image dataset (see "A. Emre Kavur, et al. CHAOS challenge-combined (CT-MR) healthy abdominal organ segmentation [J]. Medical Image Analysis, 2021, 69:101950," by A. Emre Kavur et al.), was used as the target domain dataset for passive domain adaptive medical image segmentation. The dataset includes abdominal images and their corresponding true labels. The labels are 1, 2, 3, and 4 for the spleen, right kidney, left kidney, and liver, respectively, and 0 for other background regions. CHAOS contains a total of 20 3D abdominal images.

[0134] 2.2 De-identify the passive domain adaptive medical image segmentation target domain dataset and strip the subject's privacy, medical history and other personal information from the image. Remove the background slices from each three-dimensional abdominal image, retain only the slices containing the segmented object, and then perform two-dimensional slicing on each three-dimensional abdominal image and its three-dimensional label to obtain a series of two-dimensional images and corresponding two-dimensional labels. Group three consecutive two-dimensional slices into a three-channel image in the slicing order, and the label corresponding to this three-channel image is the two-dimensional label of the second two-dimensional slice. Name the processed three-channel image and the corresponding two-dimensional label according to the three-dimensional image name plus the slicing order. After the above processing, the passive domain adaptive medical image segmentation target domain dataset has a total of 492 three-channel images and corresponding two-dimensional labels, which are called the CHAOS' dataset;

[0135] 2.3 According to the dataset division method commonly used by researchers, the CHAOS dataset is divided into training set D train , validation set D val The CHAOS' dataset is divided into the following ways: the total number of images in the training set is T = 395, and the total number of images in the validation set is V = 97;

[0136] In the third step, the passive domain adaptive medical image segmentation system based on selective online self-training constructed in the first step is trained using the gradient backpropagation method. The segmentation accuracy of the passive domain adaptive medical image segmentation system is verified using the validation set after each β iterations. When the segmentation accuracy no longer improves, the network weight parameters of the optimal student model image segmentation module are obtained. The method is:

[0137] 3.1 Load the model network parameters of the source domain model provided by the user (generally, the user will use deep learning methods to train the source domain model on labeled source domain data (abdominal CT images)) into the teacher model in the teacher model image segmentation module and the student model in the student model image segmentation module;

[0138] 3.2 Set the network training configuration, use the Adam optimizer, and set the initial learning rate to 1×10 (4 , weight decay is 5×10 (4 , the maximum number of iterations T ma+ =2000, initialization iteration number β=100;

[0139] 3.3 Let the number of iterations t = 1, and train one batch of images in each iteration, with a total of B = 4 images in each batch; initialize the maximum update interval K = 30;

[0140] 3.4 Strong enhancement module and weak enhancement module are simultaneously trained from the training set D train Read the B images of the tth batch in the image, recorded as tensor form image I train , strong enhancement module for I train Perform strong enhancement (i.e., train Perform random cropping, rotation, random brightness, random contrast, random gamma transformation) to obtain the strongly enhanced tensor QI train , QI train Send to the student model image segmentation module; weak enhancement module to I train Perform weak enhancement (i.e., train Randomly crop and rotate) to obtain the weakly enhanced tensor RI train , change RI train Sent to the teacher model image segmentation module. train Contains B H×W×3 images. Where H represents the height of the input image, W represents the width of the input image, and "3" represents the three slices contained in each image; B is the number of images in a batch;

[0141] 3.5 Student model image segmentation module receives QI from strong enhancement module train , execute steps 3.5.1 to 3.5.3; at the same time, the teacher model image segmentation module receives RI from the weak enhancement module train , execute steps 3.5.4 to 3.5.5.

[0142] 3.5.1 Student model image segmentation module uses the first feature extraction and segmentation method to QI train Perform feature extraction and segmentation by:

[0143] 3.5.1.1 Student Model Image Segmentation Module for Q trainPerform feature extraction to obtain feature F1;

[0144] 3.5.1.2 The student model image segmentation module segments feature F1 and obtains the first probability map of round t It is a four-dimensional tensor of size B×C×H×W, where C represents the number of categories to be segmented, C=5, i.e., five segmentation categories: background, spleen, right kidney, left kidney, and liver.

[0145] 3.5.2 Student model image segmentation module will Send to the consistency loss function calculation module;

[0146] 3.5.3 Student model image segmentation module based on the first segmentation probability map Calculate the entropy value of round t The calculation method is shown in formula (1):

[0147]

[0148] Will Send to the entropy-guided selective update module, go to 3.6;

[0149] 3.5.4 The teacher model image segmentation module uses the second feature extraction and segmentation method to RI train Perform feature extraction and segmentation by:

[0150] 3.5.4.1 Teacher Model Image Segmentation Module for RI train Perform feature extraction to obtain feature F2;

[0151] 3.5.4.2 The teacher model image segmentation module segments feature F2 to obtain the second probability map of round t It is also a four-dimensional tensor of size B×C×H×W.

[0152] 3.5.5 Send to the pseudo-label screening module guided by class priors, go to 3.6;

[0153] 3.6 Entropy-guided Selective Update Module Preservation Adopting a selective update strategy, And the entropy value of the image segmentation module segmentation result of the student model when updating the teacher model in the tkth round (k is the number of iterations from the last update) Compare and determine whether to update the parameters of the teacher model based on the comparison results. The method is:

[0154] 3.6.1 If t = 1, there is only one iteration left. Set k = 1 and go to 3.7. If t > 1, the value of k has been obtained in the previous iteration. Go to 3.6.2.

[0155] 3.6.2 Compare with If it indicates that the reliability of the current result is enhanced. At this time, the performance of the student model becomes better, which means that the network weight parameters of the student model need to be used to update the network weight parameters of the teacher model. Send the network weight parameters of the student model to the image segmentation module of the teacher model, and go to 3.6.4.

[0156] Otherwise, go to 3.6.3;

[0157] 3.6.3 Judge whether k is greater than or equal to the maximum update interval K. If k≥K, it means that the network weight parameters of the student model need to be used to update the network weight parameters of the teacher model. Send the network weight parameters of the student model to the image segmentation module of the teacher model, and go to 3.6.4; if k<K, do not send the network weight parameters of the student model to the image segmentation module of the teacher model, let k = k + 1, and directly go to 3.7;

[0158] 3.6.4 The image segmentation module of the teacher model uses the network weight parameters of the student model for exponential moving average EMA update to obtain the image segmentation module of the teacher model with updated network weight parameters. The EMA update method is shown in formula (2):

[0159]

[0160] where represents the network weight parameters of the teacher model at the (t - k)-th iteration, θ t represents the network weight parameters of the student model at the t-th iteration, represents the network weight parameters of the teacher model at the t-th iteration, and μ is the EMA update weight, μ = 0.99.

[0161] 3.6.5 Let k = 1, and go to 3.7.

[0162] 3.7 The pseudo-label screening module guided by class priors receives the second probability map of the t-th round from the image segmentation module of the teacher model Select pseudo-labels using the confidence threshold and send the pseudo-labels to the consistency loss function calculation module. The method is:

[0163] 3.7.1 The received from the image segmentation module of the teacher model is a four-dimensional tensor of size B×C×H×W, and each value corresponding to the four-dimensional coordinates is a probability between 0 and 1. Use

[0164]

[0165] in express The b-th element in the first dimension of , b = 1,…,B; is a three-dimensional tensor of size C×H×W, corresponding to the probability map of the bth image among the B images read in the tth round, Represents the maximum value of all probabilities in this three-dimensional tensor;

[0166] 3.7.2 Let b = 1, representing the bth item in a batch of size B;

[0167] 3.7.3 Initialize category number c = 0;

[0168] 3.7.4 Calculation The local threshold χ of the category number c b,c , as shown in formula (4):

[0169]

[0170] express The cth element in the first dimension is a two-dimensional tensor of size H×W;

[0171] 3.7.5 Set the global confidence threshold τ and the local threshold χ b,c Combined, we get The category number is c, and the initial confidence threshold is scaled by the upper bound factor γ to obtain The confidence threshold τ of the category number c b,c , as shown in formula (5):

[0172]

[0173] The upper bound factor γ = 0.8;

[0174] 3.7.6 Calculation The class prior α of the class number c b,c , as shown in formula (6):

[0175]

[0176] Where h and w represent The height and width of h=1,…,H,w=1,…,W, represent The probability value corresponding to the four-dimensional coordinates (b, c, h, w);

[0177] 3.7.7 Class Prior α b,c Incorporate into the confidence threshold τ b,c In, get The final confidence threshold τ′ for class serial number c b,c , as shown in formula (7):

[0178]

[0179] where the hyperparameter σ = 0.001;

[0180] 3.7.8 If c < C - 1, let c = c + 1, and go to 3.7.4; if c = C - 1, it means that the probability map of all classes has been obtained, and the final confidence threshold τ′ b,0 , τ′ b,1 , τ′ b,2 , τ′ b,3 , τ′ b,4 , go to 3.7.9;

[0181] 3.7.9 Filter using the confidence threshold the b-th probability map in to obtain the pseudo-label, the method is:

[0182] 3.7.9.1 Initialize height h = 1;<​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​3.7.9.6 If h < H, let h = h + 1, and go to 3.7.9.2; if h ≥ H, it means the segmentation is completed, and the pseudo labels are obtained, then go to 3.7.10;

[0190] 3.7.10 If b < B, let b = b + 1, and go to 3.7.3; if b ≥ B, it means the loop is completed, and the pseudo labels of the images in the t-th round are obtained then go to 3.7.11;

[0191] 3.7.11 The class prior-guided pseudo label screening module sends the pseudo labels to the consistency loss function calculation module;

[0192] 3.8 The consistency loss function calculation module receives the first probability map from the student model image segmentation module and receives the pseudo labels from the class prior-guided pseudo label screening module uses the pseudo labels as the supervision signal of the first probability map to calculate the cross entropy (CE) loss and the Dice loss and add them together to obtain the consistency loss as shown in formula (9);

[0193]

[0194] represents the number of pixel points of represents the number of pixel points of represents and the number of pixel points with the same segmentation result;

[0195] 3.9 Use the Adam optimizer to backpropagate the consistency loss to update the parameters of the student model;

[0196] 3.10 If t cannot be divided evenly by β, let t = t + 1, and go to 3.4; if t is an integer multiple of β, go to 3.11;

[0197] 3.11 Use the validation set D val to verify the segmentation performance of the unsupervised domain adaptation medical image segmentation system. When the segmentation performance no longer improves, save the network weight parameters. The method is:

[0198] 3.11.1 The student model image segmentation module reads the validation set D val , including the images, labels, and names in the validation set D val ;

[0199] 3.11.2 Let batch number l = 1, which represents the lth batch. Each batch contains B′ validation set images, B′ = 4, and the total number of batches is

[0200] 3.11.3 Take out the B′ validation set images from the lth batch and record these B′ images as tensor form images I val , I val contains B′ H×W×3 images;

[0201] 3.11.4 The student model image segmentation module receives B′ validation set D val The tensor form image I in val , to I val Perform feature extraction and segmentation to obtain the segmentation results I of the B′ validation set images in the lth batch l ;

[0202] 3.11.5 The segmentation results of the B′ validation set images in the lth batch I l Save the corresponding labels and the names of the respective image files (3D image name + slice order) mentioned in the second step;

[0203] 3.11.6 If l <N l , let l=l+1, go to 3.11.3; if l≥N l , indicating that all validation set data has been segmented, and the segmentation results, corresponding labels, and names of the respective image files for V validation set images have been obtained. Go to 3.11.7.

[0204] 3.11.7 Use the Dice coefficient to evaluate the segmentation performance of the student model and determine whether to continue training. The method is:

[0205] 3.11.7.1 The segmentation results of the V validation set images are grouped together in slice order according to the names of their image files. That is, V segmentation results of size H×W are combined into J 3D images, namely x1, x2, …, x j ,…,x J , where J is the number of 3D images in the validation set, J = 4 on the CHAOS dataset, X j Size: H×W×S j ,S j is the number of slices of the jth 3D image in the validation set, ∑ j S j =V, j = 1,…, J. The same operation is performed on the true labels corresponding to the segmentation results of V validation set images to obtain J three-dimensional image labels Y1, Y2,…, Y j ,…,YJ ;

[0206] 3.11.7.2 Let j = 1, representing the j-th 3D image in the validation set;

[0207] 3.11.7.3 For the segmentation result X j and the corresponding ground truth label Y j of the j-th 3D image in the validation set, calculate the Dice coefficients for all segmentation classes as follows:

[0208] 3.11.7.3.1 Initialize the class c = 1;

[0209] 3.11.7.3.2 Calculate the Dice coefficient Dice j of X j and Y j,c for class c according to formula (10):

[0210]

[0211] where X j,c represents the region in the segmentation result X j that belongs to class c, Y j,c represents the region in the ground truth label Y j that belongs to class c, |X j,c ∩Y j,c | represents the number of pixel points where the two regions overlap, |X j,c | represents the number of pixel points in the region of the segmentation result X j that belongs to class c, |Y j,c | represents the number of pixel points in the region of the ground truth label Y j that belongs to class c;

[0212] 3.11.7.3.3 If c < C - 1, let c = c + 1 and go to 3.11.7.3.2; if c = C - 1, it means that the Dice coefficients for all segmentation classes of the j-th 3D image have been calculated, and the Dice coefficients for all segmentation classes of the j-th 3D image, namely Dice j,1 , Dice j,2 , Dice j,3 , Dice j,4 are obtained, and go to 3.11.7.4;

[0213] 3.11.7.4 If j < J, let j = j + 1 and go to 3.11.7.3; if j ≥ J, it means that the Dice coefficients for J 3D images have been calculated, and the Dice coefficients of all segmentation results in the validation set compared with the ground truth labels, namely Dice 1,1 , Dice 1,2 , Dice 1,3 , Dice1,4 ,…,Dice j,1 ,Dice j,2 ,Dice j,3 ,Dice j,4 ,…,Dice J,1 ,Dice J,2 ,Dice J,3 ,Dice J,4 , go to 3.11.7.5;

[0214] 3.11.7.5 Calculate the average Dice coefficients of the four segmentation classes (spleen, right kidney, left kidney, and liver) excluding the background in J 3D images, and obtain the average Dice coefficients ω1, ω2, ω3, and ω4 for the four classes.

[0215] The first type average Dice coefficient ω1=(Dice 1,1 +…+Dice j,1 …+Dice J,1 ) / J

[0216] The second type of average Dice coefficient ω2=(Dice 1,2 +…+Dice j,2 …+Dice J,2 ) / J

[0217] The third type average Dice coefficient ω3=(Dice 1,3 +…+Dice j,3 …+Dice J,3 ) / J

[0218] The fourth type average Dice coefficient ω4=(Dice 1,4 +…+Dice j,4 …+Dice J,4 ) / J

[0219] 3.11.7.6 Take the average of ω1, ω2, ω3, and ω4 to get the average Dice coefficient Ω for all images and all segmentation classes in the validation set in round t. t ,Ω t =(ω1+ω2+ω3+ω4) / 4;

[0220] 3.11.7.7 In this case, t is an integer multiple of β (see 3.10). If t = β, only Ω is saved. t and the network weight parameters of the student model with iteration number t, go to 3.3; if t is greater than β, go to 3.11.7.8;

[0221] 3.11.7.8 If t <T ma+ , go to 3.11.7.9; if t ≥ T ma+, save the network weight parameters of the student model when the number of iterations is t, and go to step 4;

[0222] 3.11.7.9 Saving the Average Dice Coefficient Ω t and the network weight parameters of the student model with the number of iterations t, and compare the average Dice coefficient Ω in the tth round t and the average Dice coefficient Ω in the t-β round t(β , if Ω t >Ω t(β And (Ω t -Ω t(β )≥threshold η, 0<η<1, preferably η=0.001, indicating that the student model is still being optimized during the training process, and the current number of iterations may not have reached the optimal number, go to 3.3; if Ω t >Ω t(β And (Ω t -Ω t(β )<η, there is no need to continue training the student model image segmentation module, and go to step 4; if Ω t ≤Ω t(β , indicating that the training is overfitting, and the effect of further training becomes worse. At this time, the model effect is best when the number of iterations is t-β. Since the number of iterations loaded in the fourth step is the network weight parameter of the student model at t, let the number of iterations t = t-β and go to the fourth step.

[0223] The fourth step is to load the network weight parameters of the student model when the number of iterations is t into the student model image segmentation module to obtain the passive domain adaptive medical image segmentation system based on selective online self-training with the best segmentation performance after training.

[0224] In the fifth step, the image to be segmented input by the user is segmented using the passive domain adaptive medical image segmentation system based on selective online self-training with the best segmentation performance, and the image segmentation result is obtained by:

[0225] 5.1 The student model image segmentation module receives an image to be segmented I input by the user user ;

[0226] 5.2 Using the Student Model Image Segmentation Module to I user Perform segmentation to obtain the final segmentation result, and the image segmentation is completed.

[0227] In order to verify the effect of the present invention, the abdominal MRI three-dimensional image dataset CHAOS was selected instead of the user's input image, and the segmentation results of the present invention were qualitatively and quantitatively compared, wherein the quantitative comparison used the segmentation Dice coefficient and ASSD. The experiment was carried out on a server, the experimental environment was Ubuntu20.04 (a version of the Linux system), and an NVIDIA Quadro RTX 6000 GPU with 24GB of memory was used. The validation set of the MRI image dataset CHAOS' was input into the passive domain adaptive image segmentation system of the present invention, the validation set data was segmented, the segmentation Dice coefficient and ASSD were statistically analyzed and compared with other methods to compare and measure the effectiveness of the present invention.

[0228] First, the segmentation results of the present invention are qualitatively compared with those of AdaMI, a passive domain adaptive image segmentation method based on entropy minimization and domain-independent priors in the background art. The other two passive domain adaptive medical image segmentation methods FVP and SFS in the background art do not provide source code, so qualitative comparison cannot be performed. Figure 3 As shown, the first column of images is the MRI abdominal image input by the user, the second column is the segmentation prediction result of the AdaMI method, the third column is the segmentation prediction result of the present invention, and the fourth column is the true label. Different colors represent different organs. Figure 3 It can be seen that the segmentation prediction results of the present invention are more complete than those of AdaMI and closer to the true labels, which shows that the present invention can perform segmentation prediction on organs in abdominal images very well.

[0229] Then, we conduct a quantitative comparison. First, we define image segmentation algorithm performance evaluation metrics. This experiment uses a medical image segmentation performance evaluation method with two specific metrics: Dice and ASSD. Dice stands for the Dice coefficient, which is twice the correctly segmented area divided by the union of the predicted result and the label. A larger Dice value indicates better segmentation performance. ASSD stands for Average Symmetric Surface Distance, which indicates average symmetric surface distance. A smaller ASSD value indicates better segmentation performance.

[0230] Based on the experimental results of this invention, we analyzed the results of experiments on the CHAOS abdominal MRI 3D image dataset. The source domain model was trained on an abdominal CT image dataset, so it was adapted from CT images to MRI images. Table 1 shows a quantitative comparison of the performance of this invention on the CHAOS dataset. “No adaptation” means directly using the source domain CT data segmentation model to segment MRI data. The segmentation performance at this time is very poor, which represents the lower bound of the segmentation performance and illustrates the necessity of domain adaptation. “Fully supervised” means training a segmentation model on MRI data to segment MRI data with labels, which represents the most ideal segmentation performance, that is, the upper bound of the segmentation performance. The five methods in the middle (SynSeg-Net, AdaOutput, CycleGAN, CyCADA, SIFA) are unsupervised domain adaptation methods. They do not meet the passive setting. The results in the table are from the literature “Cheng Chen, et al. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation [J]. IEEE transactions on medical imaging, 2020, 39(7): 2494-2505. Cheng Chen et al.'s paper: "Unsupervised Bidirectional Cross-Modal Adaptive Medical Image Segmentation Based on Deep Collaborative Image and Feature Alignment"; the bottom row shows the passive domain adaptation method, which compares the performance of the present invention with the current SOTA (State-of-the-Arts) passive domain adaptive medical image segmentation method SFS, and the previous passive domain adaptive medical image segmentation methods AdaMI and FVP. The experimental results show that the present invention can quickly and accurately perform image segmentation, achieving consistent performance improvements. The present invention outperforms all other SFDA methods on the right kidney, left kidney, and spleen, achieving the highest average Dice coefficient of 87.6% and the lowest average ASSD of 0.9.FVP and SFS do not provide code. The results in the table are respectively from the literature "Yan Wang, et al. Fvp: Fourier visual prompting for source-free unsupervised domain adaptation of medical image segmentation [J]. IEEE Transactions on Medical Imaging, 2023. Yan Wang et al.'s paper: Fvp: Passive unsupervised domain medical image segmentation based on Fourier visual prompts" and the literature "Serban Stan, et al. Unsupervised model adaptation for source-free segmentation of medical images [J]. Medical Image Analysis, 2024, 95: 103179. Serban Stan et al.'s paper: Unsupervised model adaptation for source-free segmentation of medical images [J]. Medical Image Analysis, 2024, 95: 103179. Serban Stan et al.'s paper: Unsupervised model adaptation for passive segmentation of medical images"; AdaMI provides source code, and the present invention uses the source code provided to conduct experiments on the CHAOS dataset.

[0231] Table 1

[0232]

[0233]

[0234] In summary, on the CHAOS dataset, the present invention achieved a new SOTA (state-of-the-art) in the field of passive domain adaptive medical image segmentation. As can be seen from Table 1, in the comparison of the mean Dice values, the Dice value of the present invention is significantly improved to 87.6% compared to 57.7% of "no adaptation", indicating the necessity of domain adaptation; compared with the other passive domain adaptive methods FVP, AdaMI, and SFS at the bottom, the Dice value of the present invention is 4.1% higher than the largest 83.5%, indicating that the present invention has a significant improvement and can achieve better segmentation results; and compared with the five classic unsupervised domain adaptive methods in the middle, SynSeg-Net, AdaOutput, CycleGAN, CyCADA, and SIFA, the Dice value of the present invention is also higher, indicating that the segmentation prediction results of the present invention are even better than those of the active domain methods. In the comparison of ASSD values, the ASSD value of the present invention is smaller than all the unsupervised domain adaptive methods in the middle and all the passive domain adaptive methods below, only 0.9, close to the upper limit of 0.6 of "full supervision". This shows that the organs segmented by the present invention are closer to the positions of the organs with real labels, and the boundary distances are similar. The Dice and ASSD values ​​of the present invention have achieved the best results compared with other passive domain adaptive medical image segmentation methods, indicating that the present invention has achieved the best segmentation effect.

Claims

1. A passive domain adaptive medical image segmentation method based on selective online self-training, characterized by The following steps are involved: The first step is to build a passive domain adaptive medical image segmentation system based on selective online self-training; The passive domain adaptive medical image segmentation system consists of a strong enhancement module, a weak enhancement module, a student model image segmentation module, a teacher model image segmentation module, an entropy-guided selective update module, a class prior-guided pseudo-label screening module, and a consistency loss function calculation module. The strong enhancement module is connected to the student model image segmentation module; during training, the strong enhancement module receives the training set, performs strong enhancement processing on the training set image, obtains the strongly enhanced image of the training set, and sends the strongly enhanced image of the training set to the student model image segmentation module; The weak enhancement module is connected to the teacher model image segmentation module; during training, the weak enhancement module receives the training set, performs weak enhancement processing on the training set image, obtains the weakly enhanced image of the training set, and sends the weakly enhanced image of the training set to the teacher model image segmentation module; The student model image segmentation module is connected to the strong enhancement module, the entropy-guided selective update module and the consistency loss function calculation module; during training, the student model image segmentation module receives the strongly enhanced image of the training set from the strong enhancement module, performs feature extraction and segmentation on the strongly enhanced image of the training set, obtains a first segmentation probability map, and sends the first segmentation probability map to the consistency loss function calculation module; at the same time, the entropy value is calculated based on the first segmentation probability map, and the entropy value is sent to the entropy-guided selective update module; during verification, the student model image segmentation module receives the verification set, performs feature extraction and segmentation on the verification set image, obtains the verification result, and determines whether to continue training based on the verification result; when segmenting the image input by the user, the student model image segmentation module receives the image to be segmented input by the user, performs feature extraction and segmentation on the image to be segmented, and obtains the segmentation result of the image to be segmented; the student model image segmentation module consists of a segmentation model, namely the student model, whose initial parameters adopt the parameters of the source domain segmentation model; the student model adopts the same neural network structure DeepLabv2 architecture as the source domain model; The entropy-guided selective update module is connected to the student model image segmentation module and the teacher model image segmentation module. Its function is to determine whether to update the parameters of the teacher model image segmentation module. During training, the entropy-guided selective update module receives entropy values ​​from the student model image segmentation module and saves the entropy values ​​of each iteration. It determines whether to update the teacher model image segmentation module based on the change in entropy values. If the entropy value decreases, it determines that the teacher model needs to be updated and sends the student model parameters to the teacher model image segmentation module. The teacher model image segmentation module is connected to the weak enhancement module, the entropy-guided selective update module, and the class prior-guided pseudo-label screening module. During training, the teacher model image segmentation module receives the weakly enhanced image data of the training set from the weak enhancement module, performs feature extraction and segmentation, obtains a second segmentation probability map, and sends the second segmentation probability map to the class prior-guided pseudo-label screening module. The teacher model image segmentation module is also composed of a segmentation model, namely the teacher model. The teacher model adopts the same neural network structure DeepLabv2 architecture as the source domain model, and its initial parameters adopt the parameters of the source domain segmentation model. When the student model parameters are received from the entropy-guided selective update module, the exponential moving average method EMA is used to update the parameters. The class prior-guided pseudo-label screening module is connected to the teacher model image segmentation module and the consistency loss function calculation module. During training, the class prior-guided pseudo-label screening module receives the second segmentation probability map from the teacher model image segmentation module and, based on the second segmentation probability map, gives confidence thresholds corresponding to the five segmentation classes of background, spleen, right kidney, left kidney, and liver in the abdominal image. The module then filters the segmentation class of each pixel in the second segmentation probability map based on the confidence threshold to obtain a pseudo-label, which is then sent to the consistency loss function calculation module. The consistency loss function calculation module is connected to the student model image segmentation module and the class prior-guided pseudo-label screening module. During training, the consistency loss function calculation module receives the first segmentation probability map from the student model image segmentation module and the pseudo-label from the class prior-guided pseudo-label screening module, performs a consistency comparison on the first segmentation probability map and the pseudo-label, obtains the gap between the first segmentation probability map and the pseudo-label, and uses the gap as the loss function to back-propagate and update the parameters of the student model image segmentation module. In the passive domain adaptive medical image segmentation system based on selective online self-training, the strong enhancement module, weak enhancement module, teacher model image segmentation module, entropy-guided selective update module, class prior-guided pseudo-label screening module, and consistency loss function calculation module are all auxiliary training modules for the student model image segmentation module. They only work during training and do not participate in the segmentation of the image to be segmented input by the user. Only the trained student model image segmentation module completes the segmentation of the image to be segmented input by the user. The second step is to construct the training set and validation set by: 2.1 We collected abdominal medical images as the target domain dataset for passive domain adaptive medical image segmentation. We used the CHAOS (Chemical Abdominal MRI) 3D image dataset as the target domain dataset for passive domain adaptive medical image segmentation. The dataset includes abdominal images and their corresponding true labels. The labels are 1, 2, 3, and 4 for the spleen, right kidney, left kidney, and liver, respectively, and 0 for other background regions in the image. CHAOS contains a total of 20 3D abdominal images. 2.2 Anonymize the target domain dataset of the passive domain adaptive medical image segmentation, stripping personal information such as the privacy and medical history of the subjects from the images; remove the background slices from each three-dimensional abdominal image, only retain the slices containing the segmentation object, and then perform two-dimensional slicing on each three-dimensional abdominal image and its three-dimensional label to obtain a series of two-dimensional images and corresponding two-dimensional labels; compose three consecutive two-dimensional slices in the slice order into a three-channel image, and the label corresponding to this three-channel image is the two-dimensional label of the second two-dimensional slice among them; name the processed three-channel image and the corresponding two-dimensional label according to the three-dimensional image name plus the slice order; after the above processing, there are a total of 492 three-channel images and corresponding two-dimensional labels in the target domain dataset of the passive domain adaptive medical image segmentation, denoted as the "CHAOS'" dataset; 2.3 According to the dataset division method commonly used by researchers, the CHAOS dataset is divided into training set D train , validation set D val , the total number of images in the training set is T, the total number of images in the validation set is V, T and V are positive integers and T+V=492; In the third step, use the gradient backpropagation method to train the passive domain adaptive medical image segmentation system based on selective online self-training constructed in the first step, and verify the segmentation accuracy of the passive domain adaptive medical image segmentation system using the validation set after every β iterations. When the segmentation accuracy no longer improves, obtain the network weight parameters of the best student model image segmentation module. The method is as follows: 3.1 Load the model network parameters of the source domain model provided by the user into the teacher model in the teacher model image segmentation module and the student model in the student model image segmentation module; 3.2 Set the network training configuration, including selecting the optimizer, initializing the learning rate, weight decay, and the maximum number of iterations T (ax , the number of iterations β, T (ax and β are both positive integers greater than 1; 3.3 Let the iteration number t = 1, train one batch of images in each iteration, with a total of B images in each batch, initialize the maximum update interval K, and both B and K are positive integers; 3.4 Strong enhancement module and weak enhancement module are simultaneously trained from the training set D train Read the B images of the tth batch in the image, recorded as tensor form image I train , strong enhancement module for I train Perform strong enhancement to obtain the strongly enhanced tensor QI train , QI train Send to the student model image segmentation module; weak enhancement module to I train Perform weak enhancement to obtain the weakly enhanced tensor RI train , change RI train Send to the teacher model image segmentation module; I train Contains B H×W×3 images; where H represents the height of the input image, W represents the width of the input image, and "3" represents the three slices contained in each image; 3.5 Student model image segmentation module receives QI from strong enhancement module train , execute steps 3.5.1 to 3.5.3; at the same time, the teacher model image segmentation module receives RI from the weak enhancement module train , execute steps 3.5.4 to 3.5.5; 3.5.1 Student model image segmentation module uses the first feature extraction and segmentation method to QI train Perform feature extraction and segmentation by: 3.5.1.1 Student Model Image Segmentation Module for QI train Perform feature extraction to obtain feature F1; 3.5.1.2 The student model image segmentation module segments feature F1 and obtains the first probability map of round t It is a four-dimensional tensor of size B×C×H×W, where C represents the number of categories to be segmented, C=5, i.e., background, spleen, right kidney, left kidney, and liver. 3.5.2 Student model image segmentation module will Send to the consistency loss function calculation module; 3.5.3 Student model image segmentation module based on the first segmentation probability map Calculate the entropy value of round t The calculation method is shown in formula (1): Will Send to the entropy-guided selective update module, go to 3.6; 3.5.4 The teacher model image segmentation module uses the second feature extraction and segmentation method to RI train Perform feature extraction and segmentation by: 3.5.4.1 Teacher Model Image Segmentation Module for RI train Perform feature extraction to obtain feature F2; 3.5.4.2 The teacher model image segmentation module segments feature F2 to obtain the second probability map of round t It is also a four-dimensional tensor of size B×C×H×W; 3.5.5 Send to the pseudo-label screening module guided by class priors, go to 3.6; 3.6 Entropy-guided Selective Update Module Preservation Adopting a selective update strategy, And the entropy value of the segmentation result of the student model image segmentation module when updating the teacher model in the tkth round Compare and determine whether to update the parameters of the teacher model based on the comparison results. k is the number of iterations since the last update. The method is: 3.6.1 If t = 1, it means there is only one round of iteration result. Let k = 1 and go to 3.7; if t > 1, the value of k has been obtained in the previous round of iteration, go to 3.6.2; 3.6.2 and For comparison, if This indicates that the reliability of the current results has increased, indicating that the student model's performance has improved. This indicates that the network weight parameters of the student model need to be used to update the network weight parameters of the teacher model. Send the student model's network weight parameters to the teacher model's image segmentation module and proceed to 3.6.

4. Otherwise, proceed to 3.6.

3. 3.6.3 If k ≥ K, it means that the network weight parameters of the teacher model need to be updated using the network weight parameters of the student model. Send the network weight parameters of the student model to the teacher model image segmentation module and go to 3.6.4; if k < K, let k = k + 1 and directly go to 3.7; 3.6.4 The teacher model image segmentation module performs exponential moving average EMA update using the network weight parameters of the student model to obtain the teacher model image segmentation module with updated network weight parameters; 3.6.5 Let k = 1 and go to 3.7; 3.7 Class Prior-Guided Pseudo-Label Filtering Module Receives the Second Probability Map of Round t from the Teacher Model Image Segmentation Module Use the confidence threshold to select pseudo labels and send the pseudo labels to the consistency loss function calculation module as follows: 3.7.1 Received from the Teacher Model Image Segmentation Module It is a four-dimensional tensor of size B×C×H×W, and each value corresponding to the four-dimensional coordinate is a probability between 0 and 1; Calculate the global confidence threshold τ as shown in formula (3): in express The b-th element in the first dimension of , b = 1,…,B; is a three-dimensional tensor of size C×H×W, corresponding to the probability map of the bth image among the B images read in the tth round, Represents the maximum value of all probabilities in this three-dimensional tensor; 3.7.2 Let b = 1, representing the b-th in the batch size B; 3.7.3 Initialize the class serial number c = 0; 3.7.4 Calculation The local threshold χ of the category number c b,c , as shown in formula (4): express The cth element in the first dimension is a two-dimensional tensor of size H×W; 3.7.5 Set the global confidence threshold τ and the local threshold X b,c Combined, we get The category number is c, and the initial confidence threshold is scaled by the upper bound factor γ to obtain The confidence threshold τ of the category number c b,c , as shown in formula (5): Where the upper bound factor γ = 0.8; 3.7.6 Calculation The class prior α of the class number c b,c , as shown in formula (6): Where h and w represent The height and width of h=1,…,H,w=1,…,W, represent The probability value corresponding to the four-dimensional coordinates (b, c, h, w); 3.7.7 Class Prior α b,c Incorporate into the confidence threshold τ b,c In, get The final confidence threshold τ′ of the category number c b,c , as shown in formula (7): Where the hyperparameter σ = 0.001; 3.7.8 If c < C - 1, let c = c + 1, and go to 3.7.4; if c = C - 1, it means the probability graph has been obtained The final confidence threshold for all categories is τ′ b,0 ,τ′ b,1 ,τ′ b,1 ,τ′ b,3 ,τ′ b,4 , and go to 3.7.9; 3.7.9 Using confidence threshold filtering The bth probability graph in get Pseudo labels, go to 3.7.10; 3.7.10 If b < B, let b = b + 1, and go to 3.7.3; if b ≥ B, it means the loop is completed, and the pseudo-label of the image in the t-th round is obtained Go to 3.7.11; 3.7.11 Class prior-guided pseudo-label screening module Send to the consistency loss function calculation module; 3.8 The consistency loss function calculation module receives the first probability map from the student model image segmentation module Receive pseudo labels from the pseudo label filtering module guided by class priors The pseudo label As the first probability map The supervisory signal calculates the cross entropy CE loss and Dice loss After adding them together, we get the consistency loss As shown in formula (9); express The number of pixels, express The number of pixels, express and The number of pixels with the same segmentation results; 3.9 Backpropagating Consistency Loss Using the Optimizer Update student model parameters; 3.10 If t is not divisible by β, let t = t + 1 and go to 3.4; if t is an integer multiple of β, go to 3.ll; 3.11 Using the validation set D val Verify the segmentation performance of the passive domain adaptive medical image segmentation system and save the network weight parameters when the segmentation performance no longer improves. The method is: 3.11.1 Student Model Image Segmentation Module Reads Validation Set D val , including the validation set D val Images, labels, and names in ; 3.11.2 Let batch number l = 1, which represents the lth batch. Each batch contains B′ validation set images, where B′ is a positive integer. The total number of batches is 3.11.3 Take out the B′ validation set images from the lth batch and record these B′ images as tensor form images I val , I val contains B′ H×W×3 images; 3.11.4 The student model image segmentation module receives B′ validation set D val The tensor form image I in val , to I val Perform feature extraction and segmentation to obtain the segmentation results I of the B′ validation set images in the lth batch l ; 3.11.5 The segmentation results of the B′ validation set images in the lth batch I l and save the corresponding labels and the names of the respective image files mentioned in the second step, where the names of the image files are the 3D image name + the slice order; 3.11.6 If l <N l , let l=l+1, go to 3.11.3; if l≥N l , indicating that all validation set data has been segmented, and the segmentation results, corresponding labels, and names of the respective image files for V validation set images have been obtained. Go to 3.11.

7. 3.11.7 Evaluate the segmentation performance of the student model using the Dice coefficient to determine whether to continue training. The method is as follows: 3.11.7.1 The segmentation results of the V validation set images are grouped together in slice order according to the names of their image files. That is, V segmentation results of size H×W are combined into J 3D images, namely X1, X2, …, X j ,…,X J , where J is the number of 3D images in the validation set, J = 4 on the CHAOS dataset, X j Size: H×W×S j ,S j is the number of slices of the jth 3D image in the validation set, ∑ j S j =V, j = 1, ..., J; the same operation is performed on the true labels corresponding to the segmentation results of the V validation set images to obtain J three-dimensional image labels Y1, Y2, ..., Y j ,…,Y J ; 3.11.7.2 Let j = 1, representing the j-th three-dimensional image in the validation set; 3.11.7.3 Segmentation result X for the j-th 3D image in the validation set j and the corresponding true label Y j , calculate the Dice coefficient of all segmentation categories, and get the Dice coefficient of all segmentation categories of the j-th three-dimensional image, that is, Dice j,1 ,Dice j,2 ,Dice j,3 ,Dice j,4 ; 3.11.7.4 If j < J, let j = j + 1 and go to 3.11.7.3; if j ≥ J, it means that the Dice coefficients of J three-dimensional images have been calculated, and the Dice coefficients of all segmentation results in the validation set compared with the true labels are obtained, that is, Dice 1,1 , Dice 1,2 , Dice 1,3 , Dice 1,4 , …, Dice j,1 , Dice j,2 , Dice j,3 , Dice j, x , …, Dice J,1 , Dice J, x , Dice J,3 , Dice J,4 , go to 3.11.7.5; 3.11.7.5 Calculate the average Dice coefficient for the four segmentation classes except the background, namely the spleen, right kidney, left kidney, and liver, for the J three-dimensional images, and obtain the average Dice coefficients ω1, ω2, ω3, ω4 for the 4 classes, The first type average Dice coefficient ω1=(Dice 1,1 +…+Dice j,1 …+Dice J,1 ) / J The second type of average Dice coefficient ω2=(Dice 1,2 +…+Dice j,2 …+Dice J,2 ) / J The third type average Dice coefficient ω3=(Dice 1,3 +…+Dice j,3 …+Dice J,3 ) / J The fourth type average Dice coefficient ω4=(Dice 1,4 +…+Dice j,4 …+Dice J,4 ) / J 3.11.7.6 Take the average of ω1, ω2, ω3, and ω4 to get the average Dice coefficient Ω for all images and all segmentation classes in the validation set in round t. t ,Ω t =(ω1+ω2+ω3+ω4) / 4; 3.11.7.7 In this case, t is an integer multiple of β. If t = β, save Ω t and the network weight parameters of the student model with iteration number t, go to 3.3; if t is greater than β, go to 3.11.7.8; 3.11.7.8 If t <T (ax , go to 3.11.7.9; if t ≥ T (ax , save the network weight parameters of the student model when the number of iterations is t, and go to step 4; 3.11.7.9 Saving Ω t and the network weight parameters of the student model with the number of iterations t, if Ω t >Ω t-β And (Ω t -Ω t-β )≥threshold η, 0<η<1, indicating that the student model is still being optimized during the training process, and the current number of iterations may not have reached the optimal number, so go to 3.3; if Ω t >Ω t-β And (Ω t -Ω t-β )<η, go to step 4; if Ω t ≤Ω t-β , indicating that the training is overfitting, and the training effect becomes worse. Let the number of iterations t = t-β and go to step 4; Step 4: Load the network weight parameters of the student model at iteration t into the student model image segmentation module to obtain the trained domain - adaptive medical image segmentation system based on selective online self - training with the best segmentation performance; Step 5: Use the trained domain - adaptive medical image segmentation system based on selective online self - training with the best segmentation performance to segment the input image to be segmented by the user, and obtain the image segmentation result. The method is as follows: 5.1 The student model image segmentation module receives an image to be segmented I input by the user user ; 5.2 Using the Student Model Image Segmentation Module to I user Perform segmentation to obtain the final segmentation result, and the image segmentation is completed.

2. The method for medical image segmentation based on selective online self-training passive domain adaptation according to claim 1, characterized in that The strong augmentation processing includes random cropping, rotation, random brightness, random contrast, and random gamma transformation. The weak augmentation processing includes random cropping, rotation, and scaling.

3. The method for medical image segmentation based on selective online self-training passive domain adaptation according to claim 1, characterized in that The partitioning method for the CHAOS’ dataset in step 2.3 is as follows: The total number of images in the training set is T = 395, and the total number of images in the validation set is V = 97.

4. The method for medical image segmentation based on selective online self-training passive domain adaptation according to claim 1, characterized in that The method for setting the network training configuration in step 3.2 is: the optimizer uses the Adam optimizer, and the initial learning rate is set to 1×10 -4 , weight decay is 5×10 -4 , the maximum number of iterations T (ax =2000, the initialization iteration number β=100; B=4 and K=30 in step 3.

3.

5. The method for medical image segmentation based on selective online self-training passive domain adaptation according to claim 1, characterized in that The EMA update method in step 3.6.4 is shown in formula (2): in represents the network weight parameter of the teacher model at the tkth iteration, θ t represents the network weight parameter of the student model at the tth iteration, represents the network weight parameter of the teacher model at the tth iteration, μ is the EMA update weight, μ = 0.

99.

6. The method for medical image segmentation based on selective online self-training passive domain adaptation according to claim 1, characterized in that 3.7.9 step using confidence threshold filtering The bth probability graph in get The pseudo-labeling method is: 3.7.9.1 Initialize the height h = 1; 3.7.9.2 Initialize the width w = 1; 3.7.9.3 The pixel with coordinates (h, w) in is classified into the category cls with the highest probability, as shown in formula (8): is a probability map of size C×H×W The category probability vector when the second and third dimension coordinates are (h, w) is of size C, that is, the C probabilities corresponding to classifying the pixel point with coordinates (h, w) into C categories; express The category corresponding to the maximum probability is denoted as cls; 3.7.9.4 If the probability Confirm that the category of the pixel with coordinates (h, w) is cls and go to step 3.7.9.

5. Otherwise, the category of the pixel with coordinates (h, w) is background, that is, 0, and go to step 3.7.9.

5. 3.7.9.5 If w < W, let w = w + 1, and go to 3.7.9.3; if w ≥ W, it means that the segmentation is completed when the height is h, and go to 3.7.9.6; 3.7.9.6 If h < H, let h = h + 1, and go to 3.7.9.2; if h ≥ H, it means the segmentation is completed, and obtain the pseudo labels, and end.

7. The method for medical image segmentation based on selective online self-training passive domain adaptation according to claim 1, characterized in that B′ = 4 in step 3.11.2; η = 0.001 in step 3.11.7.

8.

8. The method for medical image segmentation based on selective online self-training passive domain adaptation according to claim 1, characterized in that 3.11.7.3 The segmentation result X for the j-th 3D image in the validation set j and the corresponding true label Y j , the method to calculate the Dice coefficient of all segmentation categories is: 3.11.7.3.1 Initialize the category c = 1; 3.11.7.3.2 Calculate X according to formula (10) j and Y j Dice coefficient for category c j,c : where X j,c Represents the segmentation result X j The area belonging to category c, Y j,c Represents the true label Y j The area belonging to category c in |X j,c ∩Y j,c | represents the number of pixels that overlap between the two regions, |X j,c | represents the segmentation result X j The number of pixels in the region belonging to category c, |Y j,c | represents the true label Y j The number of pixels in the region belonging to category c; 3.11.7.3.3 If c < C - 1, let c = c + 1, and go to 3.11.7.3.2; if c = C - 1, it means that the Dice coefficients of all segmentation categories of the j-th 3D image have been calculated, and the Dice coefficients of all segmentation categories of the j-th 3D image, that is, Dice j,1 , Dice j,2 , Dice j,3 , Dice j,4 .

Images