Domain adaptation cross-modal medical image segmentation method and device based on boundary comparison, equipment and medium

Abstract

The invention provides a domain adaptation cross-modal medical image segmentation method and device based on boundary comparison, equipment and a medium, and relates to the technical field of medical image segmentation. The method comprises the following steps: respectively acquiring a source sample and a target domain image from a source domain data set and a target domain data set; the first image segmentation model is used as a teacher model, and weights are distributed for the pseudo labels of the target domain images; a training sample is obtained through bidirectional cross-domain cutmix; performing supervision and self-training by taking a weight learnable first model as a student model; combining supervision loss and comparison loss during training to pull a query sample to a prototype positive sample and push a boundary negative sample as a target optimization parameter, and obtaining a second image segmentation model; and finally, segmenting a second modal medical image by using the model. The method has significant advantages in cross-modal medical image segmentation, and the segmentation performance can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a domain-adaptive cross-modal medical image segmentation method, device, equipment and medium based on boundary comparison. Background Art

[0002] In recent years, deep convolutional neural networks have shown good performance in medical image segmentation. However, when the model is applied to real clinical scenarios, the performance often drops significantly because the characteristics of medical images acquired by different scanning protocols are different. To address the challenge of domain shift, unsupervised domain adaptation (UDA) is an effective solution that can transfer knowledge from a labeled source domain (such as MRI) to an unlabeled target domain (such as CT).

[0003] In the UDA research of medical image segmentation, most of them use adversarial training to align the distribution of two domains. For example, the image appearance is transformed by the cycle consistency adversarial generative network (CycleGAN) to align the distribution, or the image and feature alignment is used to solve the domain shift, or the adversarial entropy minimization is used to bridge the gap. However, these methods have problems such as difficulty in network convergence and easy collapse of training. In addition, contrastive learning aims to learn the discriminative representation space, and some works extend it to semantic segmentation tasks.

[0004] At the same time, self-training achieves knowledge transfer by leveraging the consistency constraints of the student and teacher models on the target predictions. For example, domain adaptation through cross-domain mixed sampling (DACS) performs well, which performs self-training by constructing training samples. However, the current state-of-the-art UDA methods usually focus on the overall segmentation performance of the entire object and ignore the object boundaries. In cross-modal medical image segmentation, this phenomenon is more serious due to the large distribution gap between modalities and the low intensity contrast of organ structures of different categories, resulting in poor segmentation performance. Summary of the invention

[0005] The present invention provides a domain-adaptive cross-modal medical image segmentation method, apparatus, device and medium based on boundary comparison to improve at least one of the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a domain-adaptive cross-modal medical image segmentation method based on boundary comparison, which comprises steps S1 to S7.

[0007] S1. Obtain source samples from the source domain dataset of the first modality. The source samples include source domain images and source labels.

[0008] S2. Acquire a target domain image corresponding to the source domain image from a target domain dataset of a second modality.

[0009] S3. Using the first image segmentation model trained based on the first modality data as a teacher model, inputting the target domain image into the teacher model, obtaining pseudo labels, and assigning different weights to the pseudo labels according to the predicted entropy of the pixels based on a dynamic weight assignment strategy.

[0010] S4. Through bidirectional cross-domain cutmix, the source domain image and the target domain image are mixed into a mixed image, and the source label and the pseudo label are mixed into a mixed label to obtain a training sample.

[0011] S5. Using the first image segmentation model with learnable weights as a student model, inputting the source samples into the student model for supervised training, and inputting the training samples into the student model for self-training.

[0012] S6. During training, query samples are obtained based on the features extracted by the student model, prototype positive samples are obtained based on the features extracted from the source domain image by the teacher model, and boundary negative samples are obtained based on the boundary features extracted from the boundary of each object by the source label. Iterative training is performed in combination with supervision loss and contrast loss, with the goal of pulling the query sample toward the prototype positive sample and pushing it away from the boundary negative sample, optimizing the parameters of the student model, enhancing the model's discrimination ability in the boundary area, and obtaining a second image segmentation model.

[0013] S7. Segment the medical image of the second modality according to the second image segmentation model, and obtain the segmented medical image of the second modality.

[0014] In an optional implementation, the dynamic weight allocation strategy is: ; ; in, is the confidence weight, Indicates the number of current training iterations, is the image sequence number, is the pixel number in the image, To predict entropy, and Respectively The first entropy threshold and the second entropy threshold of the iteration, is the confidence weight of the predefined target unsupervised loss, The number of categories representing labels, is the serial number of the label category, is the predicted probability, For the The iteration The first image The pixel The predicted probability of the class label category, and Represents the height and width of the image respectively.

[0015] and Respectively by and Calculating the quantiles yields: ; ; ; ; in, is a function in the numpy library that calculates quantiles, is the pixel-level entropy map, Indicates converting a two-dimensional array into a one-dimensional array for quantile calculation. and are the first parameter and the second parameter related to dynamic weight allocation, They are and The initial ratio of Indicates the maximum number of iterations.

[0016] In an optional implementation, step S4 specifically includes steps S41 to S43.

[0017] S41. Generate a zero-center mask , used to perform bidirectional mixing of samples of the source image and the target image to obtain a mixed image.

[0018] ; in, For the Mixed images, Indicates mixed, For the Source domain image, Indicates the image number, For the A target domain image, Indicates the target domain, is the zero-centered mask, Indicates multiplication, and Represents the height and width of the image respectively.

[0019] S42: Mix the source label and the pseudo label in the same mixing manner as the mixed image to obtain the mixed label .

[0020] S43: Obtain training samples according to the mixed image and the mixed label. , Indicates the image number, Indicates the number of mixed images.

[0021] In an optional implementation, obtaining a query sample based on features extracted by a student model specifically includes: For each category except the background category, reliable pixels in the current mini-batch that are below a preset entropy threshold are taken as query candidates.

[0022] In an optional implementation, obtaining a prototype positive sample based on features extracted from a source domain image by a teacher model specifically includes: Initialize category-level prototypes using category centers of original source domain pixel features : ; in, represents the number of samples in the source domain dataset, represents the source domain image, is the image sequence number, and Represents the height and width of the feature, is the pixel number in the image, is the output of the teacher model indicator head, For the The output of the teacher model indicator head corresponding to pixels, The actual label corresponding to the output Downsampling, represents the set of real numbers, yes The channel dimension of is the serial number of the label category, is the indicator function, when the condition is met , the value of the indicator function is 1, otherwise it is 0.

[0023] In each iteration, the category-level prototype is updated in a progressively refined manner. Iteration No. Category prototype It is defined by the class-level mean vector of pixel features in a mini-batch: ; in, is the momentum coefficient, Indicates The iteration Category prototypes, Indicates that it belongs to The number of pixels of the class, Indicates the size of the mini-batch.

[0024] In an optional implementation, obtaining a boundary negative sample based on a boundary feature extracted from the boundary of each object based on a source label specifically includes: Sample pixels that do not belong to the current object category from the periphery of each object area, and use their corresponding feature vectors as boundary negative samples, and use the category-level memory library to store the boundary negative samples. No. Class object, using downsampled labels Perform morphological operations to obtain a binary boundary mask , then, use and the downsampled labels Extract feature vectors and obtain boundary negative samples .in, .

[0025] In an optional embodiment, the loss function during training is for: ; in, is the supervision loss of labeled source domain images, represents minimizing the weighted cross entropy loss of the predicted value and the mixed pseudo-label, and is the balance coefficient, is the pixel-level contrastive learning loss of the source domain image, is the pixel-wise contrastive learning loss for mixed images.

[0026] ; in, represents the number of samples in the source domain dataset, is the image sequence number, is the cross entropy loss, is the Dice loss, is the true label, The labels predicted by the student model.

[0027] ; in, is the number of mixed images, subscripts, or superscripts Represents a mixed image, and Represents the height and width of the image, is the weight of the pixel in the mixed image, is the pixel number in the image, is the indicator function, The mixed image Mixed labels for pixels, is the predicted probability of the student model for the mixed image.

[0028] In the source domain dataset And the training sample set For the query sample, the pixel-level contrast loss is calculated using the prototype positive sample and the boundary negative sample to obtain the pixel-level contrast learning loss of the source domain image. Pixel-wise contrastive learning loss for mixed images .

[0029] Pixel-level contrast loss model for: ; in, is the number of label categories, is the serial number of the label category, The number of query samples, is the natural exponential function, category No. query samples, Represents the prototype, For temperature, is the number of negative samples, Represents negative samples.

[0030] In an optional embodiment, the source domain image of the first modality is one of a magnetic resonance imaging (MRI) image and a tomography (CT) image, and the target domain image of the second modality is the other of a magnetic resonance imaging (MRI) image and a tomography (CT) image.

[0031] In a second aspect, the present invention provides a domain-adaptive cross-modal medical image segmentation device based on boundary comparison, which includes a source sample acquisition module, a target domain image acquisition module, a pseudo-label acquisition module, a mixing module, a training module and a segmentation module.

[0032] The source sample acquisition module is used to acquire source samples from the source domain dataset of the first modality. The source samples include source domain images and source labels.

[0033] The target domain image acquisition module is used to acquire a target domain image corresponding to the source domain image from a target domain dataset of the second modality.

[0034] The pseudo-label acquisition module is used to use the first image segmentation model trained based on the first modality data as a teacher model, input the target domain image into the teacher model, obtain the pseudo-label, and assign different weights to the pseudo-label according to the predicted entropy of the pixel based on the dynamic weight assignment strategy.

[0035] The mixing module is used to mix the source domain image and the target domain image into a mixed image through bidirectional cross-domain cutmix, and mix the source label and the pseudo label into a mixed label to obtain a training sample.

[0036] The student model module is used to use the first image segmentation model with learnable weights as the student model, input the source samples into the student model for supervised training, and input the training samples into the student model for self-training.

[0037] The training module is used for training. During training, query samples are obtained based on the features extracted by the student model, prototype positive samples are obtained based on the features extracted from the source domain image by the teacher model, and boundary negative samples are obtained based on the boundary features extracted from the boundary of each object by the source label. Iterative training is performed in combination with the supervision loss and the contrast loss, with the goal of pulling the query sample toward the prototype positive sample and pushing it away from the boundary negative sample, optimizing the parameters of the student model, enhancing the model's discrimination ability in the boundary area, and obtaining a second image segmentation model.

[0038] A segmentation module is used to segment the medical image of the second modality according to the second image segmentation model, and obtain the segmented medical image of the second modality.

[0039] In a third aspect, the present invention provides a domain-adaptive cross-modal medical image segmentation device based on boundary contrast, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a domain-adaptive cross-modal medical image segmentation method based on boundary contrast as described in any paragraph of the first aspect.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a domain-adaptive cross-modal medical image segmentation method based on boundary comparison as described in any paragraph of the first aspect.

[0041] By adopting the above technical solution, the present invention can achieve the following technical effects: Compared with the traditional framework, the domain-adaptive cross-modal medical image segmentation method based on boundary contrast of the present invention actively focuses on the fuzzy boundary area and has excellent segmentation effect at the category boundary. For example, when segmenting a cross-modal cardiac dataset, the entropy at the boundary of different cardiac structure categories is low and the confidence is high, which effectively improves the segmentation performance. Secondly, unlike the previous method of one-way mixed samples, the present invention creates mixed images from two directions for self-training, by pasting blocks of the source domain in the target domain and pasting blocks of the target domain in the source domain, which greatly promotes the intermediate domain to better learn domain-invariant features. Furthermore, the proposed strategy can finely adjust the confidence weights of the pseudo-labels, effectively preventing training instability and early performance degradation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the specific implementation methods of the present invention. It should be understood that the following drawings only show certain specific implementation methods of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flowchart of a domain-adaptive cross-modal medical image segmentation method based on boundary comparison.

[0044] Figure 2 It is a logical block diagram of a domain-adaptive cross-modal medical image segmentation method based on boundary comparison. DETAILED DESCRIPTION

[0045] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] Example 1, please refer to Figure 1 and Figure 2 , a first embodiment of the present invention provides a domain-adaptive cross-modal medical image segmentation method based on boundary comparison, which includes steps S1 to S7.

[0047] S1. Obtain source samples from the source domain dataset of the first modality. The source samples include source domain images and source labels.

[0048] S2. Acquire a target domain image corresponding to the source domain image from a target domain dataset of a second modality.

[0049] In this embodiment, the source domain image of the first modality is one of a magnetic resonance imaging (MRI) image and a tomography (CT) image. The target domain image of the second modality is the other of a magnetic resonance imaging (MRI) image and a tomography (CT) image. In other embodiments, the first modality and the second modality may be other medical image acquisition methods, which are not specifically limited in the present invention.

[0050] The present invention first defines a labeled source domain dataset and an unlabeled target domain dataset = Our goal is to train a segmentation model that knowledge transfer to middle. is the source domain image, is the source tag, is the number of source samples, is the target domain image, is the number of samples in the target domain.

[0051] Figure 2 The overall framework of the unsupervised domain adaptation for medical image segmentation of the present invention is shown. The entire framework has two segmentation models, namely, the student model with learnable weights and a teacher model with exponential moving average (EMA) weights Each model consists of an encoder, a classifier, and an indicator head.

[0052] During the training process, the target data is input into the teacher model to obtain the predicted probability and pseudo labels , and dynamically assign weights later. and Represents the height and width of the image respectively, Indicates the number of categories. Then, bidirectional cross-domain cutmix is ​​used to mix the images of the two domains as well as the source domain labels and target pseudo labels to construct training samples. The specific operations are as follows: Step S3 to Step S6.

[0053] S3, using the first image segmentation model trained based on the first modality data as a teacher model, inputting the target domain image into the teacher model, obtaining pseudo labels, and assigning different weights to the pseudo labels according to the predicted entropy of the pixels based on a dynamic weight assignment strategy. Specifically, the present invention proposes a strategy to finely adjust the confidence weights of the pseudo labels to prevent training instability and early performance degradation.

[0054] Preferably, the dynamic weight allocation strategy is: ; ; in, is the confidence weight, Indicates the number of current training iterations, is the image sequence number, is the pixel number in the image, To predict entropy, and Respectively The first entropy threshold and the second entropy threshold of the iteration, is the confidence weight of the predefined target unsupervised loss, The number of categories representing labels, is the serial number of the label category, is the predicted probability, For the The iteration The first image The pixel The predicted probability of the class label category, and Represents the height and width of the image respectively.

[0055] and Respectively by and Calculating the quantiles yields: ; ; in, is a function in the numpy library that calculates quantiles, is the pixel-level entropy map, Indicates converting a two-dimensional array into a one-dimensional array for quantile calculation. and The first parameter and the second parameter are respectively related to the dynamic weight allocation. The first parameter and the second parameter change dynamically during the training process and are used to determine different entropy thresholds.

[0056] In an optional embodiment, the dynamic partition adjustment strategy is used in a linear manner.

[0057] ; ; in, They are and The initial ratio of Indicates the number of current training iterations, Indicates the maximum number of iterations.

[0058] This paper proposes a new strategy to dynamically assign weights to different parts of pseudo-labels to prevent early performance degradation. Specifically, since pseudo-labels are usually noisy, Domain Adaptation with Cross-Domain Mixed Sampling (DACS) defines a confidence weight for the target unsupervised loss. , which is the proportion of pixels in the entire image that exceed the threshold. However, this approach gives the same confidence weight to each pixel in the entire image, which means it neither removes unreliable pixels nor assigns higher weights to reliable pixels, leading to confirmation bias. In this case, model degradation will occur as training proceeds. To address this issue, we assign different weights to pixels based on entropy.

[0059] S4. Through bidirectional cross-domain cutmix, the source domain image and the target domain image are mixed into a mixed image, and the source label and the pseudo label are mixed into a mixed label to obtain a training sample. Specifically, unlike the traditional method of unidirectional mixed samples, the present invention creates mixed images for self-training from two directions, that is, pasting patches of the source domain in the target domain and pasting patches of the target domain in the source domain. This promotes better learning of domain-invariant features in the intermediate domain.

[0060] On the basis of the above embodiment, in an optional embodiment of the present invention, step S4 specifically includes steps S41 to S43.

[0061] S41. Generate a zero-center mask , used to perform bidirectional mixing of samples of the source image and the target image to obtain a mixed image.

[0062] ; in, For the Mixed images, Indicates mixed, For the Source domain image, Indicates the image number, For the A target domain image, Indicates the target domain, is the zero-centered mask, Indicates multiplication, and Represents the height and width of the image respectively.

[0063] S42: Mix the source label and the pseudo label in the same mixing manner as the mixed image to obtain the mixed label .

[0064] S43: Obtain training samples according to the mixed image and the mixed label. , Indicates the image number, Indicates the number of mixed images.

[0065] Specifically, Figure 2 As shown, two source images are randomly selected from the training set in each iteration. and two target images . And generate a zero-centered mask , used to perform bidirectional copy-paste of source and target images, the method is as follows: ; ; in, Is the image sequence number.

[0066] The source label and target pseudo label are also mixed in the same way to generate a mixed label and In this way, we get the training sample In addition, we will The confidence weight of the source image is set as .Will Calculated with the dynamic weight allocation strategy Mix to get .

[0067] The present invention constructs a bidirectional cross-domain mixed sample for self-training, and uses the intermediate domain to better learn domain-invariant features. Specifically, the patch of the source image is pasted on the target image, and the patch of the target image is also pasted on the source image.

[0068] S5. Using the first image segmentation model with learnable weights as a student model, inputting the source samples into the student model for supervised training, and inputting the training samples into the student model for self-training.

[0069] S6. During training, query samples are obtained based on the features extracted by the student model, prototype positive samples are obtained based on the features extracted from the source domain image by the teacher model, and boundary negative samples are obtained based on the boundary features extracted from the boundary of each object by the source label. Iterative training is performed in combination with supervision loss and contrast loss, with the goal of pulling the query sample toward the prototype positive sample and pushing it away from the boundary negative sample, optimizing the parameters of the student model, enhancing the model's discrimination ability in the boundary area, and obtaining a second image segmentation model.

[0070] The present invention constructs positive prototypes and negative boundary samples for contrast adaptation, so that the model actively pays attention to ambiguous boundary areas. Compared with the traditional medical imaging field adaptation segmentation framework, the boundary contrast domain adaptation framework of cross-modal medical image segmentation of the present invention actively pays attention to fuzzy boundary areas to improve segmentation performance. When segmenting cross-modal cardiac datasets, this method has relatively low entropy at the boundaries of different cardiac structure categories, high confidence, and better segmentation effect at category boundaries.

[0071] Based on the above embodiment, in an optional embodiment of the present invention, obtaining a prototype positive sample based on features extracted from a source domain image by a teacher model specifically includes: Initialize category-level prototypes using category centers of original source domain pixel features : ; in, represents the number of samples in the source domain dataset, represents the source domain image, is the image sequence number, and Represents the height and width of the feature, is the pixel number in the image, is the output of the teacher model indicator head, For the The output of the teacher model indicator head corresponding to pixels, The actual label corresponding to the output Downsampling, represents the set of real numbers, yes The channel dimension of is the serial number of the label category, is the indicator function, when the condition is met , the value of the indicator function is 1, otherwise it is 0.

[0072] In order to improve the domain-invariant representation ability of the prototype, we use a progressive refinement approach to update the category-level prototype in each iteration. Iteration No. Category prototype It is defined by the class-level mean vector of pixel features in a mini-batch: ; in, is the momentum coefficient, Indicates The iteration Category prototypes, Indicates that it belongs to The number of pixels of the class, Indicates the size of the mini-batch.

[0073] Based on the above embodiment, in an optional embodiment of the present invention, obtaining a boundary negative sample based on a boundary feature extracted from the boundary of each object based on a source label specifically includes: In order to enhance the discriminability of boundary regions, pixels that do not belong to the current object category are sampled from the periphery of each object region, and their corresponding feature vectors are used as boundary negative samples.

[0074] Output of the teacher model indicator head No. Class object, using downsampled labels Perform morphological operations to obtain a binary boundary mask , then, use and the downsampled labels Extract feature vectors and obtain boundary negative samples .

[0075] The model for obtaining boundary negative samples is: .

[0076] In order to keep the number of negative samples stable, a category-level memory bank is used to store the boundary negative samples.

[0077] Based on the above embodiment, in an optional embodiment of the present invention, obtaining a query sample based on features extracted by a student model specifically includes: For each category except the background category, reliable pixels with low entropy (reliable pixels below a preset entropy threshold) in the current mini-batch are taken as query candidates. Pixels.

[0078] Based on the above embodiment, in an optional embodiment of the present invention, the loss function during training is for: ; in, is the supervision loss of labeled source domain images, represents minimizing the weighted cross entropy loss of the predicted value and the mixed pseudo-label, and is the balance coefficient, is the pixel-level contrastive learning loss of the source domain image, is the pixel-wise contrastive learning loss for mixed images.

[0079] ; in, represents the number of samples in the source domain dataset, is the image sequence number, is the cross entropy loss, is the Dice loss, is the true label, The labels predicted by the student model.

[0080] ; in, is the number of mixed images, subscripts, or superscripts Represents a mixed image, and Represents the height and width of the image, is the weight of the pixel in the mixed image, is the pixel number in the image, is the indicator function, The mixed image Mixed labels for pixels, is the predicted probability of the student model for the mixed image.

[0081] Each query sample has a positive prototype and Negative samples Pairing. In the source domain dataset And the training sample set For the query sample, the pixel-level contrast loss is calculated using the prototype positive sample and the boundary negative sample to obtain the pixel-level contrast learning loss of the source domain image. Pixel-wise contrastive learning loss for mixed images .

[0082] Pixel-level contrast loss model for: ; in, is the number of label categories, is the serial number of the label category, The number of query samples, is the natural exponential function, category No. query samples, Represents the prototype, For temperature, is the number of negative samples, Represents negative samples.

[0083] Current state-of-the-art unsupervised domain adaptation (UDA) methods usually focus on the overall segmentation performance of the entire object, while ignoring the object's boundaries. The segmentation results given by previous methods usually have higher entropy and lower confidence in the boundary area, resulting in poor segmentation performance. In cross-modal medical image segmentation, this phenomenon is more serious due to the large distribution gap between different modalities and the low intensity contrast between organ structures of different categories.

[0084] The embodiment of the present invention proposes a new unsupervised domain adaptation framework based on boundary contrast. First, the centroid of the features on the source domain is calculated to obtain the category prototype features. At the same time, boundary features are extracted from the boundaries of each object according to the true label and stored in a category-level repository. Then, contrastive learning is introduced into the domain adaptation process. A set of pixel-level representations (queries) are pulled closer to their corresponding prototypes (positive samples) and away from the corresponding boundary features (negative samples). In this way, we explicitly enhance the similarity between pixel features and corresponding prototypes to reduce the category-level distribution differences between domains, while increasing the discriminative ability on the boundaries.

[0085] S7. Segment the medical image of the second modality according to the second image segmentation model, and obtain the segmented medical image of the second modality.

[0086] It should be noted that in the field of medical imaging, different image modalities (such as magnetic resonance imaging (MRI), tomography (CT), etc.) usually provide different image information. However, when a trained model is applied to images of different modalities, its performance often decreases significantly. Effective prediction of new modality images usually requires retraining. This not only consumes a lot of computing and storage resources, but also requires experienced radiologists to perform pixel-level annotations on new modality data, which is very expensive and time-consuming and not suitable for practical applications. Currently, most unsupervised domain adaptation (UDA) for medical image segmentation uses adversarial training to align the distribution between the two domains. Although these methods are effective, they have problems such as difficulty in network convergence and easy collapse during training.

[0087] like Figure 2As shown, a domain-adaptive cross-modal medical image segmentation method based on boundary contrast of the present invention. First, the source data is directly input into the student model for supervised training. Secondly, we construct mixed training samples for self-training through bidirectional cross-domain cutmix and dynamically assign weights. Third, the query sample is pulled towards its corresponding prototype and pushed away from the corresponding boundary feature. Among them, the query samples are all from students, while the positive and negative samples are from teachers. Negative samples are stored in a category-level repository. This method actively focuses on the fuzzy boundary areas to improve the segmentation performance, while constructing training samples through bidirectional cross-domain cutmix to further narrow the domain gap, and introduces a dynamic weight allocation strategy to prevent the model's early performance from degrading.

[0088] Compared with the traditional framework, the domain-adaptive cross-modal medical image segmentation method based on boundary contrast of the present invention actively focuses on the fuzzy boundary area and has excellent segmentation effect at the category boundary. For example, when segmenting a cross-modal cardiac dataset, the entropy at the boundary of different cardiac structure categories is low and the confidence is high, which effectively improves the segmentation performance. Secondly, unlike the previous one-way mixed sample method, the present invention creates training samples from two directions for self-training, by pasting blocks of the source domain in the target domain and pasting blocks of the target domain in the source domain, which greatly promotes the intermediate domain to better learn domain-invariant features. Furthermore, the proposed strategy can finely adjust the confidence weights of the pseudo-labels, effectively preventing training instability and early performance degradation.

[0089] Embodiment 2: The present invention provides a domain-adaptive cross-modal medical image segmentation device based on boundary comparison, which includes a source sample acquisition module, a target domain image acquisition module, a pseudo-label acquisition module, a mixing module, a training module and a segmentation module.

[0090] The source sample acquisition module is used to acquire source samples from the source domain dataset of the first modality. The source samples include source domain images and source labels.

[0091] The target domain image acquisition module is used to acquire a target domain image corresponding to the source domain image from a target domain dataset of the second modality.

[0092] The pseudo-label acquisition module is used to use the first image segmentation model trained based on the first modality data as a teacher model, input the target domain image into the teacher model, obtain the pseudo-label, and assign different weights to the pseudo-label according to the predicted entropy of the pixel based on the dynamic weight assignment strategy.

[0093] The mixing module is used to mix the source domain image and the target domain image into a mixed image through bidirectional cross-domain cutmix, and mix the source label and the pseudo label into a mixed label to obtain a training sample.

[0094] The student model module is used to use the first image segmentation model with learnable weights as the student model, input the source samples into the student model for supervised training, and input the training samples into the student model for self-training.

[0095] The training module is used for training. During training, query samples are obtained based on the features extracted by the student model, prototype positive samples are obtained based on the features extracted from the source domain image by the teacher model, and boundary negative samples are obtained based on the boundary features extracted from the boundary of each object by the source label. Iterative training is performed in combination with the supervision loss and the contrast loss, with the goal of pulling the query sample toward the prototype positive sample and pushing it away from the boundary negative sample, optimizing the parameters of the student model, enhancing the model's discrimination ability in the boundary area, and obtaining a second image segmentation model.

[0096] A segmentation module is used to segment the medical image of the second modality according to the second image segmentation model, and obtain the segmented medical image of the second modality.

[0097] Embodiment 3, the present invention provides a domain-adaptive cross-modal medical image segmentation device based on boundary contrast, which includes a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a domain-adaptive cross-modal medical image segmentation method based on boundary contrast as described in any paragraph of Embodiment 1.

[0098] Embodiment 4: The present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a domain-adaptive cross-modal medical image segmentation method based on boundary comparison as described in any paragraph of Embodiment 1.

[0099] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0100] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0101] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0102] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0103] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0104] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0105] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A domain-adaptive cross-modal medical image segmentation method based on boundary contrast, characterized in that: Include: Obtain source samples from a source domain dataset of the first modality; the source samples include source domain images and source labels; Acquire a target domain image corresponding to the source domain image from a target domain dataset of a second modality; Using a first image segmentation model trained based on first modality data as a teacher model, inputting a target domain image into the teacher model, obtaining pseudo labels, and assigning different weights to the pseudo labels according to predicted entropy of pixels based on a dynamic weight assignment strategy; By bidirectional cross-domain cutmix, the source domain image and the target domain image are mixed into a mixed image, and the source label and the pseudo label are mixed into a mixed label to obtain a training sample; Using a first image segmentation model with learnable weights as a student model, inputting the source samples into the student model for supervised training, and inputting the training samples into the student model for self-training; During training, query samples are obtained based on the features extracted by the student model, prototype positive samples are obtained based on the features extracted from the source domain image by the teacher model, and boundary negative samples are obtained based on the boundary features extracted from the boundary of each object by the source label. Iterative training is performed in combination with supervision loss and contrast loss, with the goal of pulling the query sample toward the prototype positive sample and pushing it away from the boundary negative sample, optimizing the parameters of the student model, enhancing the model's discrimination ability in the boundary area, and obtaining a second image segmentation model; According to the second image segmentation model, the medical image of the second modality is segmented to obtain the segmented medical image of the second modality.

2. The method for cross-modal medical image segmentation based on boundary contrast according to claim 1, characterized in that: The dynamic weight allocation strategy is: ； ； in, is the confidence weight, Indicates the number of current training iterations, is the image sequence number, is the pixel number in the image, To predict entropy, and Respectively The first entropy threshold and the second entropy threshold of the iteration, is the confidence weight of the predefined target unsupervised loss, Indicates the number of categories of labels, is the serial number of the label category, is the predicted probability, For the The iteration The first image The pixel The predicted probability of the class label category, and Respectively represent the height and width of the image; and Respectively by and Calculating the quantiles yields: ； ； ； ； in, is a function in the numpy library that calculates quantiles, is the pixel-level entropy map, Indicates converting a two-dimensional array into a one-dimensional array for quantile calculation. and are the first parameter and the second parameter related to dynamic weight allocation, They are and The initial ratio of Indicates the maximum number of iterations.

3. The domain-adaptive cross-modal medical image segmentation method based on boundary contrast according to claim 1, characterized in that: By bidirectional cross-domain cutmix, the source domain image and the target domain image are mixed into a mixed image, the source label and the pseudo label are mixed into a mixed label, and a training sample is obtained, specifically including: Generate a zero-centered mask , used to perform bidirectional mixing of samples of the source image and the target image to obtain a mixed image; ； in, For the Mixed images, Indicates mixed, For the Source domain image, Indicates the image number, For the A target domain image, Indicates the target domain, is the zero-centered mask, Indicates multiplication, and Respectively represent the height and width of the image; The source label and the pseudo label are mixed in the same way as the mixed image to obtain the mixed label. ; According to the mixed image and the mixed label, a training sample is obtained. , Indicates the image number, Indicates the number of mixed images.

4. The domain-adaptive cross-modal medical image segmentation method based on boundary contrast according to claim 1, characterized in that: Prototype positive samples are obtained based on the features extracted from the source domain image by the teacher model, including: Initialize category-level prototypes using category centers of original source domain pixel features : ； in, represents the number of samples in the source domain dataset, represents the source domain image, is the image sequence number, and Represents the height and width of the feature, is the pixel number in the image, is the output of the teacher model indicator head, For the The output of the teacher model indicator head corresponding to pixels, The actual label corresponding to the output Downsampling, represents the set of real numbers, yes The channel dimension of is the serial number of the label category, is the indicator function, when the condition is met When , the value of the indicator function is 1, otherwise it is 0; In each iteration, the category-level prototype is updated in a progressively refined manner; Iteration No. Category prototype It is defined by the class-level mean vector of pixel features in a mini-batch: ； in, is the momentum coefficient, Indicates The iteration Category prototypes, Indicates that it belongs to The number of pixels of the class, Indicates the size of the mini-batch.

5. The domain-adaptive cross-modal medical image segmentation method based on boundary contrast according to claim 4, characterized in that: The boundary features extracted from the boundary of each object based on the source label are used to obtain boundary negative samples, including: Sample pixels that do not belong to the current object category from the periphery of each object area, and use the corresponding feature vector as the boundary negative sample, and use the category-level memory library to store the boundary negative sample; wherein, for the output of the teacher model indicator head No. Class object, using downsampled labels Perform morphological operations to obtain a binary boundary mask , then, use and the downsampled labels Extract feature vectors and obtain boundary negative samples ;in, .

6. A domain-adaptive cross-modal medical image segmentation method based on boundary contrast according to any one of claims 1 to 5, characterized in that: The source domain image of the first modality is one of a nuclear magnetic resonance imaging (MRI) image and a tomographic scanning (CT) image; the target domain image of the second modality is the other of the nuclear magnetic resonance imaging (MRI) image and the tomographic scanning (CT) image; The query samples are obtained based on the features extracted by the student model, including: For each category except the background category, reliable pixels in the current mini-batch that are below a preset entropy threshold are taken as query candidates.

7. The domain-adaptive cross-modal medical image segmentation method based on boundary contrast according to any one of claims 1 to 5, characterized in that: Loss function during training for: ； in, is the supervision loss of labeled source domain images, represents minimizing the weighted cross entropy loss of the predicted value and the mixed pseudo-label, and is the balance coefficient, is the pixel-level contrastive learning loss of the source domain image, is the pixel-wise contrastive learning loss for mixed images; ； in, represents the number of samples in the source domain dataset, is the image sequence number, is the cross entropy loss, is the Dice loss, is the true label, The labels predicted by the student model; ； in, is the number of mixed images, subscripts, or superscripts Represents a mixed image, and Represents the height and width of the image, is the weight of the pixel in the mixed image, is the pixel number in the image, is the indicator function, The mixed image Mixed labels for pixels, is the predicted probability of the student model for the mixed image; In the source domain dataset And the training sample set For the query sample, the pixel-level contrast loss is calculated using the prototype positive sample and the boundary negative sample to obtain the pixel-level contrast learning loss of the source domain image. Pixel-wise contrastive learning loss for mixed images ; Pixel-level contrast loss model for: ； in, is the number of label categories, is the serial number of the label category, The number of query samples, is the natural exponential function, category No. query samples, Represents the prototype, For temperature, is the number of negative samples, Represents negative samples.

8. A domain-adaptive cross-modal medical image segmentation device based on boundary contrast, characterized in that: Include: A source sample acquisition module is used to acquire source samples from a source domain dataset of the first modality; the source samples include source domain images and source labels; A target domain image acquisition module, used to acquire a target domain image corresponding to the source domain image from a target domain dataset of a second modality; a pseudo-label acquisition module, configured to use a first image segmentation model trained based on the first modality data as a teacher model, input a target domain image into the teacher model, acquire pseudo-labels, and assign different weights to the pseudo-labels according to the predicted entropy of the pixels based on a dynamic weight assignment strategy; A mixing module, used for mixing the source domain image and the target domain image into a mixed image, mixing the source label and the pseudo label into a mixed label through bidirectional cross-domain cutmix, and obtaining a training sample; A student model module, configured to use a first image segmentation model with learnable weights as a student model, input the source sample into the student model for supervised training, and input the training sample into the student model for self-training; A training module, used for obtaining query samples based on features extracted by the student model, obtaining prototype positive samples based on features extracted from source domain images by the teacher model, obtaining boundary negative samples based on boundary features extracted from the boundary of each object by the source label, and performing iterative training in combination with supervision loss and contrast loss, with the goal of pulling the query samples toward the prototype positive samples and pushing them away from the boundary negative samples, optimizing the parameters of the student model, enhancing the model's discrimination ability in the boundary area, and obtaining a second image segmentation model; A segmentation module is used to segment the medical image of the second modality according to the second image segmentation model, and obtain the segmented medical image of the second modality.

9. A domain-adaptive cross-modal medical image segmentation device based on boundary contrast, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory; the computer program can be executed by the processor to implement a domain-adaptive cross-modal medical image segmentation method based on boundary contrast as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute a domain-adaptive cross-modal medical image segmentation method based on boundary comparison as described in any one of claims 1 to 7.

Images