Medical image segmentation method based on semi-supervised domain self-adaption and cross-domain collaborative learning

Abstract

The invention provides a medical image segmentation method based on semi-supervised domain self-adaption and cross-domain collaborative learning. The method comprises the implementation steps that a mixed training sample set, a verification sample set and a test sample set of a source domain and a target domain are acquired; constructing a medical image segmentation network model based on semi-supervised domain self-adaption and cross-domain collaborative learning, and performing iterative training on the medical image segmentation network model; and obtaining a segmentation result of the medical image. In the process of training the image segmentation network model, the cross-domain contrast loss function can integrate the dice similarity into the loss, and the context information of the to-be-segmented region is considered. According to the pseudo-label re-learning loss function, labels generated by the network with high segmentation accuracy in the source domain and the target segmentation teacher network are selected as pseudo labels of label-free data in the target domain, label-free target domain data are utilized, domain self-adaptive processing is performed on the target domain and source domain training sample set points, source domain data are utilized, and the pseudo-label re-learning loss function is obtained. And the segmentation precision of the medical image is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and relates to a medical image segmentation method, and specifically to a semi-supervised medical image segmentation method based on domain adaptation and cross-domain collaborative learning. Background Art

[0002] Image segmentation is a key task in the field of computer vision. It is the process of dividing a digital image into different parts or regions. Its purpose is to classify the pixels in the image according to their semantic information, so as to separate and understand the image. Image segmentation is currently widely used in autonomous driving, video surveillance, medical imaging and other fields.

[0003] The purpose of medical image segmentation is to separate different structures, organs or lesion areas in medical images (such as CT, MRI, X-ray, etc.) from the background, so as to help doctors and medical experts better analyze and diagnose the patient's health status. However, due to the privacy protection of patients and the professionalism of the required annotations, the training of medical image segmentation models usually faces the problem of insufficient training data and lack of annotations. In real life, many medical challenges are held on many well-known open source platforms. These challenges can provide open source medical image datasets with large data volumes, which contain professional and detailed pixel-level segmentation annotations for medical image tissue areas or lesion areas. These datasets can be used to help train the model of the target task, that is, to adapt the model trained by the existing open source dataset (source domain) to a specific target domain dataset (target domain). This type of method is called domain adaptation.

[0004] In order to reduce the performance gap caused by the difference in data distribution between different domains, the existing medical image segmentation method based on semi-supervised domain adaptation, such as the patent application with application publication number CN115731178A and name “A cross-modal unsupervised domain adaptive medical image segmentation method”, discloses a medical image segmentation method based on unsupervised domain adaptation, and the implementation steps are: data preprocessing; iterative training and parameter updating; acquisition of segmentation model; acquisition of segmentation results; the invention has the ability to learn the common features between the two modalities without using any target domain annotations, and has good segmentation accuracy on both the source modality and the target modality images. However, its shortcoming is that this method needs to ensure that the source domain image and the target domain image are different modalities of the same lesion. However, in most cases, different modality images of different lesions or the same modality of different lesions are easier to obtain. When using such a dataset for training, due to the lack of supervision from the target domain and the large semantic gap between the style and content information of the source and target domains, and when the content-specific information of the image is coupled with the style information of its specific domain, it is impossible to learn the correct representation of domain-independent content. At the same time, it is interfered by the style of the specific domain, which affects the further improvement of the segmentation accuracy. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and propose a medical image segmentation method based on semi-supervised domain adaptation and cross-domain collaborative learning to solve the technical problem of low segmentation accuracy in the prior art.

[0006] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0007] (1) Obtain a mixed training sample set, a validation sample set, and a test sample set from the source domain and the target domain:

[0008] For Z, which includes multiple pixel categories and has labels s 3D source domain medical image and Z t The unlabeled target domain 3D medical images are preprocessed respectively, and the source domain mixed training sample set and the target domain mixed training sample set based on domain adaptation, as well as the verification sample set and the test sample set are obtained through the preprocessed 2D medical images. s ≥150, Z t ≥50;

[0009] (2) Construct a medical image segmentation network model based on cross-domain collaborative learning:

[0010] Constructing a target domain segmentation student model including parallel arrangement t , target domain segmentation teacher model θ t , source domain segmentation student model ξ sand the source domain segmentation teacher model θ s , and s and t The cross-domain collaborative loss function is the medical image segmentation network model O of Loss, where:

[0011]

[0012] L unsup =L mse +λ asc L ASC1 +λ rel L rel

[0013] Where L sup , L unsup They represent the supervised loss function and the unsupervised loss function of cross-domain collaborative learning, respectively, and λ ramp Indicates L unsup The weight coefficient, R represents the maximum number of iterations, r represents the number of iterations, L mse represents the cross-domain mean square error consistency loss, L ASC1 represents the cross-domain contrast loss, L rel represents the pseudo-label relearning loss, λ asc and λ rel Respectively represent L ASC1 and L rel The weight coefficient of

[0014] (3) Iterative training of the medical image segmentation network model:

[0015] Iteratively train the medical image segmentation network model O through the mixed training sample set and verification sample set of the source domain and the target domain to obtain the trained medical image segmentation network model O*;

[0016] (4) Obtain the segmentation results of medical images:

[0017] The target domain test sample set T tla As the input of the trained medical image segmentation network model O*, the target domain segmentation student network ξ t * Perform forward reasoning on each target domain test sample to obtain the segmentation result map of each test sample.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] 1. In the process of iterative training of the image segmentation network model of the present invention, the cross-domain contrast loss function in the loss function of the target domain segmentation student model and the source domain segmentation student model can integrate the dice similarity into the loss, and fully consider the contextual information of the area to be segmented. At the same time, the pseudo-label relearning loss function selects the labels generated by the networks with higher segmentation accuracy in the source domain segmentation teacher network and the target domain segmentation teacher network as the pseudo-labels of the unlabeled data in the target domain, which makes full use of the unlabeled target domain data and effectively improves the segmentation accuracy of medical images.

[0020] 2. The present invention performs domain adaptation on the target domain training sample set and the source domain training sample set, and constructs a source domain mixed training sample set and a target domain mixed training sample set based on domain adaptation, which are used to train the source domain segmentation network and the target domain segmentation network respectively. This effectively utilizes the source domain data, avoids the limitation of the prior art that the source domain image and the target domain image must be of the same lesion, and effectively improves the segmentation accuracy of medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart for realizing the present invention;

[0022] Figure 2 This is a flowchart for implementing domain adaptive processing of a target domain training sample set and a source domain training sample set in the present invention;

[0023] Figure 3 It is a structural schematic diagram of the domain segmentation model of the present invention;

[0024] Figure 4 Schematic diagram of the structure of the convolution module of the present invention. DETAILED DESCRIPTION

[0025] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0026] Reference Figure 1 , the present invention comprises the following steps:

[0027] Step 1) Obtain a mixed training sample set, a validation sample set, and a test sample set from the source domain and the target domain:

[0028] Step 1a) Obtain Z from the BraTS brain tumor segmentation dataset s The dimension is D s ×H s ×W s The 3D brain tumor medical images in the flair mode and their corresponding segmentation labels constitute the 3D source domain medical images and their corresponding 3D segmentation labels. The Z t The dimension is D t ×Ht ×W t The 3D cerebral hemorrhage medical image constitutes a 3D target domain medical image. In this embodiment, Z s =246, D s =155,H s =240,W s =240, Z t =270, D t =32, H t =240,W t =255.

[0029] Step 1b) Along Z s The dimension is D s ×H s ×W s The 3D source domain medical image and its corresponding dimension is D s ×H s ×W s The dimension D of the 3D segmentation labels s Extract the vertical s Slice the axis to get K s The dimension is H s ×W s The source domain medical image slices; while along Z t The dimension is D t ×H t ×W t The dimension D of the 3D target domain medical image t Extract the vertical t Slice the axis to get K t The dimension is H t ×W t The target domain medical image slices, where K s =Z s ×D s , K t =Z t ×D t In this embodiment, K s =38130,K t =8640.

[0030] Step 1c) K s The dimension is H s ×W s The source domain medical image slices and their corresponding labels, as well as K t The dimension is H t ×W tThe target domain medical image slices are respectively center cropped with a dimension of H×W, and the center cropped source domain medical image slices and the cropped target domain medical image slices are normalized to obtain the preprocessed K s Source domain medical image slices with dimension H×W and their corresponding segmentation labels And the preprocessed K with dimensions H×W t target domain medical image slice In this embodiment, H=256, W=256.

[0031] Step 1d) The pre-processed K t The pixels in E images of the target domain medical images are labeled, and then the P target domain medical images and their labels are combined into a labeled target domain training sample set. The remaining V t The target domain medical images and their labels constitute the target domain verification sample set The remaining unlabeled K t -U of the E target domain medical images constitute the unlabeled target domain training sample set The remaining N target domain medical images constitute the unlabeled target domain test sample set Where E+U+N=K t , P+V t =E, Where T tla ∪T tua Construct a target domain training sample set with partial labels; s V in source domain medical images and their labels s The source domain medical images and their labels constitute the source domain verification sample set The remaining K s -V s The source domain medical images and their labels constitute the source domain training sample set in In this embodiment, E=2560, P=640, N=3040, V t =1920, U=3040, V S =10000.

[0032] Step 1e) The process of performing domain adaptation processing on each target domain training sample and each source domain training sample is as follows: Figure 2 As shown, the source domain training sample x s Training sample x in the target domain t,l Perform grayscale histogram matching on the template image to obtain the grayscale matching image r stl , for x s and rst Perform fast Fourier transform respectively to get x s The amplitude spectrum F A (x s ), x s The phase spectrum F P (x s ) and r stl The amplitude spectrum F A (r stl );x in the target domain training sample set t,l Take the source domain training sample x s Perform grayscale histogram matching on the template image to obtain the grayscale matching image r tls , for x t,l and r tls Perform fast Fourier transform respectively to get x t,l The amplitude spectrum F A (x t,l ), x t,l The phase spectrum F P (x t,l ) and r tls The amplitude spectrum F A (r tls );x in the target domain training sample set t,u Take the source domain training sample x s Perform grayscale histogram matching on the template image to obtain the grayscale matching image r tus , for x t,u and r tus Perform fast Fourier transform respectively to get x t,u The amplitude spectrum F A (x t,u ), x t,u The phase spectrum F P (x t,u ) and r tus The amplitude spectrum F A (r tus ) and calculate F A (r stl ), F A (r tls ), F A (r tus ) and the binary rectangular mask M λ The Hadamard product of stl 、r tls 、r tus The low-frequency information of A (x s ), F A (x t,l ), F A (x t,u) and the binary rectangular mask inverse 1 H,W -M λ The Hadamard product of s 、x t,l 、x t,u The high frequency information of x s The high frequency information and r stl The amplitude spectrum and x of the domain adaptive image obtained by summing the low-frequency information of s The phase spectrum F P (x s ) performs inverse Fourier transform to obtain x s Domain adaptation image x s→t ; for x t,l The high frequency information and r tls The amplitude spectrum and x of the domain adaptive image obtained by summing the low-frequency information of t,l The phase spectrum F P (x t,l ) performs inverse Fourier transform to obtain x t,l Domain adaptation image x t,l→s ; for x t,u The high frequency information and r tus The amplitude spectrum and x of the domain adaptive image obtained by summing the low-frequency information of t,u The phase spectrum F P (x t,u ) performs inverse Fourier transform to obtain x t,u Domain adaptation image x t,u→s The calculation formulas are:

[0033]

[0034] Among them, x s→t 、x t,l→s 、x t,u→s Respectively represent x s 、x t,l 、x t,u Domain adaptive image, λ represents the control binary mask M λ The size parameter, λ∈(0,1), represents the indicator function, h∈(0,H], w∈(0,W], H, W represent the number of pixels in the vertical and horizontal directions of the 2D medical image, respectively, and F -1 (·) represents the inverse Fourier transform, 1 H,W represents a matrix of dimension H×W with all elements set to 1. represents the Hadamard product, r stl Represents x s With x t,l Grayscale matching image after grayscale histogram matching, F A (r stl ) indicates thestl After fast Fourier transform, stl The amplitude spectrum, F A (x s ) and F P (x s ) represent the s After fast Fourier transform x s The amplitude spectrum and phase spectrum of r tls Represents x t,l With x s Grayscale matching image after grayscale histogram matching, F A (r tls ) indicates the tls After fast Fourier transform, tls The amplitude spectrum, F A (x t,l ) and F P (x t,l ) represent the t,l After fast Fourier transform x t,l The amplitude spectrum and phase spectrum of r tus Represents x t,u With x s Grayscale matching image after grayscale histogram matching, F A (r tus ) indicates the tus After fast Fourier transform, tus The amplitude spectrum of F A (x t,u ) and F P (x t,u ) represent the t,u After fast Fourier transform x t,u In this embodiment,

[0035] Step 1f) Mix the domain adaptive images and their labels corresponding to the source domain training sample set with all the preprocessed target domain training sample sets to form the target domain mixed training sample set The domain adaptive images and their labels corresponding to the target domain training sample set are mixed with all the preprocessed source domain training sample sets to form the source domain mixed training sample set.

[0036] Step 2) Construct a medical image segmentation network model based on semi-supervised domain adaptation and cross-domain collaborative learning:

[0037] Step 2a) Construct a target domain segmentation student model ξ including parallel arrangement t , target domain segmentation teacher model θ t , source domain segmentation student model ξs and the source domain segmentation teacher model θ s Medical image segmentation network model O. The four domain segmentation models all use encoders including sequential connections Decoder And the output module Out, its structure is as follows Figure 3 As shown, the encoder It includes a sequentially connected convolution module Conv and a level I convolution Conv-encoding En module; a decoder It includes I-level convolution Conv-decoding De modules connected in sequence; the input end of the i-th level convolution Conv-encoding En module is connected to the middle position of the I-i+1-th level convolution Conv-decoding De module. In this embodiment, I=4.

[0038] The structure of the convolution module Conv is as follows Figure 4 As shown in the figure: the first convolution layer → the first batch of regularization layers → the first nonlinear function layer → Dropout layer → the second convolution layer → the second batch of regularization layers → the second nonlinear function layer. The first nonlinear function layer and the second nonlinear function layer in all Conv modules use the Leakyrelu nonlinear activation function. The loss probability of the Dropout layer is set to 0.5. The Dropout layer randomly inactivates the neurons in the network model with a probability of 0.5, that is, the output of the neuron is set to 0; the first convolution layer has the same structure as the second convolution layer; the convolution kernels in the first convolution layer and the second convolution layer in the convolution module Conv are set to 3×3, the number is 16, the step size is 1, and the surrounding padding is 1; the first-level encoding submodule En1 includes the maximum pooling layer and the first convolution module Conv1 connected in sequence, where the first convolution layer and the second convolution layer in Conv1 are The convolution kernel in the convolution layer is set to 3×3, the number is 32, the step size is 1, and the surrounding padding is 1; the second-level encoding submodule En2 includes a maximum pooling layer and a second convolution module Conv2 connected in sequence, wherein the convolution kernels in the first convolution layer and the second convolution layer in Conv2 are set to 3×3, the number is 64, the step size is 1, and the surrounding padding is 1; the third-level encoding submodule En3 includes a maximum pooling layer and a third convolution module Conv3 connected in sequence, wherein the convolution kernels in the first convolution layer and the second convolution layer in Conv3 are set to 3×3, the number is 128, the step size is 1, and the surrounding padding is 1; the fourth-level encoding submodule En4 includes a maximum pooling layer and a fourth convolution module Conv4 connected in sequence, wherein the convolution kernels in the first convolution layer and the second convolution layer in Conv4 are set to 3×3, the number is 256, the step size is 1, and the surrounding padding is 1.

[0039] Decoder It includes 4 levels of decoding submodules De connected in sequence j, wherein the first-level decoding submodule De1 includes a first deconvolution layer and a fifth-level convolution module Conv5 connected in sequence, wherein the convolution kernel of the first deconvolution layer is set to 2×2, the number is 128, the step size is 2, and the surrounding padding is 0, and the convolution kernels in the first convolution layer and the second convolution layer in Conv5 are set to 3×3, the number is 128, the step size is 1, and the surrounding padding is 1; the second-level decoding submodule De2 includes a second deconvolution layer and a sixth-level convolution module Conv6 connected in sequence, wherein the convolution kernel of the second deconvolution layer is set to 2×2, the number is 64, the step size is 2, and the surrounding padding is 0, and the convolution kernels in the first convolution layer and the second convolution layer in Conv6 are set to 3×3, the number is 64, the step size is 1, and the surrounding padding is 1. is 1; the third-level decoding submodule De3 includes a third deconvolution layer and a seventh-level convolution module Conv7 connected in sequence, wherein the convolution kernel of the third deconvolution layer is set to 2×2, the number is 32, the step size is 2, and the surrounding padding is 0, the convolution kernels in the first convolution layer and the second convolution layer in Conv7 are set to 3×3, the number is 32, the step size is 1, and the surrounding padding is 1; the fourth-level decoding submodule De4 includes a fourth deconvolution layer and an eighth-level convolution module Conv8 connected in sequence, wherein the convolution kernel of the fourth deconvolution layer is set to 2×2, the number is 16, the step size is 2, and the surrounding padding is 0, the convolution kernels in the first convolution layer and the second convolution layer in Conv8 are set to 3×3, the number is 16, the step size is 1, and the surrounding padding is 1.

[0040] The output layer Out includes an output convolution layer and a softmax layer connected sequentially, where the convolution kernel in the output convolution layer is set to 3×3, the number is 2, the stride is 1, and the surrounding padding is 1.

[0041] Step 2b) Define ξ in the medical image segmentation network model O s and t The loss function is Loss:

[0042]

[0043] Where L sup , L unsup They represent the supervised loss function and the unsupervised loss function of cross-domain collaborative learning, respectively, and λ ramp Indicates L unsup The weight coefficient, R represents the maximum number of iterations, and r represents the number of iterations. ramp =0.1.

[0044] The supervised loss is calculated using a hybrid loss function consisting of dice loss and cross entropy loss. sup The calculation formula is as follows:

[0045]

[0046] Among them, pr t,l ,pr t,u ,pr s→t Represents the current B pr smix x s→t 、x t,l 、x t,u Segment the student model ξ through the target domain .t The predicted probability map obtained, pr s ,pr t→s,l ,pr t→s,u Represents the current B pr smix x s 、x t→s,l 、x t→s,u Segment the student model ξ through the source domain s The predicted probability map obtained, y s ,y t Represents the current B pr tmix x s 、x t,l The segmentation label, L dice (·) represents the dice loss function, L ce (·) represents the cross entropy loss function.

[0047] In order to enable the model to take into account regional context information and fully mine the information of a large amount of unlabeled target domain data to enhance the consistency between the source domain and the target domain, an unsupervised loss L for cross-domain collaborative learning is proposed. unsup The calculation formula is as follows:

[0048] L unsup =L mse +λ asc L ASC1 +λ rel L rel

[0049] Among them, L mse represents the cross-domain mean square error consistency loss, L ASC1 represents the cross-domain contrast loss, L rel represents the pseudo-label relearning loss, λ asc , rel L ASC1 and L rel In this embodiment, λ asc =0.5,λ rel =0.5.

[0050] Specifically, for the output feature maps of the source domain segmentation student network and the output feature maps of the target domain segmentation student network, the mean square error is used as the consistency loss to help the collaborative learning between the two segmentation networks.

[0051]

[0052] However, the mean square error can usually only calculate the predicted similarity between the paired source domain image and the target domain image segmentation results, but cannot focus on the similarity between the unpaired source domain image and the target domain image segmentation results. This makes the network prone to overfitting, causing the network to only focus on the easy-to-learn inter-domain similarities, while ignoring the deeper complementary information between the unpaired source domain and the target domain. Secondly, the mean square error value only focuses on the Euclidean distance metric between each pixel in the two prediction results, but cannot focus on the contextual information of the edges and regions of the predicted targets, which is a disadvantage for image segmentation tasks that need to focus on the edges of the predicted targets. For this reason, for deeper collaborative learning, the present invention designs a cross-domain contrastive learning loss. Specifically, for a training batch with a number of B u The unlabeled target domain training set images x t,u , and its corresponding quantity is B u The unlabeled domain adaptation image x t→s,u , the two images before and after domain adaptation are regarded as positive sample pairs, and a total of B u positive sample pairs, defined as the set X + , the remaining 2 in the batch (B u -1) data are regarded as negative samples, defined as set X - The definition of the positive and negative sample sets in a batch is as follows:

[0053]

[0054] The predicted probability graph set P of the domain segmentation student network corresponding to the positive samples in one batch + The set of predicted probability maps of the domain segmentation student network corresponding to negative samples is defined as follows:

[0055]

[0056] The present invention adopts dice similarity as the metric for contrastive learning, replacing the traditional cosine similarity. The designed cross-domain contrast loss function can make the network more focused on learning pixel-level information, making the algorithm more adaptable to the needs of image segmentation tasks. ASC1 The calculation formula is as follows:

[0057]

[0058] In order to further increase the reliability of the two teacher models, specifically, the present invention adds a 2D dropout layer in each convolutional module of the segmentation network to the input unlabeled sample x t,u or x t→s,u Gaussian noise is added to obtain multiple different prediction probability maps output after multiple forward propagations of the two teacher models. The average value of multiple prediction probability map points output by the teacher model of the domain with higher accuracy is selected through the accuracy of the validation set in the student model to obtain a reliable pseudo-label P, so that the more reliable pseudo-label P can be used to guide the student model with poor performance to re-learn. Pseudo-label re-learning loss L rel The teacher pseudo-label contrast loss L ASC2 and teacher pseudo-label supervision loss L sup2 The calculation formula is as follows:

[0059] L rel =L ASC2 +L sup2

[0060]

[0061] in, The target domain validation set The tags in Is the source domain verification set .

[0062] {f q,r (x t→s,u ,θ sr );1≤q≤Q} represents the sr The model with parameters θ s For x in the unlabeled sample in one iteration t→s,u The set of probability graphs obtained by performing Q calculations, {f q,r (x t,u ,θ tr );1≤q≤Q} represents the tr The model with parameters θ t For x in the unlabeled sample in one iteration t,u The set of probability graphs obtained by performing Q calculations, Indicated by ξ sr The model with parameter ξ s right The calculated probability map is Indicated by ξ tr The model with parameter ξ t right The calculated probability map, P + Represents X +The set of predicted probability maps corresponding to the samples in P - Represents X - A set of predicted probability maps corresponding to the samples in . In this embodiment, λ asc =0.5,λ rel =0.5, H=W=256, Q=4, B=16, B u =2.

[0063] Step 3) Iteratively train the medical image segmentation network model:

[0064] Step 3a) Initialize the number of iterations to r, the maximum number of iterations to R, R>10000, the target domain segmentation student model ξ in the current medical image segmentation network model O .t , target domain segmentation teacher model θ t And the source domain segmentation student model ξ s , source domain segmentation teacher model θ s The weight parameters are ξ tr ,θ tr and sr ,θ sr , and let r = 1. In this embodiment, R = 30000;

[0065] Step 3b) Take B samples in the source domain mixed training sample set and the target domain mixed training sample set as a batch as ξ in the medical image segmentation network model O s and .t The input is forward propagated to obtain the r-th iteration source domain segmentation student model ξ .t and the target domain segmentation student model ξ s The predicted probability map pr for each source domain mixed training sample and each target domain mixed training sample smix and pr tmix . By pr smix and pr tmix The segmentation model O is calculated using the loss function formula in step 2a). r The current loss value d r , where B≥8. In this embodiment, B=8.

[0066] Step 3c) According to d r For the weight parameter ξ tr ,θ tr and sr ,θ sr Update and get O r , the update formulas are:

[0067]

[0068]

[0069] in Denote the weight parameter ξ tr , sr The updated results, Respectively represent θ tr ,θ sr The updated result of α r represents the learning rate, α s , α t Respectively represent θ sr ,θ tr The update ratio.

[0070] Step 3f) Determine whether r = R. If so, obtain the trained semi-supervised medical image segmentation network model O*. Otherwise, set r = r + 1, O r =0 and execute step (3b);

[0071] Step 4) Obtain the segmentation results of the medical image:

[0072] The target domain test sample set T tla As the input of the trained medical image segmentation network model O*, the target domain segmentation student network Perform forward reasoning on each target domain test sample to obtain the segmentation result map of each test sample.

Claims

1. A medical image segmentation method based on semi-supervised domain adaptation and cross-domain collaborative learning, characterized in that: The steps include: (1) Obtain a mixed training sample set, a validation sample set, and a test sample set from the source domain and the target domain: For a Z containing multiple pixel categories and with labels s 3D source domain medical image and Z t The unlabeled target domain 3D medical images are preprocessed respectively, and the source domain mixed training sample set and the target domain mixed training sample set based on domain adaptation, as well as the verification sample set and the test sample set are obtained through the preprocessed 2D medical images. s ≥150, Z t ≥50; (2) Construct a medical image segmentation network model based on semi-supervised domain adaptation and cross-domain collaborative learning: Constructing a target domain segmentation student model including parallel arrangement t , target domain segmentation teacher model θ t , source domain segmentation student model ξ s and the source domain segmentation teacher model θ s , and s and t The cross-domain collaborative loss function is the medical image segmentation network model O of Loss, where: L unsup =L mse +λ asc L ASC1 +λ rel L rel Where L sup , L unsup They represent the supervised loss function and the unsupervised loss function of cross-domain collaborative learning, respectively, ramp Indicates L unsup The weight coefficient, R represents the maximum number of iterations, r represents the number of iterations, L mse represents the cross-domain mean square error consistency loss, L ASC1 represents the cross-domain contrast loss, L rel represents the pseudo-label relearning loss, λ asc and λ rel Respectively represent L ASC1 and L rel The weight coefficient of (3) Iterative training of the medical image segmentation network model: Iteratively train the medical image segmentation network model O through the mixed training sample set and verification sample set of the source domain and the target domain to obtain the trained medical image segmentation network model O*; (4) Obtain the segmentation results of medical images: The target domain test sample set T tla As the input of the trained medical image segmentation network model O*, the target domain segmentation student network ξ t * Perform forward reasoning on each target domain test sample to obtain the segmentation result map of each test sample.

2. The method according to claim 1, characterized in that The pair Z described in step (1) includes multiple pixel categories and is labeled s 3D source domain medical image and Z t The unlabeled target domain 3D medical images are preprocessed separately, and the implementation steps are as follows: (1a) Each labeled 3D source domain medical image and each unlabeled 3D target domain medical image are cut along the depth direction to obtain K s 2D source domain medical image slices and K t A 2D target domain medical image slice; (1b) Each 2D source domain medical image slice and each 2D target domain medical image slice are center-cropped and the center-cropped results are normalized to obtain the preprocessed K s 2D source domain medical image slices and K t A 2D target domain medical image slice.

3. The method according to claim 2, characterized in that The steps of obtaining the mixed training sample set, verification sample set and test sample set of the source domain and the target domain through the preprocessed 2D medical image in step (1) are as follows: The pixel points in no more than half of the target domain medical images after preprocessing are labeled, and then half of the target domain medical images with labels after preprocessing are used as a target domain verification sample set, more than half of the unlabeled target domain medical images are used as a target domain test sample set, and the remaining unlabeled target domain medical images and the remaining labeled target domain medical images are used as a target domain training sample set; then no more than half of the labeled source domain medical images after preprocessing are used as a source domain verification sample set, and the remaining part is used as a source domain training sample set, and finally the target domain training sample set and the source domain training sample set are domain adaptively processed, and the source domain training sample set and the target domain training sample set processed by domain adaptively are combined into a source domain mixed training sample set containing Y samples, and the target domain training sample set and the source domain training sample set processed by domain adaptively are combined into a target domain mixed training sample set containing Y samples.

4. The method according to claim 3, characterized in that The source domain medical image and the target domain medical image mentioned in step (1) refer to a labeled 3D medical image of a certain lesion and a certain modality, an unlabeled 3D medical image of the same lesion and a different modality, and a different lesion and a different modality, respectively.

5. The method according to claim 4, characterized in that The target domain training sample set and the source domain training sample set described in step (1) are subjected to domain adaptation, wherein for each source domain training sample x s , each unlabeled target domain training sample x t,l And each labeled target domain training sample x t,u The formulas for domain adaptation are: Among them, x s→t 、x t,l→s 、x t,u→s Respectively represent x s 、x t,l 、x t,u Domain adaptive image, λ represents the control binary mask M λ The size parameter, λ∈(0,1), represents the indicator function, h∈(0,H], w∈(0,W], H, W represent the number of pixels in the vertical and horizontal directions of the 2D medical image, respectively, and F -1 (·) represents the inverse Fourier transform, 1 H,W represents a matrix of dimension H×W with all elements set to 1. represents the Hadamard product, r stl Represents x s With x t,l Grayscale matching image after grayscale histogram matching, F A (r stl ) indicates the stl After fast Fourier transform, stl The amplitude spectrum, F A (x s ) and F P (x s ) represent the s After fast Fourier transform x s The amplitude spectrum and phase spectrum of r tls Represents x t,l With x s Grayscale matching image after grayscale histogram matching, F A (r tls ) indicates the tls After fast Fourier transform, tls The amplitude spectrum, F A (x t,l ) and F P (x t,l ) represent the t,l After fast Fourier transform x t,l The amplitude spectrum and phase spectrum of r tus Represents x t,u With x s Grayscale matching image after grayscale histogram matching, F A (r tus ) indicates the tus After fast Fourier transform, tus The amplitude spectrum of F A (x t,u ) and F P (x t,u ) represent the t,u After fast Fourier transform x t,u Amplitude spectrum and phase spectrum.

6. The method according to claim 5, characterized in that The medical image segmentation network model O described in step (2), where ξ t ,θ t , s and θ s , both include cascaded encoders Decoder and output module Out; encoder Includes cascaded convolution modules and pooling layers; decoder It includes cascaded convolution modules and deconvolution; the output module Out includes stacked convolution layers and softmax layers.

7. The method according to claim 6, characterized in that The iterative training of the medical image segmentation network model described in step (3) is implemented as follows: (3a) Initialize the number of iterations to r, the maximum number of iterations to R, R>10000, the current target domain segmentation student model ξ t , target domain segmentation teacher model θ t , source domain segmentation student model ξ s and the domain segmentation teacher model θ s The weight parameters are ξ tr ,θ tr , sr and θ sr , and let r = 1; (3b) Randomly extract B samples from the source domain mixed training sample set and the target domain mixed training sample set as ξ in the medical image segmentation network model O. s and .t The input is forward propagated to obtain the r-th iteration source domain segmentation student model ξ .t and the target domain segmentation student model ξ s The predicted probability map pr for each source domain mixed training sample and each target domain mixed training sample smix and pr tmix , where B≥8; (3c) The true labels and pr of each source domain mixed training sample and each target domain mixed training sample smix ,pr tmix Calculate ξ s and t The loss Loss, and the weight parameter ξ through Loss tr ,θ tr , sr and θ sr Update to obtain the medical image segmentation model O of this iteration r ; (3d) Determine whether r = R. If so, obtain the trained medical image segmentation network model O*. Otherwise, set r = r + 1, O r =O, and execute step (3b).

8. The method according to claim 7, characterized in that The ξ described in step (3c) s and t The loss Loss, which contains the supervised loss L sup , cross-domain mean square error consistency loss L mse , cross-domain contrast loss L ASC1 , pseudo-label relearning loss L rel , the calculation formulas are: L rel =L ASC2 +L sup2 Among them, pr t,l ,pr t,u ,pr s→t Represents the current B pr tmix lix s→t 、x t,l 、x t,u Segment the student model ξ through the target domain t The predicted probability map obtained, pr s ,pr t→s,l ,pr t→s,u Represents the current B pr smix x s 、x t→s,l 、x t→s,u Segment the student model ξ through the source domain s The predicted probability map obtained, y s ,y t Represents the current B pr tmix lix s 、x t,l The segmentation label, B u represents the total number of unlabeled samples in each iteration, L dice (·) represents the dice loss function, L ce (·) represents the cross entropy loss function, L ASC2 represents the teacher pseudo-label contrast loss, L sup2 represents the teacher pseudo-label supervision loss, P + represents a set of predicted probability maps corresponding to paired samples in each iteration of unlabeled samples, P - Represents a collection of predicted probability maps corresponding to unpaired samples in unlabeled samples for each iteration.

9. The method according to claim 7, characterized in that The weight parameter ξ described in step (3c) tr ,θ tr , sr and θ sr To update, the update formulas are: in Denote the weight parameter ξ tr , sr The updated result, d r represents the loss value calculated at the rth iteration, Respectively represent θ tr ,θ sr The update result, α r represents the learning rate, α s , α t Respectively represent θ sr and θ tr The update ratio.

Images