Unsupervised domain adaptation medical semantic segmentation method based on style extraction and alignment

Abstract

The application discloses an unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment, comprising the following steps: step 1, obtaining a medical image dataset; step 2, constructing a style extraction and alignment network model; step 3, training the style extraction and alignment network model; step 4, reading the trained feature encoder S, classifier C1 and classifier C2 in the style extraction and alignment network model to constitute a generator G, inputting a single medical image I into the generator G for forward inference, and obtaining a semantic segmentation result P of the medical image I. The method does not require additional parameters and labels, can effectively generalize the segmentation capability of the semantic segmentation model, and thus overcomes the problems of high labeling cost and insufficient generalization capability of the existing medical semantic segmentation processing method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and particularly relates to an unsupervised domain adaptation medical semantic segmentation method based on style extraction and alignment. BACKGROUND

[0002] The target of medical image segmentation is to accurately extract the region of interest in the medical image. However, due to the special nature of the medical image, such as complex anatomical structure and limited data, the traditional image segmentation method is difficult to achieve ideal performance in this field. Therefore, in recent years, the deep learning method based on unsupervised domain adaptation has attracted widespread attention in the field of medical image segmentation.

[0003] To solve the problem of unsupervised domain adaptation for medical image segmentation, many solutions have been proposed by the academic and industrial communities. For example, Q. Dou et al. (Q. Dou, C. Ouyang, C. Chen, H. Chen, and P.-A. Heng, “Unsupervised cross-modality domain adaptation of convnets for biomedical image segmentations with adversarial loss,” arXiv preprint arXiv:1804.10916, 2018) use adversarial learning to reduce the distribution difference between the source domain and the target domain. This method mainly aligns the global marginal distribution of visual representation without considering the consistency of semantics, which means that semantic confusion may occur during the alignment process. For another example, Y. Li et al. (Y. Li, L. Yuan, and N. Vasconcelos, “Bidirectional learning for domain adaptation of semantic segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 6936-6945) propose a method of generating pseudo labels, which can provide stronger supervision information for the target domain. However, if error propagation and label bias occur during the generation of pseudo labels, it will have a certain impact on the results. For another example, Q. Zhang et al. (Q. Zhang, J. Zhang, W. Liu, and D. Tao, “Category anchor-guided unsupervised domain adaptation for semantic segmentation,” in Advances in Neural Information Processing Systems, 2019, pp. 435-445) propose a novel class anchor-guided (CAG) UDA model for semantic segmentation, which introduces an anchor-based pixel-level distance loss and a discrimination loss to ensure that the model can better align the domain differences between different classes during the learning process.

[0004] Therefore, it is urgent to develop a new unsupervised domain adaptive medical image segmentation method to overcome the problems of high labeling cost, semantic confusion and insufficient generalization ability of the existing medical semantic segmentation processing method.

[0005] Therefore, it is urgent to develop a new unsupervised domain adaptive medical image segmentation method to overcome the problems of high labeling cost, semantic confusion and insufficient generalization ability of the existing medical semantic segmentation processing method. SUMMARY

[0006] The present application provides an unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment, which does not require additional parameters and labels and can effectively improve the segmentation ability of the semantic segmentation model, thereby overcoming the problems of high labeling cost, semantic confusion and insufficient generalization ability of the existing medical semantic segmentation processing method.

[0007] To solve the above technical problems, the technical scheme of the present application is as follows:

[0008] An unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment, comprising the following steps:

[0009] Step 1, obtaining a medical image dataset;

[0010] Step 2, constructing a style extraction and alignment network model,

[0011] The extraction and alignment network model comprises an image segmentation module, a style alignment module and a segmentation extraction module,

[0012] The image segmentation module comprises a feature encoder S, a classifier C1 and a discriminator D; the style alignment module comprises two discriminators D Alig_S and D Alig_T ; the style extraction module comprises a discriminator D Sty and a classifier C2;

[0013] A single medical image I in the medical image dataset is input into the feature encoder S, and the output is a corresponding feature map x, which is a source domain image or a target domain image;

[0014] The extracted feature map x is input into a classifier C1, and the output is a semantic segmentation result P of the image I n ;

[0015] The semantic segmentation result P n is input into a discriminator D, and the output is a judgment of the source of the input image. If the input is a source domain image feature, the output is Source; if the input is a target domain image feature, the output is Target.

[0016] The source domain image is divided by channel average, and the obtained source domain semantic feature subgraph x S Seg and the source domain style feature subgraph x S Sty The target domain image is divided by channel average, and the obtained target domain semantic feature subgraph x T Seg and the target domain style feature subgraph x T Sty ;

[0017] The source domain semantic feature subgraph x S Seg and the target domain style feature subgraph x T Sty are reconstructed to obtain a reconstructed feature map f1.

[0018] The reconstructed feature map f1 is input into the discriminator D Alig_S , and the output is a judgment of whether the input is a reconstructed image. If yes, the output is true; otherwise, the output is false. f1 satisfies formula (1):

[0019]

[0020] The target domain semantic feature subgraph x T Seg and the source domain style feature subgraph x S Sty are reconstructed to obtain a reconstructed feature map f2.

[0021] The reconstructed feature map f2 is input into the discriminator D Alig_T , and the output is a judgment of whether the input is a reconstructed image. If yes, the output is true; otherwise, the output is false. f2 satisfies formula (2):

[0022]

[0023] The source domain style feature subgraph x S Sty and the target domain style feature subgraph x T Sty are input into the discriminator D Sty, the output determines the source of the input image. If the input is the source domain image feature, the output is Source; otherwise, the output is Target;

[0024] The reconstructed feature map f1 is input into the classifier C2, and the output is the semantic segmentation result P of the image I. s ;

[0025] Step 3: Train the style extraction and alignment network model:

[0026] Step 3.1, read the feature encoder S, discriminator D, and discriminator D from the local file Alig_S , Discriminator D Alig_T , Discriminator D Sty The pre-training parameters of and initialize the classifiers C1 and C2;

[0027] Step 3.2: Get the known source domain dataset and target domain dataset, and randomly read n samples from the source domain dataset and target domain dataset as training samples S = {s i |i=1,2,3,…,n,s i =(I i S ,I i T ,l i S )},I i S and I i T Represents the i-th pair of training samples s i The source domain image and target domain image in l i S is the source domain image I i S Semantic segmentation labels of

[0028] Step 3.3: Set the training sample s i Input the style extraction and alignment network models in sequence for model training;

[0029] Step 4: Read the trained feature encoder S, classifier C1, and classifier C2 in the style extraction and alignment network model to form a generator G, read in a single medical image I, and input the single medical image I into the generator G for forward reasoning to obtain the semantic segmentation result P of the medical image I.

[0030] Preferably, the specific method of step 3.3 is:

[0031] Step 3.3.1, sample s i Input into the feature encoder S for feature extraction to obtain the source domain feature map x S ∈RC ×H×W and target domain feature map x T ∈R C×H×W ; wherein C, H, W are respectively the channel number, height and width of the feature map;

[0032] Step 3.3.2, split the feature map x S into 2 feature sub-maps x S Seg and x S Sty , wherein x S Seg ∈R C / 2×H×W is the source domain semantic feature map, x S Sty ∈R C / 2×H×W is the source domain style feature map, and satisfies formula (3); split the feature map x T into 2 feature sub-maps x T Seg and x T Sty , wherein x T Seg ∈R C / 2×H×W is the target domain semantic feature map, x T Sty ∈R C / 2×H×W is the target domain style feature map, and satisfies formula (3):

[0033] x = concat(x Seg , x Sty ) (3)

[0034] Step 3.3.3, reconstruct the feature maps x S Seg , x S Sty , x T Seg and x T Sty according to formula (1) and (2);

[0035] Step 3.3.4, input the feature maps x S and f1 into the discriminator D Alig_S in turn, and calculate the loss function L adv Alig_S and L D Alig_S according to formula (4) and (5):

[0036]

[0037]

[0038] where p1 Alig_S and p S Alig_S respectively represent f1and x S The prediction result obtained by the discriminator D Alig_S is L B (·) represents a binary cross-entropy loss function.

[0039] Step 3.3.5, the feature map f2and x T are input into the discriminator D Alig_T in turn, and the loss function L adv Alig_T and L D Alig_T are calculated according to formulas (6) and (7).

[0040]

[0041]

[0042] where p2 Alig_T and p T Alig_T respectively represent f2and x T The prediction result obtained by the discriminator D Alig_T is L

[0043] Step 3.3.6, the feature map x S Sty and x T Sty are input into the discriminator D Sty in turn, and the loss function L adv Sty and L D Sty are calculated according to formulas (8) and (9).

[0044]

[0045]

[0046] where p S Sty and p T Sty respectively represent the feature map x S Sty and x T Sty The prediction result obtained by the discriminator D Sty is L

[0047] Step 3.3.7, f1is input into the classifier C2, and the segmentation loss function Lseg :

[0048] L seg = L CE (l S , P s ) + L Dice (l S , P s ) (10)

[0049] wherein, l S is a semantic segmentation label, L CE (·) and L Dice (·) are cross-entropy loss function and Dice loss function respectively;

[0050] Step 3.3.8, input x S and x T into the classifier C1 respectively, and input the output results into the discriminator D respectively, and calculate the loss functions L seg n and L D n :

[0051]

[0052]

[0053] wherein, a and b are hyperparameters given in advance;

[0054] Step 3.3.9, calculate the loss function L G of the generator G according to formula (13), wherein G is composed of the feature encoder S, the classifier C1 and the classifier C2:

[0055]

[0056] Step 3.3.10, the generator G, the discriminator D Alig_S , the discriminator D Alig_T , the discriminator D Sty , the discriminator D respectively perform the back propagation of the network according to the loss functions L G , L D Alig_S , L D Alig_T , L D Sty and L D n , and update the model parameters.

[0057] As preferred, during the training of the style extraction and alignment network model, steps 3.2-3.3 are repeated until iter iterations are completed:

[0058]

[0059] wherein epoch is a pre-given training round number.

[0060] The present application has the following characteristics and beneficial effects:

[0061] 1、The present application can divide the feature map of the to-be-tested medical image into semantic features and style features according to the channel number by adding a style extraction module and a style alignment module on the generator. This method effectively distinguishes the semantic features and style features of the image, thereby realizing the alignment of the style features between the source domain and the target domain. In this process, no additional parameters or labels are needed. In addition, this method can effectively generalize the segmentation ability of the semantic segmentation model, thereby overcoming the problems of high labeling cost and insufficient generalization ability of existing medical semantic segmentation processing methods.

[0062] 2、The present application can directly integrate into the generator without additional parameters or labels by adding a style extraction module and a style alignment module to the generator. This method has good generalization ability and can be used in different medical image segmentation tasks. This technical innovation not only reduces the complexity of medical image segmentation processing, but also improves the processing efficiency and the reliability of the model, providing an effective solution to the problems of high labeling cost, semantic confusion and insufficient generalization ability in existing medical image segmentation methods. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0064] Figure 1 The framework structure diagram of the unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment provided by the embodiment of the present application;

[0065] Figure 2 The inference process diagram of the medical image semantic segmentation model in the embodiment of the present application;

[0066] Figure 3 The source domain image and the target domain image; wherein (a) is the source domain image, and (b) is the target domain image;

[0067] Figure 4The actual segmentation result P of the unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment provided by the embodiment of the present application. DETAILED DESCRIPTION

[0068] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0069] In the description of the present application, it should be understood that the terms 'center', 'longitudinal', 'transverse', 'upper', 'lower', 'front','rear', 'left', 'right','vertical', 'horizontal', 'top', 'bottom', 'inner', 'outer' and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms 'first','second' and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with 'first','second' and the like can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise stated, the meaning of 'a plurality of' is two or more.

[0070] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms'mounting', 'connection', 'connection' should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0071] The present application provides an unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment, as shown in Figure 1 It mainly includes three parts of network structure of style extraction and style alignment, unsupervised model training and medical image semantic segmentation model inference:

[0072] The network structure of style extraction and style alignment is composed of multiple substructures, mainly including an image segmentation module, a style alignment module and a style extraction module:

[0073] Specifically, as shown in Figure 1 The image segmentation module includes a feature encoder S, a classifier C1 and a discriminator D;

[0074] Further, the feature encoder S is a pre-defined network feature extraction structure, which takes a single medical image I as input and outputs a corresponding extracted feature map x; in this example, the feature encoder S is a ResNet18;

[0075] The classifier C1 is a pre-defined artificial neural network classification structure, which takes the extracted feature map x as input and outputs a semantic segmentation result P of the image I n ; in this example, the classifier C1 is a convolutional layer with a 3x3 convolution kernel;

[0076] The discriminator D is a pre-defined classification artificial neural network, which takes P n as input and outputs a judgment of the source of the input image; if the input is a source domain image feature, the output is Source; otherwise, the output is Target; in this example, the discriminator D is a stack of three convolutional layers with a 3x3 convolution kernel;

[0077] Further, the style alignment module includes two discriminators D Alig_S and D Alig_T :

[0078] D Alig_S is a pre-defined classification artificial neural network, which takes the source domain image feature map x S or the reconstructed feature map f1 as input and outputs a judgment of whether the input is a reconstructed image, true if yes; otherwise, false; f1 satisfies equation (1):

[0079]

[0080] where x S Seg is a source domain semantic feature map obtained by dividing the source domain image by the feature encoder S and the feature map by channel average; x T Sty is a target domain style feature map obtained by dividing the target domain image by the feature encoder S and the feature map by channel average; in this example, the discriminator D Alig_S is a stack of three convolutional layers with a 3x3 convolution kernel;

[0081] D Alig_T is a pre-defined classification artificial neural network, which takes the reconstructed feature map f2 or the target domain feature map x T as input and outputs a judgment of whether the input is a reconstructed image, true if yes; otherwise, false; f2 satisfies equation (2):

[0082]

[0083] where x T Segis the semantic feature map of the target domain obtained after the target domain image is divided by the feature encoder S and the feature map is averaged by channel; x S Sty The source domain style feature map is obtained after the source domain image is divided by the feature encoder S and the feature map is averaged by channel; in this example, the discriminator D Alig_T It is composed of 3 convolution kernels stacked as 3×3 convolution layers;

[0084] Furthermore, the style extraction module includes a discriminator D Sty And classifier C2:

[0085] Discriminator D Sty It is a given classification artificial neural network whose input is the style feature map x of the source domain or target domain S Sty and x T Sty , the output determines the source of the input image. If the input is the source domain image feature, the output is Source; otherwise, the output is Target. In this example, the discriminator D Sty It is composed of 3 convolution kernels stacked as 3×3 convolution layers;

[0086] Classifier C2 is a pre-given artificial neural network classification structure, whose input is the reconstructed feature map f1 and output is the semantic segmentation result P of image I. s ; In this example, classifier C2 is a convolutional layer with a convolution kernel of 3×3;

[0087] A further configuration of this embodiment is a model training method, which mainly includes the following steps:

[0088] Step 1: Read the feature encoder S, discriminator D, and discriminator D from the local file Alig_S , Discriminator D Alig_T , Discriminator D Sty The pre-training parameters of and initialize the classifiers C1 and C2;

[0089] Step 2: Randomly read n samples from the source domain dataset and the target domain dataset as training samples S = {s i |i=1,2,3,…,n,s i =(I i S ,I i T ,l i S )},I i S and I i T Represents the i-th pair of training samples s isource domain image and the target domain image in the training set, l i S is the semantic segmentation label of the source domain image I i S .

[0090] Step 3: input the training sample s i into the network structure in sequence for model training, specifically as follows:

[0091] Step 3.1: input the sample s i into the feature encoder S for feature extraction to obtain the source domain feature map x S ∈R C ×H×W and the target domain feature map x T ∈R C×H×W ; wherein C, H and W are respectively the channel number, height and width of the feature map;

[0092] Step 3.2: average cut the feature map x S into 2 feature sub-maps x S Seg and x S Sty in the channel, wherein x S Seg ∈R C / 2×H×W is the source domain semantic feature map, x S Sty ∈R C / 2×H×W is the source domain style feature map, and satisfies formula (3); average cut the feature map x T into 2 feature sub-maps x T Seg and x T Sty in the channel, wherein x T Seg ∈R C / 2×H×W is the target domain semantic feature map, x T Sty ∈R C / 2×H×W is the target domain style feature map, and satisfies formula (3):

[0093] x=concat(x Seg ,x Sty ) (3)

[0094] Step 3.3: reconstruct the feature maps x S Seg , x S Sty , x T Seg and x T Sty according to formulas (1) and (2);

[0095] Step 3.4: the feature map x S is input into the discriminator D Alig_S in turn, and the loss function L adv Alig_S and L D Alig_S is calculated according to equations (4) and (5):

[0096]

[0097]

[0098] where p1 Alig_S and p S Alig_S respectively represent the prediction result of f1and x S obtained by the discriminator D Alig_S ; B (·) represents a binary cross-entropy loss function.

[0099] Step 3.5: the feature map f2and x T is input into the discriminator D Alig_T in turn, and the loss function L adv Alig_T and L D Alig_T is calculated according to equations (6) and (7):

[0100]

[0101]

[0102] where p2 Alig_T and p T Alig_T respectively represent the prediction result of f2and x T obtained by the discriminator D Alig_T ;

[0103] Step 3.6: the feature map x S Sty and x T Sty is input into the discriminator D Sty in turn, and the loss function L adv Sty and L D Sty is calculated according to equations (8) and (9):

[0104]

[0105]

[0106] where p S Sty and p T Sty respectively represent the feature map x S Sty and x T Sty The prediction result obtained by the discriminator D Sty ;

[0107] Step 3.7: input f1 into the classifier C2, and calculate the segmentation loss function L seg according to formula (10):

[0108] L seg = L CE (l S , P s ) + L Dice (l S , P s ) (10)

[0109] where l S is the semantic segmentation label, L CE (·) and L Dice (·) are cross-entropy loss function and Dice loss function respectively;

[0110] Step 3.8: input x S and x T into the classifier C1 respectively, and input the output results into the discriminator D respectively, and calculate the loss functions L seg n and L D n according to formula (11) and (12):

[0111]

[0112]

[0113] Step 3.9: calculate the loss function L G of the generator G according to formula (13), where G is composed of the feature encoder S, the classifier C1 and the classifier C2:

[0114]

[0115] where α and β are hyperparameters given in advance; in this example, α = 0.03 and β = 0.01;

[0116] Step 3.10: the generator G, the discriminator D Alig_S , the discriminator D Alig_T , the discriminator DSty , the discriminator D respectively according to the loss function L G , L D Alig_S , L D Alig_T , L D Sty and L D n performing back propagation of the network, updating the model parameters;

[0117] Step 4: repeat steps 2 to 3 until the completion of iter iterations:

[0118]

[0119] wherein epoch is a pre-given training round number; in the present example, epoch = 5000;

[0120] Further settings of the present embodiment, medical image semantic segmentation model inference, as shown in Figure 2 , includes the following steps:

[0121] Step 1: reading the network weight file and network configuration file of the trained generator G;

[0122] Step 2: reading a single medical image I, as shown in Figure 3 ; inputting I into the generator G for forward inference to obtain the semantic segmentation result P of the medical image I, and the segmentation result is as shown in Figure 4 .

[0123] The embodiments of the present application are described in detail above in combination with the drawings, but the present application is not limited to the described embodiments. For those skilled in the art, various changes, modifications, replacements and variations of these embodiments including components without departing from the principles and spirits of the present application still fall within the protection scope of the present application.

Claims

1. An unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment, characterized by: The steps include: Step 1: Obtain a medical image dataset; Step 2: Build a style extraction and alignment network model. The extraction and alignment network model includes an image segmentation module, a style alignment module and a style extraction module. The image segmentation module includes a feature encoder S, a classifier C1 and a discriminator D; the style alignment module includes two discriminators D Alig_S and D Alig_T ; The style extraction module includes a discriminator D Sty and classifier C2; A single medical image I in the medical image dataset is input into a feature encoder S, and the output is a corresponding extracted feature map x, where the feature map x belongs to a source domain image or a target domain image; The extracted feature map x is input into the classifier C1 to output the semantic segmentation result P of the image I. n ; The semantic segmentation result P n Input discriminator D, output to judge the source of the input image. If the input is the source domain image feature, the output is Source; if the input is the target domain image feature, the output is Target. The source domain image is divided into two equal parts by channel, and the resulting source domain semantic feature subgraph x is S Seg and the source domain style feature subgraph x S Sty ; After the target domain image is divided by channel average, the target domain semantic feature subgraph x is obtained T Seg and the target domain style feature subgraph x T Sty ; The source domain semantic feature subgraph x S Seg and the target domain style feature subgraph x T Sty Reconstruction is performed to obtain the reconstructed feature map f1; The reconstructed feature map f1 is input into the discriminator D Alig_S , the output determines whether the input is the reconstructed graph, if it is true, otherwise it is false; f1 satisfies formula (1): The reconstructed feature map f1 is input into the classifier C2, and the output is the semantic segmentation result P of the image I. s ; Step 3: Train the style extraction and alignment network model: Step 3.1, read the feature encoder S, discriminator D, and discriminator D from the local file Alig_S , Discriminator D Alig_T , Discriminator D Sty The pre-training parameters of and initialize the classifiers C1 and C2; Step 3.2: Get the known source domain dataset and target domain dataset, and randomly read n samples from the source domain dataset and target domain dataset as training samples S = {s i |i=1,2,3,…,n,s i =(I i S ,I i T ,l i S )},I i S and I i T Represents the i-th pair of training samples s i The source domain image and target domain image in l i S is the source domain image I i S Semantic segmentation labels; Step 3.3: Set the training sample s i Input the style extraction and alignment network models in sequence for model training; Step 4: Read the trained feature encoder S, classifier C1, and classifier C2 in the style extraction and alignment network model to form a generator G, read in a single medical image I, and input the single medical image I into the generator G for forward reasoning to obtain the semantic segmentation result P of the medical image I.

2. The unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment according to claim 1 is characterized in that In step 2, the target domain semantic feature subgraph x T Seg and the source domain style feature subgraph x S Sty Reconstruction is performed to obtain the reconstructed feature map f2; The reconstructed feature map f2 is input to the discriminator D Alig_T , the output determines whether the input is the reconstructed graph, if so, it is true; otherwise, it is false; f2 satisfies formula (2): The source domain style feature subgraph x S Sty and the target domain style feature subgraph x T Sty Input discriminator D Sty , the output determines the source of the input image. If the input is the source domain image feature, the output is Source; Otherwise, the output is Target.

3. The unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment according to claim 2 is characterized in that The specific method of step 3.3 is: Step 3.3.1, sample s i Input into the feature encoder S for feature extraction to obtain the source domain feature map x S ∈R C×H×W and target domain feature map x T ∈R C×H×W ; Where C, H, and W are the number of channels, height, and width of the feature map respectively; Step 3.3.2, the feature map x S Divide the channel into 2 feature subgraphs x on average S Seg and x S Sty , where x S Seg ∈R C / 2×H×W is the source domain semantic feature map, x S Sty ∈R C / 2×H×W is the source domain style feature map, and satisfies formula (3); the feature map x T Divide the channel into 2 feature subgraphs x on average T Seg and x T Sty , where x T Seg ∈R C / 2×H×W is the semantic feature map of the target domain, x T Sty ∈R C / 2×H×W is the target domain style feature map, and satisfies formula (3): x=concat(x Seg ,x Sty ) (3) Step 3.3.3, the feature map x S Seg 、x S Sty 、x T Seg and x T Sty Reconstruct according to formula (1) and (2); Step 3.3.4, the feature map x S and f1 are input to the discriminator D in sequence Alig_S , and calculate the loss function L according to formulas (4) and (5) adv Alig_S and L D Alig_S : Among them, p1 Alig_S and p S Alig_S Represent f1 and x respectively S Through the discriminator D Alig_S The prediction result obtained by inference; L B (·) represents the binary cross entropy loss function; Step 3.3.5, feature map f2 and x T Input to the discriminator D in sequence Alig_T , and calculate the loss function L according to formulas (6) and (7) adv Alig_T and L D Alig_T : Among them, p2 Alig_T and p T Alig_T Represent f2 and x respectively T Through the discriminator D Alig_T The prediction results obtained by inference; Step 3.3.6, the feature map x S Sty and x T Sty Input the discriminator D in sequence Sty And calculate the loss function L according to formulas (8) and (9) adv Sty and L D Sty : Among them, p S Sty and p T Sty Represents the feature map x S Sty and x T Sty Through the discriminator D Sty The prediction results obtained by inference; Step 3.3.7: Input f1 into the classifier C2 and calculate the segmentation loss function L according to formula (10): seg : L seg =L CE (L S ,P s )+L Dice (L S ,P s ) (10) Among them, l S is the semantic segmentation label, L CE (·) and L Dice (·) are cross entropy loss function and Dice loss function respectively; Step 3.3.8, x S and x T Input them into the classifier C1 respectively, and input the output results into the discriminator D respectively, and calculate the loss function L according to formulas (11) and (12): seg n and L D n : Among them, α and β are pre-given hyperparameters; Step 3.3.

9. Calculate the loss function L of the generator G according to formula (13) G , G consists of feature encoder S, classifier C1 and classifier C2: Step 3.3.10, Generator G, Discriminator D Alig_S , Discriminator D Alig_T , Discriminator D Sty , the discriminator D is based on the loss function L G , L D Alig_S , L D Alig_T , L D Sty and L D n Perform back propagation of the network and update the model parameters.

4. The unsupervised domain adaptive medical semantic segmentation method based on style extraction and alignment according to claim 1 is characterized in that When training the style extraction and alignment network model, repeat steps 3.2-3.3 until iter iterations are completed: epoch is the number of training rounds given in advance.

Images