Cross-disease medical image segmentation method and system based on transfer learning

Abstract

The invention belongs to the technical field of medical image processing, and particularly relates to a cross-disease medical image segmentation method and system based on transfer learning, and the method comprises the steps: obtaining a source domain image and a target domain image, and obtaining a source domain style image and a target domain style image through image mapping; obtaining a teacher-student model integrated with specific domain batch normalization, and preliminarily constructing a medical image segmentation model based on the teacher-student model integrated with specific domain batch normalization; inputting the source domain image, the target domain image, the source domain style image and the target domain style image into a medical image segmentation model for iterative updating, and constructing a cross-disease medical image segmentation model based on transfer learning; and inputting the target domain image into the cross-disease medical image segmentation model based on transfer learning to obtain a sample label. According to the method, knowledge migration is adopted to realize unsupervised segmentation of cross-disease data from a source domain to a target domain, and the problem of model performance reduction caused by cross-disease image distribution difference is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a cross-disease medical image segmentation method and system based on transfer learning. Background Art

[0002] The purpose of medical image segmentation methods is to separate the lesion area from the normal tissue area in medical images, providing doctors with clearer and more accurate diagnosis basis. However, deep learning-based medical image segmentation algorithms usually require a large amount of manually annotated data to train the segmentation model, and these annotated data often need to be manually annotated by professional doctors, which is time-consuming, labor-intensive and costly. The transfer learning method uses a source domain image set with labels to train the model, and then transfers the knowledge of the source domain image to the target domain image set to achieve the segmentation task on the target domain image. This method can use labeled source domain images and unlabeled target domain images to train the model at the same time, reducing the workload of data annotation. The transfer learning method generates target-like images or makes the model focus on domain-invariant information to minimize the difference between the source domain and target domain images, so that the model adapts to the target domain image and improves the performance of the model on the target domain image set. However, the existing knowledge transfer method in medical image segmentation uses adversarial learning to transfer the style of the source domain image at the image level. The adversarial training process is relatively complex and prone to fall into local optimal solutions. In addition, most existing methods require that the source domain and target domain images come from different imaging methods of the same tissue organ or lesion, which makes it impossible for such methods to use other similar disease images or organ images to train models.

[0003] In order to introduce transfer learning into the medical image segmentation model to improve the segmentation performance of the model, the patent application number, CN115018865A, named "A medical image segmentation method based on transfer learning", discloses a medical image segmentation method based on transfer learning, which is implemented in the following steps: build a neural network for lung infection segmentation in two-dimensional CT images; establish a data set, the data set consists of a training set and a test set; first generate pseudo labels for all unlabeled images, then pre-train them, process the images to a uniform size, then set the network parameters of the neural network, and iteratively train the neural network with the training set, and finally save the trained neural network model; test the trained neural network model with all the images in the test set, and record the corresponding test index values; add Transformer to the neural network, and execute steps 3 and 4, while adaptively adjusting the network parameters of the neural network; compare the test index values ​​obtained in step 5 with the test index values ​​obtained in step 4 to evaluate the segmentation effect. Therefore, the main shortcomings of existing medical image segmentation:

[0004] 1) Most methods use fully supervised learning, which requires large-scale labeled datasets and it is difficult to obtain accurate segmentation results in unlabeled datasets.

[0005] 2) Most segmentation methods based on transfer learning use adversarial learning to transfer the style of source domain images at the image level or feature level. The training process is relatively complicated and it is easy to fall into the local optimal solution.

[0006] 3) Existing transfer learning methods require that the source domain images and the target domain images have the same case tissue structure. This requirement limits these methods from using images of other pathological tissue structures to train models. Summary of the invention

[0007] The purpose of the present invention is to provide a cross-disease medical image segmentation method and system based on transfer learning to solve the technical problem that the prior art cannot decompose unlabeled data across disease medical images.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect, the present application discloses a cross-disease medical image segmentation method based on transfer learning, comprising:

[0010] Obtain a source domain image and a target domain image, and obtain a source domain style image and a target domain style image through image mapping;

[0011] Obtain a teacher-student model integrating domain-specific batch normalization, and preliminarily build a medical image segmentation model based on the teacher-student model integrating domain-specific batch normalization;

[0012] Input the source domain image, target domain image, source domain style image, and target domain style image into the medical image segmentation model for iterative update, and build a cross-disease medical image segmentation model based on transfer learning;

[0013] The target domain image is input into the cross-disease medical image segmentation model based on transfer learning to obtain the sample label.

[0014] Preferably, the source domain style image and the target domain style image are obtained by image mapping, specifically:

[0015] Use the histogram matching algorithm to generate a matching image for the source domain image using the target domain image as a template;

[0016] The source domain image and the matching image are decomposed by Fourier transform to obtain the corresponding amplitude spectrum and phase spectrum;

[0017] According to the corresponding amplitude spectrum and phase spectrum, the source domain style image and the target domain style image are obtained.

[0018] Preferably, the acquiring of the teacher-student model integrating domain-specific batch normalization and the preliminary construction of the medical image segmentation model based on the teacher-student model integrating domain-specific batch normalization are specifically as follows:

[0019] Using U-Net as the backbone network of the teacher-student model, the BN layer in the U-Net network was replaced by the DSBN layer; a medical image segmentation model was obtained.

[0020] Preferably, the step of inputting the source domain image, the target domain image, the source domain style image and the target domain style image into the medical image segmentation model for iterative updating to construct a cross-disease medical image segmentation model based on transfer learning comprises:

[0021] The medical image segmentation model uses the bidirectional generation method of inter-domain data to generate source domain images and target domain images mutually, so that the unlabeled target domain image outputs a pseudo label after passing through the DSBN of the teacher model; the target domain image and the source domain style image are processed by the medical image segmentation model to obtain the target domain image probability map and the source domain style image probability map; the unsupervised loss is calculated based on the pseudo label, the target domain image probability map and the source domain style image probability map;

[0022] The source domain image is processed by the medical image segmentation model to obtain a source domain image probability map, and the full supervision loss is calculated based on the source domain image probability map;

[0023] The medical image segmentation model normalizes and extracts high-dimensional features from the source domain image, the target domain image, the source domain style image, and the target domain style image to obtain feature maps; the overall contrast loss is calculated based on the feature maps;

[0024] Using the loss function L = L sup +λ(t)×L unsup +ω×L ct Perform back propagation to update the weight parameters of the medical image segmentation model; obtain a cross-disease medical image segmentation model based on transfer learning;

[0025] Among them, L sup is the full supervision loss; L unsup is the loss of unsupervised training; L ct is the overall contrast loss; λ(t) is the balance parameter; ω is a hyperparameter.

[0026] Preferably, the calculation formula of the unsupervised training is as follows:

[0027]

[0028] in is the target domain probability map, is the source domain style image probability map, Ω T is the parameter of the student model when the input is the target domain data, ΩS is the parameter of the student model when the input is source domain data, B T represents the number of source domain images in each batch during the training process, represents the predicted label of the target domain image by the teacher model, Ω T′ is the parameter of the teacher model, δ is the random perturbation added to the input teacher model data; the mean square error is used to implement the consistency loss function L cons .

[0029] Preferably, the calculation formula of the full supervision loss is as follows:

[0030]

[0031] Among them, y i,c is the true label of pixel i in the source domain image, and the true label is the label of the source domain image itself; p i,c is the predicted probability of the student model for that pixel, M is the number of pixels in the image, and C is the number of categories.

[0032] Preferably, the characteristic graph comprises: h s , h s→t , h t 、h t→s and The overall contrast loss is calculated according to the feature map, specifically:

[0033] The feature maps are divided into positive and negative pairs according to whether they are similar or not: the feature maps processed by the same DSBN layer are positive pairs, and the feature maps processed by different DSBN layers are negative pairs;

[0034] The contrast loss from the source domain image to the target domain image is calculated as:

[0035]

[0036] The contrast loss from the target domain image to the source domain image is defined as:

[0037]

[0038] Calculate the following loss functions respectively; replace h in the above formula t Replace with h s→t ,get:

[0039] Contrastive loss from source domain image to target domain style image And the contrast loss from the target domain style image to the source domain image

[0040] In the above formula, h s Replace with ht→s ,get:

[0041] Contrastive loss from source domain style image to target domain image And the contrast loss from the target domain image to the source domain style image

[0042] In the above formula, h s and h t Replace with h t→s and h s→t ,get:

[0043] Contrastive loss from target domain style image to source domain style image And the contrast loss from source domain style image to target domain style image

[0044] The overall contrast loss is the average of the above contrast losses:

[0045]

[0046] Where N = {h s ,h s→t ,h t ,h t→s} is a set of negative pairs, sim(·) indicates the calculation of cosine similarity; It is the contrast loss from source domain image to target domain image; Contrastive loss from target domain images to source domain images; represents the mathematical expectation of all source domain and target domain samples; h s Represents the batch normalized features of the source domain image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; Represents the batch normalized features obtained by processing the source domain image through the Unet encoder of the student network, the target domain DSBN and the projection head H; h represents the batch normalized features of the target domain style image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; s→t represents the batch normalized features of the target domain style image processed by the Unet encoder of the student network, the target domain DSBN and the projection head H; h represents the batch normalized features of the target domain image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; t h represents the batch normalized features of the target domain image after being processed by the Unet encoder of the student network, the target domain DSBN, and the projection head H; t→s Represents the batch normalized features of the source domain style image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; represents the batch normalized features of the source domain style image processed by the Unet encoder of the student network, the target domain DSBN and the projection head H; τ is the temperature coefficient; L sup is the full supervision loss; L unsup is the loss of unsupervised training; L ct is the overall contrast loss; λ(t) is the balance parameter; ω is the hyperparameter; It is the contrast loss from the source domain image to the target domain style image; is the contrast loss from the target domain style image to the source domain image; is the contrast loss from the source domain style image to the target domain image; It is the contrast loss from the target domain image to the source domain style image; is the contrast loss from the target domain style image to the source domain style image; is the contrast loss from the source domain style image to the target domain style image.

[0047] In a second aspect, the present application discloses a cross-disease medical image segmentation system based on transfer learning, comprising:

[0048] A mapping unit, used to obtain a source domain image and a target domain image, and obtain a source domain style image and a target domain style image through image mapping;

[0049] A model building unit, used for obtaining a teacher-student model integrating domain-specific batch normalization, and preliminarily building a medical image segmentation model based on the teacher-student model integrating domain-specific batch normalization;

[0050] A model updating unit, used to input source domain images, target domain images, source domain style images, and target domain style images into the medical image segmentation model for iterative update, and to build a cross-disease medical image segmentation model based on transfer learning;

[0051] The output unit is used to input the target domain style image into the cross-disease medical image segmentation model based on transfer learning to obtain the sample label.

[0052] In a third aspect, the present application discloses an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned methods for cross-disease medical image segmentation based on transfer learning when executing the computer program.

[0053] In a fourth aspect, the present application discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of any of the above-mentioned cross-disease medical image segmentation methods based on transfer learning.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The present invention proposes a cross-disease medical image segmentation method based on transfer learning, which uses labeled medical images of source domain cases and unlabeled medical images of different cases in the target domain to train the model, and minimizes the difference between the source domain and target domain images by generating target-like images or making the model focus on domain-invariant information, thereby improving the segmentation performance of the model on the target domain image set. Existing medical image segmentation methods based on transfer learning require that the source domain and target domain images have the same tissue structure, which limits the use of other similar tissue organ images or lesion images to train the model. The present application uses knowledge transfer to achieve unsupervised segmentation of cross-disease data from the source domain to the target domain, solving the problem of decreased model performance due to differences in the distribution of cross-disease images. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 is a flow chart of the method of the present invention;

[0058] Figure 2 It is a principle flow chart of an embodiment of the present invention;

[0059] Figure 3 It is an overall framework diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0061] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0063] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0064] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0065] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0066] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0067] See also Figure 1 The present application discloses a cross-disease medical image segmentation method based on transfer learning, which is characterized by comprising:

[0068] S1: Obtain source domain image and target domain image, and obtain source domain style image and target domain style image through image mapping;

[0069] S2: Obtain a teacher-student model integrating domain-specific batch normalization, and preliminarily build a medical image segmentation model based on the teacher-student model integrating domain-specific batch normalization;

[0070] S3: Input the source domain image, target domain image, source domain style image and target domain style image into the medical image segmentation model for iterative update, and build a cross-disease medical image segmentation model based on transfer learning;

[0071] S4: Input the target domain image into the cross-disease medical image segmentation model based on transfer learning to obtain the predicted label.

[0072] This application uses knowledge transfer to achieve unsupervised segmentation of source domain data to target domain cross-disease data. This method uses images of the same part but different diseases as source domain images and target domain images respectively, and introduces domain-specific batch normalization on the basis of shared convolution kernels between domains. While reducing the dependence on labeled data, it uses their similarity in lesion structure to improve the segmentation effect of the lesion area of ​​the target data. The present invention designs a cross-disease comparative learning strategy, using the features of different domain images processed by domain-specific batch normalization as positive pairs or negative pairs, so that the model can better learn domain-invariant features. A bidirectional generation strategy between domains is designed, using labeled source domain images to improve the reliability of the model's predicted labels for unlabeled target domain images.

[0073] In some embodiments, the source domain style image and the target domain style image are obtained by image mapping, specifically:

[0074] Use the histogram matching algorithm to generate a matching image for the source domain image using the target domain image as a template;

[0075] The source domain image and the matching image are decomposed by Fourier transform to obtain the corresponding amplitude spectrum and phase spectrum;

[0076] According to the corresponding amplitude spectrum and phase spectrum, the source domain style image and the target domain style image are obtained.

[0077] In some embodiments, the acquiring of the teacher-student model integrating domain-specific batch normalization and the preliminary construction of the medical image segmentation model based on the teacher-student model integrating domain-specific batch normalization are specifically as follows:

[0078] Using U-Net as the backbone network of the teacher-student model, the BN layer in the U-Net network was replaced by the DSBN layer; a medical image segmentation model was obtained.

[0079] In some embodiments, the step of inputting the source domain image, the target domain image, the source domain style image, and the target domain style image into the medical image segmentation model for iterative updating to construct a cross-disease medical image segmentation model based on transfer learning includes:

[0080] The medical image segmentation model uses the bidirectional generation method of inter-domain data to generate source domain images and target domain images mutually, so that the unlabeled target domain image outputs a pseudo label after passing through the DSBN of the teacher model; the target domain image and the source domain style image are processed by the medical image segmentation model to obtain the target domain image probability map and the source domain style image probability map; the unsupervised loss is calculated based on the pseudo label, the target domain image probability map and the source domain style image probability map;

[0081] The source domain image is processed by the medical image segmentation model to obtain a source domain image probability map, and the full supervision loss is calculated based on the source domain image probability map;

[0082] The medical image segmentation model normalizes and extracts high-dimensional features from the source domain image, target domain image, source domain style image, and target domain style image to obtain feature maps; the overall contrast loss is calculated based on the feature maps;

[0083] Using the loss function L = L sup +λ(t)×L unsup +ω×L ct Perform back propagation to update the weight parameters of the medical image segmentation model; obtain a cross-disease medical image segmentation model based on transfer learning;

[0084] Among them, L sup is the full supervision loss; L unsup is the loss of unsupervised training; L ct is the overall contrast loss; λ(t) is the balance parameter; ω is a hyperparameter.

[0085] In some embodiments, the calculation formula for the unsupervised training is as follows:

[0086]

[0087] in is the target domain probability map, is the source domain style image probability map, Ω T is the parameter of the student model when the input is the target domain data, Ω s is the parameter of the student model when the input is source domain data, B T represents the number of source domain images in each batch during the training process, represents the pseudo label of the target domain image by the teacher model, Ω T′ is the parameter of the teacher model, δ is the random perturbation added to the input teacher model data; the mean square error is used to implement the consistency loss function L cons .

[0088] In some embodiments, the calculation formula of the full supervision loss is as follows:

[0089]

[0090] Among them, y i,c is the true label of pixel i in the source domain image, p i,c is the predicted probability of the student model for that pixel, M is the number of pixels in the image, and C is the number of categories.

[0091] In some embodiments, the feature map includes: s , hs→t , h t 、h t→s and The overall contrast loss is calculated according to the feature map, specifically:

[0092] The feature maps are divided into positive and negative pairs according to whether they are similar or not: the feature maps processed by the same DSBN layer are positive pairs, and the feature maps processed by different DSBN layers are negative pairs;

[0093] The contrast loss from the source domain image to the target domain image is calculated as:

[0094]

[0095] The contrast loss from the target domain image to the source domain image is defined as:

[0096]

[0097] Calculate the following loss functions respectively; replace h in the above formula t Replace with h s→t ,get:

[0098] Contrastive loss from source domain image to target domain style image And the contrast loss from the target domain style image to the source domain image

[0099] In the above formula, h s Replace with h t→s ,get:

[0100] Contrastive loss from source domain style image to target domain image And the contrast loss from the target domain image to the source domain style image

[0101] In the above formula, h s and h t Replace with h t→s and h s→t ,get:

[0102] Contrastive loss from target domain style image to source domain style image And the contrast loss from source domain style image to target domain style image

[0103] The overall contrast loss is the average of the above contrast losses:

[0104]

[0105] Where N = {h s ,h s→t ,ht ,h t→s} is a set of negative pairs, sim(·) indicates the calculation of cosine similarity; It is the contrast loss from source domain image to target domain image; Contrastive loss from target domain images to source domain images; represents the mathematical expectation of all source domain and target domain samples; h s Represents the batch normalized features of the source domain image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; Represents the batch normalized features obtained by processing the source domain image through the Unet encoder of the student network, the target domain DSBN and the projection head H; H represents the batch normalized features of the target domain style image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; H s→t represents the batch normalized features of the target domain style image processed by the Unet encoder of the student network, the target domain DSBN and the projection head H; h represents the batch normalized features of the target domain image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; t h represents the batch normalized features of the target domain image after being processed by the Unet encoder of the student network, the target domain DSBN, and the projection head H; t→s Represents the batch normalized features of the source domain style image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; represents the batch normalized features of the source domain style image processed by the Unet encoder of the student network, the target domain DSBN and the projection head H; τ is the temperature coefficient; L sup is the full supervision loss; L unsup is the loss of unsupervised training; L ct is the overall contrast loss; λ(t) is the balance parameter; ω is the hyperparameter; It is the contrast loss from the source domain image to the target domain style image; is the contrast loss from the target domain style image to the source domain image; is the contrast loss from the source domain style image to the target domain image; It is the contrast loss from the target domain image to the source domain style image; is the contrast loss from the target domain style image to the source domain style image; is the contrast loss from the source domain style image to the target domain style image.

[0106] [Example] Refer to Figure 2 , the present invention comprises the following steps:

[0107] Step 1) Obtain source domain data and target domain data, and divide them into training data set and test data set:

[0108] This example selects the FLAIR modality data in BraTS2020, uniformly re-labels the three glioma areas in the label as tumor areas, and selects slices with 3D brain tumor image slices between 60-120, extracts an image every two slices, and obtains a total of 7380 2D brain tumor images and their tumor annotations to form the source domain dataset. The source domain dataset is all used for training, and no test set is divided. This experiment uses the ICHS brain hemorrhage segmentation dataset in cooperation with the Air Force Medical University as the source of the target domain dataset. The ICHS dataset has a total of 270 3D brain hemorrhage medical image data, which is divided into a training set (178 cases) and a test set (92 cases), and a total of 2880 2D brain hemorrhage images are obtained to form the target domain dataset. The target domain training set includes 1920 2D brain hemorrhage images, all of which are unlabeled data. The test set includes 960 2D brain hemorrhage images. In this example, the images in the source domain dataset and the target domain data and the annotations of the annotated data are uniformly adjusted to 256*256.

[0109] Step 2) For the input source domain and target domain images, use the image mapping module to generate source domain style and target domain style images respectively to reduce the distribution difference between images in different domains:

[0110] The image mapping module first uses a histogram matching algorithm to map the input source domain image x s , with the target image x t Generate an image r(x) for the template s ;x t ), r(x s ;x t ) represents x s x t is the image after template matching, and then the grayscale image x is generated by Fourier transform s and r(x s ;x t ). For a grayscale image x of size H×W, its Fourier transform is:

[0111]

[0112] After the above transformation, the frequency signal is decomposed into the amplitude spectrum F A (x) and the phase spectrum F P (x), where the amplitude spectrum contains the image style features and the phase spectrum contains the semantic information of the image. By introducing a binary rectangular mask M λ Controls the range of the amplitude spectrum to be exchanged, thereby replacing the low-frequency components of the image and changing the image style.

[0113]

[0114] Where λ∈(0,1), I is the indicator function. -1 Represents the inverse Fourier transform, which converts the image from the frequency domain back to the spatial domain. 1 H,W represents a matrix of dimension H×W with all elements set to 1. Finally, by processing {x s ,r(x s ;x t )}, generate the target domain style image x s→t .

[0115] Step 3) See Figure 3 , we build a teacher-student model that integrates domain-specific batch normalization, fully utilize the unlabeled target domain images, and implement unsupervised consistency regularization on the unlabeled source domain images.

[0116] The domain-specific batch normalization (DSBN) module can use different batch normalization parameters for source domain images and target domain images or source domain style images and target domain style images, but the convolution kernel parameters between different domains are shared to learn domain-invariant features of similar structures between images in different domains. This example uses U-Net as the backbone network of the teacher-student model and replaces the BN layer in the U-Net network with the DSBN layer. DSBN uses unlabeled data for unsupervised training and adds random perturbations δ to the target domain number input to the teacher network. The parameters of the teacher network are updated from the student network through EMA, and its parameters at the kth training iteration are:

[0117]

[0118] α∈[0,1] is the EMA decay rate; is the parameter of the teacher model at the kth iteration; is the parameter of the teacher model at the k-1th iteration; is the parameter of the student model at the kth iteration. This method designs a bidirectional generation strategy for inter-domain data, generates source domain images and target domain images mutually, and obtains two predicted labels output by the unlabeled target domain image after passing through different DSBNs, and then takes the average of the two predicted labels as the final predicted label output of the model, further improving the segmentation accuracy of the cerebral hemorrhage area in the target domain. The loss of unsupervised training is as follows:

[0119]

[0120] in Two probability maps output by the student model, B T represents the number of source domain images in each batch during the training process, represents the prediction of the teacher model for the target domain image, and δ is the random perturbation added to the input teacher model data. The consistency loss function L is implemented using mean square error cons .

[0121] The cross entropy loss is used as the full supervision loss, and its loss function is defined as follows:

[0122]

[0123] Among them, y i,c is the true label of pixel i in the source domain image, p i,c is the predicted probability of the student model for that pixel, M is the number of pixels in the image, and C is the number of categories.

[0124] Step 4) Design a cross-disease contrastive learning strategy so that the model can capture domain-invariant features between similar lesion structures to adapt to images of different styles.

[0125] For the features output by the model encoder, a nonlinear projection head H is added to extract high-dimensional features. The source domain image and the target domain style image are represented based on the batch normalized features of the source domain and the target domain style as follows:

[0126]

[0127] Correspondingly, the target domain image and the source domain style image are represented as follows based on the source domain style and target domain batch normalization features:

[0128]

[0129] in,

[0130] h s represents the batch normalized features of the source domain image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H,

[0131] represents the batch normalized features of the source domain image processed by the Unet encoder of the student network, the target domain DSBN and the projection head H,

[0132] represents the batch normalized features of the target domain style image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H,

[0133] h s→t represents the batch normalized features of the target domain style image processed by the Unet encoder of the student network, the target domain DSBN and the projection head H,

[0134] represents the batch normalized features of the target domain image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H,

[0135] h t represents the batch normalized features of the target domain image processed by the Unet encoder of the student network, the target domain DSBN and the projection head H,

[0136] h t→s represents the batch normalized features of the source domain style image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H,

[0137] Represents the source domain style image through the Unet encoder of the student network, the target domain DSBN and the projection head

[0138] The batch normalized features obtained by H processing,

[0139] x s represents the source domain image,

[0140] x s→t represents the target domain style image,

[0141] x t represents the target domain image,

[0142] x t→s represents the source domain style image,

[0143] θ H Represents the projection head parameters,

[0144] θ en represents the model encoder parameters,

[0145] represents the source domain batch normalization parameter,

[0146] represents the target domain batch normalization parameters.

[0147] DSBN normalizes the source domain and target domain feature maps respectively and maps them to a common feature distribution space. Given a pair of images x from the source domain and the target domain s and x t ,h s and h t are the characteristics of their corresponding BN in the public distribution space. s and h t They should be similar to each other and set to be opposite. s and h s , which are the source domain images x sThe features obtained by using two different BNs are different because the two BN association domains are different. s and h s The pairs should be different from each other and set as negative pairs. Set positive and negative pairs based on the above rules. (DSBN is a strategy. The so-called specific domain refers to the source domain and the target domain. Domain specificity means doing BN in these two domains separately, so in essence they are still two different BNs, which can be seen from the eight formulas above);

[0148] The contrast loss from source domain to target domain is defined as:

[0149]

[0150] N={h s ,h s→t ,h t ,h t→s} is a negative pair set, sim(·) represents the calculation of cosine similarity. The contrast loss from the target domain to the source domain is defined as:

[0151]

[0152] Calculate the contrast loss between the source domain and the target domain style respectively and Contrast loss between source domain style and target domain and Contrast loss between source domain style and target domain style and The overall contrast loss is the average of the above contrast losses:

[0153]

[0154] Step 5) Iteratively train the cross-disease cerebral hemorrhage region segmentation network model based on transfer learning: use the loss function L = L sup +λ(t)×L unsup +ω×L ct Perform back propagation to update the model weight parameters.

[0155] The overall loss is composed of full supervision loss, unsupervised loss, and contrast loss. ω is a hyperparameter, which represents the weight used to balance the contrast loss. λ(t) is a balancing parameter, which is implemented using a Gaussian warm-up function. Control the balance between the fully supervised loss and the unsupervised consistency loss to ensure that the overall loss is dominated by the fully supervised part at the beginning of training, avoiding model degradation due to failure to learn from low-quality pseudo-labels in the early stage of training.

[0156] Step 6) Obtain the medical image segmentation results of the transfer learning cross-disease model: Use the test sample set as the input of the trained transfer learning-based cross-disease medical image segmentation network model for forward reasoning to obtain the segmentation score of each test sample, that is, the predicted label.

[0157] Compared with the medical image segmentation method in “A medical image segmentation method based on transfer learning”, in the present invention, knowledge transfer not only involves different data modalities, but also different lesion types. This method uses an image mapping module to realize the mutual conversion of cross-disease images between the source domain and the target domain. Secondly, a cross-disease contrast learning strategy is proposed to better learn domain-invariant features. This method improves the segmentation performance of the model in the target domain by using images from other domains with similar tissues, organs or lesions to assist training.

[0158] The present invention uses image mapping to achieve the mutual conversion of cross-disease images of brain tumors and cerebral hemorrhage to reduce the distribution difference between source domain and target domain images. The present invention introduces domain-specific batch normalization to deal with the domain shift problem and integrates it into the mean teacher consistency regularization framework. The present invention proposes a cross-disease contrastive learning strategy to better learn domain-invariant features, and designs a bidirectional generation strategy between domains to further improve the segmentation accuracy of the model.

[0159] The present application also discloses a cross-disease medical image segmentation system based on transfer learning, which is characterized by comprising:

[0160] A mapping unit, used to obtain a source domain image and a target domain image, and obtain a source domain style image and a target domain style image through image mapping;

[0161] A model building unit, used for obtaining a teacher-student model integrating domain-specific batch normalization, and preliminarily building a medical image segmentation model based on the teacher-student model integrating domain-specific batch normalization;

[0162] A model updating unit, used to input source domain images, target domain images, source domain style images, and target domain style images into the medical image segmentation model for iterative update, and to build a cross-disease medical image segmentation model based on transfer learning;

[0163] The output unit is used to input the target domain style image into the cross-disease medical image segmentation model based on transfer learning to obtain the sample label.

[0164] The present application also discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned methods for cross-disease medical image segmentation based on transfer learning when executing the computer program.

[0165] The present application also discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for cross-disease medical image segmentation based on transfer learning are implemented.

[0166] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A cross-disease medical image segmentation method based on transfer learning, characterized in that: include: Obtain a source domain image and a target domain image, and obtain a source domain style image and a target domain style image through image mapping; Obtain a teacher-student model integrating domain-specific batch normalization, and preliminarily build a medical image segmentation model based on the teacher-student model integrating domain-specific batch normalization; Input the source domain image, target domain image, source domain style image, and target domain style image into the medical image segmentation model for iterative update, and build a cross-disease medical image segmentation model based on transfer learning; The target domain image is input into the cross-disease medical image segmentation model based on transfer learning to obtain the predicted label.

2. According to the method for cross-disease medical image segmentation based on transfer learning in claim 1, it is characterized in that: The source domain style image and the target domain style image are obtained by image mapping, specifically: Use the histogram matching algorithm to generate a matching image for the source domain image using the target domain image as a template; The source domain image and the matching image are decomposed by Fourier transform to obtain the corresponding amplitude spectrum and phase spectrum; According to the corresponding amplitude spectrum and phase spectrum, the source domain style image and the target domain style image are obtained.

3. The cross-disease medical image segmentation method based on transfer learning according to claim 1, characterized in that: The teacher-student model integrating domain-specific batch normalization is obtained, and a medical image segmentation model is preliminarily constructed based on the teacher-student model integrating domain-specific batch normalization, specifically: U-Net is used as the backbone network of the teacher-student model. The teacher-student model includes a teacher model and a student model. The BN layers in the U-Net networks of the teacher model and the student model are replaced with DSBN layers; a medical image segmentation model is obtained.

4. The cross-disease medical image segmentation method based on transfer learning according to claim 3, characterized in that: The source domain image, the target domain image, the source domain style image and the target domain style image are input into the medical image segmentation model for iterative update to construct a cross-disease medical image segmentation model based on transfer learning, including: The medical image segmentation model uses the bidirectional generation method of inter-domain data to generate source domain images and target domain images mutually, so that the unlabeled target domain image outputs a pseudo label after passing through the DSBN of the teacher model; the target domain image and the source domain style image are processed by the medical image segmentation model to obtain the target domain image probability map and the source domain style image probability map; the unsupervised loss is calculated based on the pseudo label, the target domain image probability map and the source domain style image probability map; The source domain image is processed by the medical image segmentation model to obtain a source domain image probability map, and the full supervision loss is calculated based on the source domain image probability map; The medical image segmentation model normalizes and extracts high-dimensional features from the source domain image, the target domain image, the source domain style image, and the target domain style image to obtain feature maps; the overall contrast loss is calculated based on the feature maps; Using the loss function L = L sup +λ(t)×L unsup +ω×L ct Perform back propagation to update the weight parameters of the medical image segmentation model; obtain a cross-disease medical image segmentation model based on transfer learning; Among them, L sup is the full supervision loss; L unsup is the loss of unsupervised training; L ct is the overall contrast loss; λ(t) is the balance parameter; ω is a hyperparameter.

5. The cross-disease medical image segmentation method based on transfer learning according to claim 4, characterized in that: The calculation formula of the unsupervised loss is as follows: in is the target domain probability map, is the source domain style image probability map, Ω T is the parameter of the student model when the input is the target domain data, Ω s is the parameter of the student model when the input is source domain data, BT represents the number of source domain images in each batch during the training process, represents the pseudo label of the target domain image by the teacher model, Ω T′ is the parameter of the teacher model, δ is the random perturbation added to the input teacher model data; the mean square error is used to implement the consistency loss function L cons .

6. The cross-disease medical image segmentation method based on transfer learning according to claim 4, characterized in that: The calculation formula of the fully supervised loss is as follows: Among them, y i,c is the true label of pixel i in the source domain image, and the true label is the label of the source domain image itself; p i,c is the predicted probability of the student model for that pixel, M is the number of pixels in the image, and C is the number of categories.

7. The cross-disease medical image segmentation method based on transfer learning according to claim 4, characterized in that: The characteristic graph includes: s , h s→t , h t 、h t→s and The overall contrast loss is calculated according to the feature map, specifically: The feature maps are divided into positive and negative pairs according to whether they are similar or not: the feature maps processed by the same DSBN layer are positive pairs, and the feature maps processed by different DSBN layers are negative pairs; The contrast loss from the source domain image to the target domain image is calculated as: The contrast loss from the target domain image to the source domain image is defined as: Calculate the following loss functions respectively; replace h in the above formula t Replace with h s→t ,get: Contrastive loss from source domain image to target domain style image And the contrast loss from the target domain style image to the source domain image In the above formula, h s Replace with h t→s ,get: Contrastive loss from source domain style image to target domain image And the contrast loss from the target domain image to the source domain style image In the above formula, h s and h t Replace with h t→s and h s→t ,get: Contrastive loss from target domain style image to source domain style image And the contrast loss from source domain style image to target domain style image The overall contrast loss is the average of the above contrast losses: Where N = {h s ,h s→t ,h t ,h t→s } is a set of negative pairs, sim(·) indicates the calculation of cosine similarity; It is the contrast loss from source domain image to target domain image; Contrastive loss from target domain images to source domain images; represents the mathematical expectation of all source domain and target domain samples; h s Represents the batch normalized features of the source domain image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; Represents the batch normalized features obtained by processing the source domain image through the Unet encoder of the student network, the target domain DSBN and the projection head H; h represents the batch normalized features of the target domain style image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; s→t represents the batch normalized features of the target domain style image processed by the Unet encoder of the student network, the target domain DSBN and the projection head H; h represents the batch normalized features of the target domain image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; t h represents the batch normalized features of the target domain image after being processed by the Unet encoder of the student network, the target domain DSBN, and the projection head H; t→s Represents the batch normalized features of the source domain style image processed by the Unet encoder of the student network, the source domain DSBN and the projection head H; represents the batch normalized features of the source domain style image processed by the Unet encoder of the student network, the target domain DSBN and the projection head H; τ is the temperature coefficient; L sup is the full supervision loss; L unsup is the loss of unsupervised training; L ct is the overall contrast loss; λ(t) is the balance parameter; ω is the hyperparameter; It is the contrast loss from the source domain image to the target domain style image; is the contrast loss from the target domain style image to the source domain image; is the contrast loss from the source domain style image to the target domain image; It is the contrast loss from the target domain image to the source domain style image; is the contrast loss from the target domain style image to the source domain style image; is the contrast loss from the source domain style image to the target domain style image.

8. A cross-disease medical image segmentation system based on transfer learning, characterized in that: include: A mapping unit, used to obtain a source domain image and a target domain image, and obtain a source domain style image and a target domain style image through image mapping; A model building unit, used for obtaining a teacher-student model integrating domain-specific batch normalization, and preliminarily building a medical image segmentation model based on the teacher-student model integrating domain-specific batch normalization; A model updating unit, used to input source domain images, target domain images, source domain style images, and target domain style images into the medical image segmentation model for iterative update, and to build a cross-disease medical image segmentation model based on transfer learning; The output unit is used to input the target domain style image into the cross-disease medical image segmentation model based on transfer learning to obtain the sample label.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the cross-disease medical image segmentation method based on transfer learning as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the cross-disease medical image segmentation method based on transfer learning according to any one of claims 1 to 7.

Images