A medical image segmentation method based on multi-level asymmetric contrast learning
By employing a multi-level asymmetric contrastive learning method, this approach addresses the issues of untapped multi-level representation potential and insufficient encoder-decoder synergy in existing medical image segmentation methods, thereby achieving better medical image segmentation performance.
Patent Information
- Application Number
- CN202510609991.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-13
- Publication Date
- 2025-10-24
- Estimated Expiration
- 2045-05-13
AI Technical Summary
Existing medical image segmentation methods neglect the multi-level representation potential in self-supervised learning, and the encoder-decoder structure is not effectively utilized, resulting in poor performance of pre-trained networks.
A multi-level asymmetric contrastive learning method is adopted. By constructing an asymmetric contrastive learning framework and a multi-level contrastive learning strategy, synchronous pre-training of the encoder and decoder is achieved. Image-level, pixel-level and feature-level contrastive losses are combined, and equivariant regularization is introduced to enhance self-supervision capabilities.
Under downstream restricted supervision, the performance of the medical image segmentation model was improved, enabling precise localization and segmentation of the specific location of medical organs or lesions.
Smart Images

Figure CN120510166B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application provides a medical image segmentation method based on multi-level asymmetric contrast learning, and belongs to the field of self-supervised pre-training methods. BACKGROUND
[0002] Medical image segmentation, by dividing the whole image into different tissues, organs and other biologically related structures, plays a crucial role in computer-aided diagnosis. However, traditional supervised learning (Supervised Learning) strategy usually requires a large amount of labeled data, which not only consumes time but also costs a lot. In recent years, self-supervised learning (Self-supervised Learning) strategy provides a promising solution to this challenge: only relying on unlabeled raw data to pre-train the model, so that the model can achieve better performance in the case of limited labeled data in downstream tasks (medical image segmentation). As an important branch of self-supervised learning, contrast learning (Contrastive Learning) pre-trains the model by using contrast loss functions such as InfoNCE, to pull positive pairs (similar sample pairs) closer in the feature space while pushing negative pairs (different sample pairs) away, so that the model can learn meaningful representations from unlabeled data.
[0003] Contrast learning strategy has achieved remarkable success in natural image field, including SimCLR, MoCo, SwAV, BYOL, SimSiam, etc. In practice, the more negative pairs, the better the performance of contrast learning. Therefore, SimCLR uses a large batch size to coexist with a large number of negative samples in the current training batch, and MoCo introduces a dynamic memory bank to maintain negative sample sequences. In addition to the above classic training framework, SwAV integrates online clustering into the twin network and proposes a new "multi-crop" augmentation strategy that mixes different resolution views. BYOL introduces a slow moving average network and shows that contrast learning is effective even without negative samples. And SimSiam introduces gradient halting and proves that even without negative pairs, large batches or momentum encoders, a simple twin network can learn meaningful features. In addition, WCL introduces a graph-based method to explore more positive pairs and generate weak labels for supervised contrast learning strategy.
[0004] In the field of medical images, due to the problem of limited quantity caused by cumbersome data labeling, researchers have also begun to focus on how to use unlabeled data to improve network performance. In the aspect of contrast learning, research work mainly focuses on the design of twin networks and the construction of positive and negative sample pairs. In detail, in the design of twin networks, DeSD adopts a deep self-distillation method to reconstruct contrast learning, and improves the representation quality of shallow and deep autoencoders to cope with the challenge of shallow weak supervision. DiRA designs a novel self-supervised learning framework by integrating discriminative learning, restoration learning and adversarial learning to extract complementary visual information from unlabeled medical images. In the construction of positive and negative sample pairs, SimTriplet introduces a contrast learning framework based on triplets, which maximizes the similarity within and between samples through triplets of positive sample pairs. GCL proposes a partition-based contrast learning strategy that uses the similarity of anatomical structures at the same location of different patients in the 3D medical image dataset to divide positive and negative sample pairs: the aligned 3D dataset is divided into 2D slices according to the z-axis, and all 2D images are divided into four partitions. Images belonging to the same partition in different patients are considered to belong to the same class and are constructed as positive sample pairs. Similarly, PCL divides positive and negative sample pairs by introducing a location difference-based contrast learning strategy.
[0005] Although the above contrast learning methods have achieved success, there are still some important problems to be solved. First, existing methods focus on extracting image-level representations, ignoring the potential of rich multi-level representations, resulting in suboptimal performance of pre-trained network models. Second, the encoder-decoder structure is not effectively utilized—decoders are either randomly initialized or pre-trained separately from encoders, resulting in the loss of potential collaborative relationships. SUMMARY
[0006] In view of the problems existing in the prior art, the present application provides a medical image segmentation method based on multi-level asymmetric contrast learning, which can learn meaningful representations from unlabeled data and provide better initialization for downstream medical image segmentation models under limited supervision, improving segmentation performance.
[0007] The technical scheme provided by the present application is as follows:
[0008] A medical image segmentation method based on multi-level asymmetric contrast learning, comprising the following steps:
[0009] A. Medical image dataset construction, the three-dimensional CT / MRI medical image dataset is cropped along the z-axis to obtain the corresponding two-dimensional slice image, for the unlabeled dataset image X u used for pre-training upstream; for the labeled dataset image Xd , and the corresponding segmentation label image is Y d . Therefore, the constructed upstream pre-training dataset is denoted as The downstream medical image segmentation fine-tuning dataset is denoted as N is the number of images of the upstream pre-training dataset, and M is the number of images of the downstream medical image segmentation fine-tuning dataset.
[0010] B. Medical image segmentation task model construction. Select any existing deep learning encoder-decoder model structure for the medical image segmentation task or create a new deep learning model structure, and denote the constructed encoder structure of the medical image segmentation model as e(·) and the decoder structure as d(·). In the contrastive learning pre-training stage, an image-level projector is also needed to be constructed, which is used to obtain the image-level projection for calculating the global contrastive loss function, and a pixel-level projector is also needed to be constructed, which is used to obtain the pixel-level projection for calculating the dense contrastive loss function. The structure of the projector is usually an MLP (Multi-Layer Perceptron).
[0011] C. Upstream contrastive learning pre-training: using the unannotated dataset Ω U constructed in A
[0012] C1. Asymmetrical contrastive learning framework design. In order to realize the single-stage synchronous training of the encoder e(·) and the decoder d(·), and better assist the downstream segmentation task, the present application constructs an asymmetrical contrastive learning framework (Asymmetrical Contrastive Learning Framework), the specific steps are as follows:
[0013] C1a. Enhanced data acquisition. For a batch of input images X u ∈R B×C×H×W , B denotes the batch size, C is the number of channels of the input image, HxW is the size of the input image, H denotes the height of the image, and W denotes the width of the image. The present application adopts two different enhancement methods: (1) the first method uses a non-spatial transformation T fix (two different random parameters) to obtain a set of spatially invariant images {I 1 ,I 2} (2) the second method uses a spatial transformation T var (two different random parameters) to obtain a set of spatially variant images {V 1 ,V 2}:
[0014] {I1 I 2}=T fix (X u ),{V 1 ,V 2}=T var (X u )
[0015] C1b. Constructing auxiliary branch. The auxiliary branch includes an encoder e(·), an image-level projector g(·) and a pixel-level projector g(·). and pixel-level projector Specifically, the spatially-invariant image I 2 with fixed pixel position is input into the encoder e(·) to obtain the feature-level representation F where B is the batch size, C is the number of channels of the feature map, h and w are the height and width of the feature map, respectively, and then is transmitted into the pixel-level projector g(·) to obtain the pixel-level projection Z
[0016]
[0017] For the spatially-variant image group V 2 with changed pixel position, it is input into the encoder e(·) and the image-level projector g(·) to obtain the image-level projection F
[0018]
[0019] C1c. Constructing dominant branch. Unlike the auxiliary branch, the dominant branch introduces a partial decoder d p (·) after the encoder e(·) to pre-train synchronously and mine more feature-level information. In addition, the partial decoder d p (·) is also connected with the image-level projector g ins (·) and the pixel-level projector g pix (·). Since the dominant branch introduces the decoder, the feature map sizes obtained in the two branches will be different for the same image input. Therefore, the present application introduces a down-sampling operation down(·) in the data augmentation stage to align the feature representations of the two branches, reduce the complexity and ensure sufficient negative sample pairs. Specifically, for the spatially-invariant image I 1 with fixed pixel position, first, down-sampling processing is performed, and then the image is input into the encoder e(·) (which shares parameters with the auxiliary branch) to obtain the feature-level representation F, and then is transmitted into the decoder d(·) to restore the image details and generate the pixel-level projection Z l :
[0020] F=e(down(I1 )),Z l = d p (F)
[0021] For the spatially varying image V 1 with varying pixel positions, the same downsampling process is applied, and then input to the encoder e(·) and image-level projector g ins (·) to obtain the image-level projections Z g :
[0022] Z g = g ins (e(down(V 1 )))
[0023] C2. Multi-level contrastive learning strategy. Under the guarantee of the asymmetric contrastive learning framework designed in C1, the invention can realize the synchronous pre-training of the encoder and the decoder, and on this basis, the invention optimizes the model at the image level, pixel level and feature level through the multi-level contrastive learning strategy (Multi-Level Contrastive Learning, MLCL). The specific steps are as follows:
[0024] C2a. Global contrastive learning. The image-level representation is constrained by the global contrastive loss. Specifically, the unlabeled dataset after data augmentation, 2N samples are obtained, that is The image-level projections of the dominant branch and the auxiliary branch corresponding to it are respectively: and The global contrastive loss is defined as follows:
[0025]
[0026] Where, represents the index set forming a positive sample pair with , sim(·,·) represents the cosine similarity, which is used to calculate the similarity of two vectors in the feature space, and τ is the temperature scaling parameter, is the indicator function. Compared with the standard contrastive loss, in which the numerator part only contains one positive sample pair, the invention adopts the PCL (Position-based Contrastive Learning) strategy to generate positive and negative sample pairs combined with position information, in order to improve the contrastive learning effect.
[0027] C2b. Dense Contrastive Learning. Pixel-level representation is constrained by dense contrastive loss, which considers the correspondence between pixel-level features, extending the global contrastive loss to a dense (pixel-level) form. Unlike image-level projection and , and Pixel-level representations are generated by pixel-level projection heads in the dominant branch and the auxiliary branch, which are dense feature maps of size SxS, where S represents the spatial size. For each two input images, S 2 pixel-level projection samples (positive sample pairs or negative sample pairs, depending on the position change Δ) can be formed according to the position-based contrastive sample generation strategy. Therefore, the dense contrastive loss is defined as:
[0028]
[0029] where, and represent the projection of the s-th pixel in the dominant branch and the auxiliary branch, respectively.
[0030] C2c. Equivariant Regularization. Feature-level representation is constrained by equivalent regularization. Since the introduction of the down-sampling operation in the dominant branch results in different sizes of feature representations learned by the two encoders, the present invention introduces equivalent regularization in multi-level contrastive learning to impose consistency constraints on multi-scale feature representations, thereby providing additional self-supervised signals for network learning. The equivalent regularization loss is defined as follows:
[0031]
[0032] where, are the feature representations learned by the encoders in the dominant branch and the auxiliary branch, respectively; down(·) represents the down-sampling operation, and the L1 norm is used to measure the consistency of the feature representations. Through this equivalent regularization, the present invention can impose consistency constraints at the feature level, thereby further enhancing the self-supervised ability of the network.
[0033] C3. Loss function calculation and parameter update. In each epoch, a batch of training data is first obtained, and then the three loss functions are calculated according to the steps of C2a-c, and the final multi-level contrastive loss function
[0034]
[0035] Wherein, λ1, λ2, λ3 represent weight coefficients, and the whole pre-training is performed in a single-stage manner. The model parameters are updated using the stochastic gradient descent method, and C1-C2 are repeated until the total loss function of the model training converges to a stable minimum value, at which time the parameter states of the encoder e(·) and part of the decoder d(·) in the dominant branch are the pre-trained model parameters, which are used to initialize the encoder and decoder of the downstream medical image segmentation task.
[0036] D. Fine-tuning training of the medical image segmentation task under the downstream limited supervision condition. For the encoder e(·) and the decoder d(·) of the medical image segmentation model built in step B, the pre-trained encoder e(·) and part of the decoder d p (·) in the C3 dominant branch are loaded into the present application, and the parameters of the remaining part of the encoder are randomly initialized. The downstream labeled data set is divided into a training set and a test set according to a certain proportion (such as 8:2), and only a small part of the data (10% / 25%) in the training set is used to fine-tune the medical image segmentation model of the present application.
[0037] E. The test set of the downstream labeled data set is used to obtain the medical image segmentation result using the medical image segmentation model trained in step D.
[0038] Further, the three-dimensional CT / MRI medical image data set is cropped along the z-axis in step A to obtain the corresponding two-dimensional slice image.
[0039] Further, the medical image segmentation task model structure in step B includes but is not limited to: (1) AttUNet introducing attention mechanism; (2) UCTransNet combining transformer and UNet; (3) RollingUNet integrating MLP into UNet; (4) UKAN introducing nonlinearity, etc.
[0040] Compared with the prior art, the present application has the following beneficial effects:
[0041] (1) The present application forms an asymmetric structure by introducing part of the decoder in the dominant branch, and realizes spatial invariance and consistent representation through down-sampling, compared with the prior art which only pre-trains the encoder and randomly initializes the decoder or separates the encoder and decoder for two-stage pre-training, resulting in the loss of potential collaboration between the encoder and the decoder. The present application realizes single-stage encoder-decoder synchronous pre-training, ensuring the collaboration between the encoder and the decoder.
[0042] (2) The present application integrates image-level, pixel-level and feature-level representation correspondence by constructing a multi-level correlation mechanism from global to local, compared with the prior art focusing on extracting image-level features and ignoring the rich multi-level representation potential, the encoder-decoder simultaneously captures multi-scale detail features in the pre-training stage.
[0043] (3) The present application pre-trains the model by proposing an asymmetric contrast learning architecture and a multi-level contrast learning strategy, so that the medical segmentation model fine-tuned under the downstream limited supervision condition can accurately locate and segment the specific position of the medical organ or lesion. BRIEF DESCRIPTION OF DRAWINGS
[0044] Figure 1 is a flowchart of a specific embodiment of the present application.
[0045] Figure 2 is a schematic diagram of a specific embodiment example of the present application.
[0046] Figure 3 is a schematic diagram of multi-organ segmentation results of a specific embodiment of the present application. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific implementation method of the present application will be further described below in combination with the drawings.
[0048] The present application provides a medical image segmentation method based on multi-level asymmetric contrast learning. Specifically, the present application provides implementation steps on the multi-organ segmentation task. Among them, multi-organ segmentation is a key problem in medical image analysis, and the goal is to identify and segment multiple organ regions such as liver, kidney, pancreas, etc. in the same medical image. The shapes of different organs in this task are different, and the size difference between some organs is very large. The area ratio difference between the myocardium and the left and right ventricles in the ACDC dataset is very large. In addition, the positions of some organs are close, which makes the boundary blurred and difficult to distinguish, for example, the left atrium, left ventricle and myocardium in the MMWHS dataset. The above characteristics make the multi-organ segmentation task complex and challenging. Figure 1 The specific implementation steps of the present application are as follows:
[0049] A. Medical image dataset construction. The open-source multi-organ segmentation dataset CHD is used as the upstream unannotated dataset for pre-training, which contains 68 patients, a total of 17525 CT images of 2D slices, covering 7 substructures: left ventricle, right ventricle, left atrium, right atrium, myocardium, aorta and pulmonary artery. The downstream annotated dataset for multi-organ segmentation fine-tuning includes ACDC (left ventricle, right ventricle, myocardium) and MMWHS (left atrium, right atrium, left ventricle, right ventricle, myocardium). The number of images in the N upstream pre-training dataset is set to 17525 in the embodiment of the present application, and the number of images in the M downstream medical image segmentation fine-tuning dataset is set to 1902 for the ACDC dataset and 5305 for the MMWHS dataset.
[0050] B. Medical image segmentation task model building. In this example, the model selected by the present application is a convolutional neural network, and the model architecture is a standard UNet model, in which the encoder e(·) includes four convolutional layers and a central layer in the last layer. Each convolutional layer consists of two Conv2d→Batchnorm2d→LeakyReLU structures and a 2x2 max-pooling layer with a step size of 2. The convolution kernel size of Conv2d is 3x3, and one pixel of zero padding is performed around it. Similar to the encoder, the decoder d(·) also consists of four convolutional layers and a projector, each convolutional module containing a deconvolution block ConvTranspose2d followed by two Conv2d→BatchNorm2d→LeakyReLU structures, which are the same as the structures in the encoder. The projector is a simple 2D convolution (Conv2d) structure used to project the fused features into a segmentation map. The structure of the image-level projector used for upstream contrastive learning pre-training is AdaptiveAvgPool2d→Flatten→Linear→ReLU→Linear, and the structure of the pixel-level projector is two Conv2d→BatchNorm2d→LeakyReLU. The convolution kernel size of Conv2d in both projectors is 3x3, and one pixel of zero padding is performed around it. The parameters of the model are set using Kaiming initialization in the PyTorch library.
[0051] C. Upstream contrastive learning pre-training: As shown in Figure 2 , the present application uses the unannotated dataset CHD in step A to pre-train the encoder e(·) in the medical image segmentation model constructed in step B. The specific implementation steps are as follows:
[0052] C1. Asymmetric contrastive learning structure design. In order to realize the single-stage synchronous training of the encoder e(·) and the decoder d(·), and better assist the downstream segmentation task, the present application constructs an asymmetric contrastive learning framework, the specific steps of which are as follows:
[0053] C1a. Enhance data acquisition. For a batch of grayscale images X u ∈R 16 ×512×512 , the present application adopts two different enhancement methods: (1) The first one uses non-spatial transformation T fix including brightness, contrast, Gaussian blur to obtain a set of spatial invariant images {I 1 ,I 2}; (2) The second one uses spatial transformation T var including translation, rotation and scaling to obtain a set of spatial variant images {V 1 ,V 2}.
[0054] C1b. Construct auxiliary branch. Spatial invariant images I 2 with fixed pixel positions are input into the encoder e(·) to obtain feature-level representation which is then passed into the pixel-level projector to obtain pixel-level projection As for V 2 in the spatial variant image set with changed pixel positions, it is input into the encoder e(·) and the image-level projector to obtain image-level projection
[0055] C1c. Construct main branch. For spatial invariant images I 1 with fixed pixel positions, first down-sampling processing is performed, and then the images are input into the encoder e(·) (which shares parameters with the auxiliary branch) to obtain feature-level representation F, which is then passed into the decoder d(·) to restore image details while generating pixel-level projection Z l ∈R 16×128×32×32 ; as for spatial variant images V 1 with changed pixel positions, the same down-sampling processing is performed, and then the images are input into the encoder e(·) and the image-level projector g ins (·) to obtain image-level projection Z g ∈R 16×128 .
[0056] C2. Multi-scale contrast learning strategy. Under the guarantee of the asymmetric contrast learning framework designed in C1, the present application can realize the synchronous pre-training of the encoder and the decoder. On this basis, the present application optimizes the model at the image level, the pixel level and the feature level respectively through a multi-level contrast learning strategy. The specific steps are as follows:
[0057] C2a. Global contrastive learning. Image-level representations are constrained by global contrastive loss. Specifically, the input images of a batch in the unlabeled dataset used for upstream pre-training are After data enhancement, 32 samples are obtained, namely The corresponding image-level projections of the dominant branch and auxiliary branch are: and Used to calculate the global contrast loss:
[0058]
[0059] C2b. Dense contrast learning. Pixel-level representation is constrained by dense contrast loss, which takes into account the correspondence between pixel-level features and expands the global contrast loss into a dense (pixel-level) form. and different, and i∈[1,16] is the pixel-level representation generated by the pixel-level projection head in the leading branch and the auxiliary branch. These representations are dense feature maps of size 32 × 32. For every two input images, according to the position-based contrast sample generation strategy, 1024 pairs of pixel-level projection samples (positive sample pairs or negative sample pairs, depending on the position change Δ) can be formed to calculate the dense contrast loss:
[0060]
[0061] C2c. Equivariant regularization. Feature-level representations are constrained through equivariant regularization. This paper introduces equivariant regularization into multi-level contrastive learning to impose consistency constraints on multi-scale feature representations, thereby providing additional self-supervisory signals for network learning:
[0062]
[0063] C3. Loss function calculation and parameter update. The present invention pre-trains the encoder for 100 epochs, with a batch size of 16, an SGD optimizer, an initial learning rate of 0.1, and then uses a cosine scheduler to decay to 0 in each training iteration. In each epoch, first obtain a batch of training data, and then calculate the three loss functions according to steps C2a-c. And get the final multi-level contrast loss function
[0064]
[0065] The model parameters are updated using the stochastic gradient descent method, and C1-C2 are repeated until the total loss function of the model training converges to a stable minimum value, at which time the parameter states of the encoder e(·) and the partial decoder d(·) in the dominant branch are the pre-trained model parameters, which are used to initialize the encoder and decoder of the downstream medical image segmentation task.
[0066] D. Fine-tuning training of the medical image segmentation task under the downstream limited supervision condition. For the encoder e(·) and the decoder d(·) of the medical image segmentation model built in step B, the pre-trained encoder e(·) and partial decoder d p (·) in the C3 dominant branch are loaded into the medical image segmentation model, and the parameters of the remaining part of the encoder are randomly initialized. The downstream labeled data set (taking MMWHS as an example) is divided into a training set and a test set in a ratio of 8:2, and only 10% / 25% of the labeled data in the training set is used to fine-tune the medical image segmentation model.
[0067] E. The test set of the downstream labeled data set is used to obtain the medical image segmentation result using the medical image segmentation model trained in step D. The data used for testing is a small part of the data (20%) that is not used for fine-tuning the medical image segmentation model from the downstream labeled data set MMWHS. The test data also contains a segmentation label corresponding to each image
[0068] The present application uses DSC (Dice Similarity Coefficient), JC (Jaccard Coefficient), HD95 (95 th percentile of the Hausdorff Distance) and ASD (Average Surface Distance) to measure the segmentation performance of the model. Specifically:
[0069]
[0070] where X seg is the pixel set in the predicted segmentation map, X gt is the pixel set in the label image, and |·| represents the number of pixels in the set.
[0071]
[0072] where S seg and S gtare the surface pixels in the predicted segmentation map and the label map respectively, d(v, u) is the Euclidean distance between pixel v and pixel u. The value of DSC and JC ranges from 0 to 1, and the larger the value, the better the segmentation performance. The smaller the value of HD95 and ASD, the better the segmentation performance. Table 1 shows the segmentation results on the ACDC and MMWHS two multi-organ segmentation datasets. In terms of qualitative evaluation, the segmentation results are visualized in grayscale, and different colors are used to display different organs or lesion regions. The JC / MIoU values are calculated as shown in Figure 3 .
[0073] Table 1 Quantitative evaluation results of multi-organ segmentation tasks.
[0074]
[0075] The above examples are only some preferred embodiments of the present application and are not a limitation on the scope of protection of the present application. Any change made by using the design principles of the present application and on this basis without creative labor shall belong to the protection scope of the present application.
Claims
1. A medical image segmentation method based on multi-level asymmetric contrastive learning, comprising the following steps: A. Medical image dataset construction, for the image X of the unlabelled dataset used for pre-training upstream u ; for the image X of the labelled dataset used for medical image segmentation task fine-tuning downstream d , the corresponding segmentation label image is Y d , the constructed upstream pre-training dataset is denoted as The downstream medical image segmentation fine-tuning dataset is denoted as N is the number of images of the upstream pre-training dataset, and M is the number of images of the downstream medical image segmentation fine-tuning dataset. B. The medical image segmentation task model is built, the encoder structure of the built medical image segmentation model is e(·), the decoder structure is d(·), and an image-level projector is built The features are projected to obtain an image-level projection used for calculating a global contrast loss function and a pixel-level projector The features are projected to obtain a pixel-level projection used for calculating a dense contrast loss function; C. Upstream contrastive learning pre-training: using the unlabeled dataset Ω constructed in step A U Pre-training the encoder e(·) and the decoder d(·) in the medical image segmentation model constructed in step B, the specific implementation steps are: C1. Constructing an asymmetric contrastive learning framework, the specific steps are as follows: C1a. For a batch of input images X u ∈R B×C×H×W , B denotes batch size, C denotes the number of channels of input images, HxW denotes the size of input images, H denotes image height, and W denotes image width. Two different augmentation methods are used: the first uses a non-spatial transformation T fix to obtain a spatially invariant image set {I 1 , I 2}; the second uses a spatial transformation T var to obtain a spatially variant image set {V 1 , V 2}: {I 1 ,I 2}=T fix (X u ),{V 1 ,V 2}=T var (X u ) C1b. Constructing the auxiliary branch, the spatially invariant image I with fixed pixel positions 2 Input into the encoder e(·) to obtain feature-level representations wherein, B is the batch size, C refers to the number of channels of the feature map, h and w refer to the height and width of the feature map, respectively, which are then passed into a pixel-level projector Obtaining a pixel-level projection For spatially varying image sets where the pixel position changes, V 2 is input to the encoder e(·) and the image-level projector to obtain the image-level projection C1c. Constructing the dominant branch for a spatially invariant image I fixed for pixel position 1 , first undergoes down-sampling, then is input to the encoder e(·) to obtain a feature-level representation d, which is then passed into the decoder d(·) to recover image details while generating a pixel-level projection Z l : F = e(down(I 1 )), Z l = d p (F) For the spatially varying image V where the pixel positions change 1 , and then input to the encoder e(·) and the image-level projector g ins (·) to obtain the image-level projection Z g : Z g = g ins (e(down(V 1 ))) C2. Optimizing the model at the image level, pixel level and feature level respectively through a multi-level contrastive learning strategy, the specific steps are as follows: C2a. Unlabeled dataset for upstream pre-training After data augmentation, 2N samples are obtained, i.e. The image-level projections of the corresponding dominant branch and auxiliary branch are respectively: and The global contrast loss is defined as follows: wherein, denotes an index set of positive sample pairs, sim(·,·) represents a cosine similarity for calculating the similarity of two vectors in a feature space, and τ is a temperature scaling parameter, denotes an index set of positive sample pairs, sim(·,·) represents a cosine similarity for calculating the similarity of two vectors in a feature space, and τ is a temperature scaling parameter, is an indicator function, and a positive and negative sample pair is generated by combining position information using a PCL strategy. C2b. The pixel-level representation is constrained by a dense contrastive loss that extends the global contrastive loss to a dense form, and are pixel-level representations generated by pixel-level projection heads in the dominant branch and the auxiliary branch, which are dense feature maps of size SxS, where S represents the spatial size, and for each two input images, S 2 For the pixel-level projection samples, the dense contrastive loss is defined as: wherein and P s and P s denote the projection of the s-th pixel in the dominant and auxiliary branch, respectively. C2c. The feature level representation is constrained by isometric regularization, that is, isometric regularization is introduced in multi-level contrastive learning to impose consistency constraints on multi-scale feature representations, thereby providing additional self-supervised signals for network learning, and the isometric regularization loss is defined as follows: where F and are the feature representations learned by the encoder in the dominant and auxiliary branches, respectively; down(·) denotes a down-sampling operation, and the L1 norm is used to measure the consistency of the feature representations; C3. Loss function calculation and parameter update, in each epoch, first get a batch of training data, then calculate three loss functions according to the steps of C2a-c and get the final multi-level contrastive loss function Where λ1, λ2, λ3 represent weight coefficients, the whole pre-training is carried out in a single stage; the model parameters are updated using the stochastic gradient descent method, and C1-C2 are repeated until the total loss function of the model training converges to a stable minimum value, at this time the parameter state of the encoder e(·) and part of the decoder d(·) in the dominant branch is the pre-trained model parameter, which is used to initialize the encoder and decoder of the downstream medical image segmentation task; D. For the encoder e(·) and the decoder d(·) of the medical image segmentation model built in step B, load the parameters of the pre-trained encoder e(·) and part of the decoder d p (·) in the C3 main branch, randomly initialize the parameters of the remaining part of the encoder, divide the downstream labeled data set into a training set and a test set according to a certain proportion, and train the medical image segmentation model; E. Using the test set of the downstream labeled data set, the medical image segmentation model trained in step D is used to obtain the medical image segmentation result. 2.The medical image segmentation method based on multi-level asymmetric contrastive learning according to claim 1, wherein, In step A, the three-dimensional CT / MRI medical image data set is cropped along the z-axis to obtain the corresponding two-dimensional slice image. 3.The medical image segmentation method based on multi-level asymmetric contrastive learning of claim 1, wherein, In step B, the medical image segmentation task model structure includes: (1) AttUNet with attention mechanism; (2) UCTransNet combining transformer and UNet; (3) RollingUNet integrating MLP into UNet; (4) UKAN with nonlinearity. 4.The medical image segmentation method based on multi-level asymmetric contrastive learning of claim 1, wherein, In step B, the structure of the image level projection head g(·) is a multi-layer perception MLP. 5.The medical image segmentation method based on multi-level asymmetric contrastive learning of claim 1, wherein, In step E, the segmentation result is visualized, and different colors are used to display different organ or lesion areas.
Citation Information
Patent Citations
Retinal vessel segmentation method based on dense convolution and depthwise separable convolution
CN110097554A
Histology image segmentation model based on semi-supervised learning
CN115131565A
