Medical image segmentation method based on complementary and contrast features

Abstract

The invention relates to a medical image segmentation method based on complementary and contrast features, and belongs to the field of medical image processing. According to the method, a brand-new fully-supervised medical image segmentation framework is put forward by taking improvement of complex background interference resistance as an entry point. In a coding stage, a complementary feature extraction module is adopted to extract global information and local information, a global feature optimization module is designed, and the global representation capability of the model is enhanced by using the local information. In the decoding stage, a comparison feature extraction module is constructed, foreground-background comparison features are mined through a reverse attention mechanism of the comparison feature extraction module, and the complex background interference resistance of the model is improved. By extracting the complementary global-local features and the compared foreground-background features, the background interference problem of the medical image is effectively solved, and the medical image segmentation effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a medical image segmentation method based on complementary and contrast features. Background Art

[0002] With the development of medical imaging technology, medical imaging data has become an important basis for doctors to diagnose diseases. Intelligent medical image segmentation technology can automatically and accurately identify and mark human organs and lesion areas, providing doctors with clearer and more intuitive information such as target location and size, significantly improving the accuracy of diagnosis. However, the background of medical images often has low contrast and clutter, making accurate segmentation a huge challenge.

[0003] At present, methods based on convolutional neural networks (CNN) have achieved remarkable results in the field of medical image segmentation. Although CNN has excellent performance in extracting local information, it is insufficient in global context modeling, which results in the existing CNN-based segmentation methods having limited segmentation effects when dealing with scenes with complex backgrounds. The Transformer structure, due to its self-attention mechanism, shows great advantages in dealing with global contexts. Based on this, the present invention combines CNN with Transformer to capture global and local features at the same time and improve the accuracy of segmentation. In response to the problem of background noise interference, existing methods often use high-level semantic information to suppress background noise, but this method ignores the learning of background features, resulting in general performance when the difference between foreground and background classes is small. Therefore, in clinical practice, there is an urgent need to develop a medical image segmentation technology that can effectively suppress complex background interference to improve the performance and reliability of medical image segmentation. Summary of the invention

[0004] The purpose of the present invention is to provide a medical image segmentation method based on complementary and contrasting features to address the problem of complex background interference. The method extracts complementary global-local features in the encoding stage, captures contrasting foreground-background features in the decoding stage, optimizes the feature representation ability of the model, effectively resists the interference of complex background, and improves image segmentation performance.

[0005] The above-mentioned object of the present invention is achieved by the following technical solutions:

[0006] A medical image segmentation method based on complementary and contrast features comprises the following steps:

[0007] Step 1: Normalize and enhance the medical images using the mean value of ImageNet dataset, mean = [0.485, 0.456, 0.406] and variance std = [0.229, 0.224, 0.225] as the benchmark. The specific formula is as follows:

[0008] I=Image / 255

[0009] I m =(I-mean) / std

[0010] Input=DataAug(I m )

[0011] Among them, Image is the original image, DataAug(·) is the data enhancement strategy, and Input is the enhanced image;

[0012] Step 2: Use the segmentation network structure S θ Initialize the network for medical images. The specific formula is: Among them, the segmentation network S θ The encoder is extracted by complementary feature extraction CFE θ1 Module, global feature extraction GCR θ2 Module, Pyramid Attention Feature Extraction Module PAFE θ3 and mutual attention MA θ4 Module composition;

[0013] Step 3: Sample medical images and corresponding true values ​​from the training set, and feed the sampled images into the complementary feature extraction (CFE) of the segmentation network. θ1 module, which uses the global branch based on the Transformer structure and the local branch based on the CNN structure to extract the global feature Z g and multi-scale local features {E i |1≤i≤5};

[0014] Step 4: Transform the global feature Z g And high-level semantic features E5 are sent to the global feature optimization GCR θ2 module, strengthens the representation ability of global information, and adaptively generates a set of global features {G i |1≤i≤5}, that is, G i =GCR θ2 (Z g ,E5);

[0015] Step 5: In order to solve the problem of large scale changes in segmented targets, pyramid attention feature extraction PAFE is introduced θ3 The module mines multi-scale context information and generates optimized features E p , formally expressed as Ep =PAFE θ3 (Z g );

[0016] Step 6: G i and E p Feed into the mutual attention module MA θ4 , obtain the foreground feature F through the reverse attention mechanism i and background features B i , and then use the dual-branch fusion block to aggregate F1 and B1 to obtain the final predicted medical image S;

[0017] Step 7: Calculate the total loss function and update the model parameters through the back-propagation mechanism;

[0018] Step 8: Determine whether the number of iterations has reached the preset maximum number of iterations (max_iteration). If not, return to step 3.

[0019] Step 9: Input the medical image to be segmented into the segmentation model and perform forward propagation to obtain the final segmentation result.

[0020] The global feature optimization module GCR described in step 4 θ2 With E5 and Z g As input, firstly, the enhanced global features of axial attention are performed on E5. Secondly, convolution, pooling and other operations are performed on G to adaptively generate a set of global features, which can be formally described as:

[0021] G=AA(E5)⊙MaxP(Z g )

[0022] G i =G×AP(Act(Conv(G)))

[0023] Among them, AA(·) is axial attention, MaxP(·) represents maximum pooling, ⊙ represents concatenation operation, Conv(·) represents a convolutional layer with 5 output channels, Act(·) is the activation function, and AP(·) is the average pooling operation;

[0024] The mutual attention module MA described in step 6 θ4 G i and E i As input, it passes through its cascaded dual attention block DAB i , construct foreground and background attention maps ( and ), and adopt a top-down feature fusion method to obtain the foreground-background contrast features of enhanced medical images. The specific formula is as follows:

[0025]

[0026] Among them, A i for DAB i Internally generated attention map, Sig(·) represents the sigmoid function, f(·) represents the combination of convolution, batch normalization and nonlinear activation function, U(·) is the bilinear upsampling function, in the foreground attention map and background attention map ( and ), the dual attention block DAB i Ability to obtain contrast features: and When i=5, E in the above formula i For E p ,Furthermore, the residual connection structure is adopted to gradually optimize the foreground features and background features, and the prediction graph S is obtained by aggregating F1 and B1 through the double-branch fusion block. The formula is described as:

[0027]

[0028] S=F1+F1×Sig(1-|F1-B1|)

[0029] The total loss function described in step 7 affects the final output segmented image S and global feature Z g , Mutual Attention Module MA θ4 The foreground feature F1 and background feature B1 are supervised, and the formal expression is as follows:

[0030] L total =αL(Z g ,G f )+βL(B1,G b )+γL(F1,G f )+ηL(S,G f )

[0031] G b =1-G f

[0032] Among them, α, β, γ and η are hyper parameters, G f is the true value, G b represents the true value of the background, L(·)=L wiou (·)+L wbce (·), where L(·) is the sum of weighted IoU and weighted binary cross entropy loss.

[0033] The beneficial effects of the present invention are as follows: the present invention proposes a medical image segmentation method based on complementary and contrast features, and the method constructs a new fully supervised medical image segmentation architecture; in the encoding stage, a complementary feature extraction module is constructed to effectively extract global-local features. In addition, a global feature optimization module is designed to use axial attention to mine the global information of high-level semantic features. In the decoding stage, a mutual attention module is added to decompose the features into foreground-background contrast features through cascaded dual attention blocks, thereby improving the model's ability to resist complex background interference and ultimately obtaining excellent segmentation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings described herein are used to provide further understanding of the present invention and constitute a part of this application. The illustrative examples of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0035] Figure 1 is a flow chart of a medical image segmentation method based on complementary and contrast features of the present invention;

[0036] Figure 2 is a schematic diagram of a medical image segmentation model based on complementary and contrast features of the present invention;

[0037] Figure 3 It is a schematic diagram of the global feature optimization module of the present invention;

[0038] Figure 4 is a schematic diagram of a dual attention module of the present invention;

[0039] Figure 5 These are the polyp segmentation results of each model of the present invention. DETAILED DESCRIPTION

[0040] The technical solution will be described in more detail below in conjunction with the schematic diagram in the embodiment of the present invention. It should be noted that the embodiments only represent a part of the specific examples of the present invention and do not cover all possible implementations. Based on these embodiments, other implementation methods that can be derived by those skilled in the art without performing creative work should all be regarded as the scope of protection of the present invention.

[0041] Embodiment 1:

[0042] In view of the problems existing in the above-mentioned prior art, this embodiment provides a medical image segmentation method based on complementary and contrast features, which uses a trained model to segment the medical image, and specifically includes the following steps:

[0043] Step 1: obtaining medical image data;

[0044] Step 2: Preprocess the medical image data. First, normalize the medical image using the mean and standard deviation of the ImageNet dataset as a benchmark. Second, perform data enhancement strategies such as color jittering and random rotation. The above medical images are polyp images, and the data comes from the public CVC-ClinicDB and Kvasir datasets.

[0045] Step 3: Divide the medical image data into training set and test set according to the established ratio;

[0046] Step 4: Build a medical image segmentation model based on complementary and contrast features, and feed the training set into the model to train and optimize the model parameters;

[0047] Step 5: After the training is completed, use the model loaded with the optimal weights to process the test set to obtain the image segmentation results.

[0048] Specifically, the structure of the medical image segmentation model based on complementary and contrast features is as follows: Figure 2 As shown, specifically:

[0049] ① Construct a complementary feature extraction module. This module uses DeiT-S in the Transformer architecture as the global branch, and uses Res2Net-50 in the CNN architecture as the local branch, aiming to extract complementary information from both global and local dimensions.

[0050] ②Build the global feature optimization module (GCR), see Figure 3 The global feature optimization module performs axial attention on local features to strengthen the global representation of the model and generate a set of global information to make up for the semantic dilution problem generated in the decoding stage.

[0051] ③ The bottleneck layer introduces the PAFE module to extract multi-scale information and obtain the optimized features, which is expressed as E p =PAFE θ3 (E5)

[0052] ④ Construct a mutual attention module, which first mines foreground and background features through five dual attention blocks, and then uses a dual-branch fusion module to achieve effective feature aggregation. Its internal structure can be found in Figure 4 .

[0053] The following is an example of actual application, which includes the following steps:

[0054] Step 1, taking the polyp segmentation task as an example, select the CVC-ClinicDB and Kvasir datasets, and divide them into training set, validation set and test set according to the ratio of [0.8, 0.1, 0.1].

[0055] Step 2: Normalize and preprocess the input image. At the same time, resize all images to 352×352.

[0056] Step 3, create a medical image segmentation model based on complementary and contrast features, train the model using the training set, and evaluate the model performance using the validation set. The network uses the Adam optimizer, with an initial learning rate of 1e-4, and the batch size and total number of iterations are set to 8 and 150, respectively.

[0057] Step 4: Send the test set to the segmentation model with the optimal parameters to obtain the segmentation results. The segmentation performance of the model is evaluated by the Dice coefficient and the intersection over union (IoU) indicator.

[0058] The results of multiple experiments show that the Mean Dice of the proposed method on the CVC-ClinicDB and Kvasir datasets reaches 93.5% and 91.3% respectively, as shown in Table 1. Compared with FCN, U-Net, UNet++, DCRNet and EU-Net, the Mean IoU index of the proposed method on the CVC-ClinicDB dataset is improved by 11%, 9.3%, 8.9%, 3.9% and 3.3% respectively. The segmentation results of each model are shown in Figure 5 middle.

[0059] Table 1 Experimental results on CVC-ClinicDB and Kvasir datasets

[0060]

[0061] Embodiment 2:

[0062] This example provides a fully supervised medical image segmentation system, including:

[0063] An acquisition module is designed to extract medical image data to be segmented;

[0064] Data preprocessing, including color jitter and random flipping. The data in this example is a cell nucleus image, and the data comes from the 2018 Data Science Bowl;

[0065] A segmentation model building module, which includes a complementary feature extraction module, a global feature optimization module, a pyramid attention feature extraction module, and a mutual attention module;

[0066] The complementary feature extraction module constitutes the encoder of the model, which combines CNN and Transformer structures and aims to extract complementary global-local features.

[0067] The global feature optimization module is as follows Figure 3As shown in the figure, it uses axial attention to strengthen the global representation ability of the model and generates a set of global features to make up for the semantic dilution problem in the decoding stage.

[0068] like Figure 2 As shown, the mutual attention module consists of a cascade of dual attention blocks DAB i Composition. DAB i The reverse attention mechanism is used to decompose the input features into contrasting foreground-background features to improve the model's ability to resist complex background interference.

[0069] During the training process, the medical image dataset is fed into the segmentation model and the model parameters are updated through the back-propagation mechanism.

[0070] The total loss function of the model is:

[0071] L total =0.1×L(Z g ,G f )+0.1×L(B1,G b )+0.1×L(F1,G f )+0.7×L(S,G f )

[0072] Where L(·)=L wiou (·)+L wbce (·)

[0073] As shown in Table 2, in the task of cell nucleus segmentation, the proposed method shows advanced segmentation performance, with F1-Score, Sensitivity and Accuracy reaching 92.4%, 86.2% and 93.0%. Compared with FCN, U-Net, SegNet and PraNet, the F1-Score of the proposed method is improved by 3.1%, 1.5%, 1.7% and 2.1%.

[0074] Table 2 Results of cell nucleus segmentation experiment

[0075]

[0076] In summary, the present invention provides a new fully supervised medical image segmentation method. This method captures complementary global-local features and contrasting foreground-background features to enhance the model's ability to resist complex background interference. Specifically, the present invention designs a complementary feature extraction module to fully utilize the local information expression capabilities of CNN and the advantages of Transformer in global information modeling to obtain complementary global-local features. In addition, the global context optimization module is used to further enhance the model's global information representation capabilities. Finally, the mutual attention module mines contrasting features from the foreground-background perspective to enhance the model's ability to resist complex background interference.

[0077] The above contents are only preferred embodiments of the present invention and are not intended to limit the scope of the present invention. For those skilled in the art, the present invention can be adjusted and modified in various ways. Any modification, equivalent substitution or improvement of the present invention should be considered to be within the scope of protection of the present invention.

Claims

1. A medical image segmentation method based on complementary and contrast features, characterized in that: The steps include: Step 1: Normalize and enhance the medical images using the mean value of ImageNet dataset, mean = [0.485, 0.456, 0.406] and variance std = [0.229, 0.224, 0.225] as the benchmark. The specific formula is as follows: I=Image / 255 I m =(I-mean) / std Input=DataAug(I m ) Among them, Image is the original image, DataAug(·) is the data enhancement strategy, and Input is the enhanced image; Step 2: Use the segmentation network structure S θ Initialize the network for medical images. The specific formula is: Among them, the segmentation network S θ The encoder is extracted by complementary feature extraction CFE θ1 Module, global feature extraction GCR θ2 Module, Pyramid Attention Feature Extraction Module PAFE θ3 and mutual attention MA θ4 Module composition; Step 3: Sample medical images and corresponding true values ​​from the training set, and feed the sampled images into the complementary feature extraction (CFE) of the segmentation network. θ1 module, which uses the global branch based on the Transformer structure and the local branch based on the CNN structure to extract the global feature Z g and multi-scale local features {E i |1≤i≤5}; Step 4: Transform the global feature Z g And high-level semantic features E5 are sent to the global feature optimization GCR θ2 module, strengthens the representation ability of global information, and adaptively generates a set of global features {G i |1≤i≤5}, that is, G i =GCR θ2 (Z g ,E5); Step 5: In order to solve the problem of large scale changes in segmented targets, pyramid attention feature extraction PAFE is introduced θ3 The module mines multi-scale context information and generates optimized features E p , formally expressed as E p =PAFE θ3 (Z g ); Step 6: G i and E p Feed into the mutual attention module MA θ4 , obtain the foreground feature F through the reverse attention mechanism i and background features B i , and then use the dual-branch fusion block to aggregate F1 and B1 to obtain the final predicted medical image S; Step 7: Calculate the total loss function and update the model parameters through the back-propagation mechanism; Step 8: Determine whether the number of iterations has reached the preset maximum number of iterations (max_iteration). If not, return to step 3. Step 9: Input the medical image to be segmented into the segmentation model, perform forward propagation, and obtain the final segmentation result.

2. The medical image segmentation method based on complementary and contrast features according to claim 1, characterized in that: The global feature optimization module GCR described in step 4 θ2 With E5 and Z g As input, firstly, the enhanced global features of axial attention are performed on E5. Secondly, convolution, pooling and other operations are performed on G to adaptively generate a set of global features, which can be formally described as: G=AA(E5)⊙MaxP(Z g ) G i =G×AP(Act(Conv(G))) Among them, AA(·) is axial attention, MaxP(·) represents maximum pooling, ⊙ represents the concatenation operation, Conv(·) represents the convolution layer with 5 output channels, Act(·) is the activation function, and AP(·) is the average pooling operation.

3. The medical image segmentation method based on complementary and contrast features according to claim 1, characterized in that: The mutual attention module MA described in step 6 θ4 G i and E i As input, it passes through its cascaded dual attention block DAB i , construct foreground and background attention maps ( and ), and adopt a top-down feature fusion method to obtain the foreground-background contrast features of enhanced medical images. The specific formula is as follows: Among them, A i for DAB i Internally generated attention map, Sig(·) represents the sigmoid function, f(·) represents the combination of convolution, batch normalization and nonlinear activation function, U(·) is the bilinear upsampling function, in the foreground attention map and background attention map ( and ), the dual attention block DAB i Ability to obtain contrast features: and When i=5, E in the above formula i For E p ,Furthermore, the residual connection structure is adopted to gradually optimize the foreground features and background features, and the prediction graph S is obtained by aggregating F1 and B1 through the double-branch fusion block. The formula is described as: S=F1+F1×Sig(1-|F1-B1|).

4. The medical image segmentation method based on complementary and contrast features according to claim 1, characterized in that: The total loss function described in step 7 affects the final output segmented image S and global feature Z g , Mutual Attention Module MA θ4 The foreground feature F1 and background feature B1 are supervised, and the formal expression is as follows: L total =αL(Z g ,G f )+βL(B1,G b )+γL(F1,G f )+ηL(S,G f ) G b =1-G f Among them, α, β, γ and η are hyper parameters, G f is the true value, G b represents the true value of the background, L(·)=L wiou (·)+L wbce (·), where L(·) is the sum of weighted IoU and weighted binary cross entropy loss.

Images