Medical image segmentation method based on complementary and contrasting features

Abstract

The application relates to a medical image segmentation method based on complementary and contrast features, and belongs to the medical image processing field. The method takes improving the anti-complex background interference capability as a breakthrough point, and proposes a brand-new full-supervision medical image segmentation framework. In the coding stage, a complementary feature extraction module is adopted to extract global information and local information, and a global feature optimization module is designed to strengthen the global representation capability of the model by using the local information. In the decoding stage, a contrast feature extraction module is constructed, the foreground-background contrast features are mined through the reverse attention mechanism of the contrast feature extraction module, and the anti-complex background interference capability of the model is improved. The complementary global-local features and the contrast foreground-background features are extracted, the background interference problem of the medical image is effectively solved, and the medical image segmentation effect is improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and in particular to a medical image segmentation method based on complementary and contrastive features. Background Technology

[0002] With the development of medical imaging technology, medical image data has become an important basis for doctors to diagnose diseases. Intelligent medical image segmentation technology can automatically and accurately identify and mark human organs and lesion areas, providing doctors with clearer and more intuitive information such as target location and size, significantly improving diagnostic accuracy. However, the background of medical images often has characteristics such as low contrast and clutter, which makes accurate segmentation a huge challenge.

[0003] Currently, methods based on Convolutional Neural Networks (CNNs) have achieved significant results in medical image segmentation. While CNNs excel at extracting local information, they are insufficient in global context modeling, limiting the segmentation performance of existing CNN-based methods when dealing with complex backgrounds. The Transformer architecture, due to its self-attention mechanism, exhibits a significant advantage in handling global context. Therefore, this invention combines CNNs with Transformers to simultaneously capture both global and local features, improving segmentation accuracy. Regarding background noise interference, existing methods often utilize high-level semantic information to suppress background noise, but these methods neglect the learning of background features, resulting in mediocre performance when the difference between foreground and background classes is small. Therefore, in clinical practice, there is an urgent need to develop a medical image segmentation technique that can effectively suppress complex background interference to improve the performance and reliability of medical image segmentation. Summary of the Invention

[0004] The purpose of this invention is to provide a medical image segmentation method based on complementary and contrastive features to address the problem of complex background interference. This method extracts complementary global-local features during the encoding stage and captures contrastive foreground-background features during the decoding stage, optimizing the model's feature representation capability to effectively resist interference from complex backgrounds and improve image segmentation performance.

[0005] The above-mentioned objective of the present invention is achieved through the following technical solution:

[0006] A medical image segmentation method based on complementary and contrastive features includes the following steps:

[0007] Step 1: Normalize and augment medical images using the mean (mean = [0.485, 0.456, 0.406]) and variance (std = [0.229, 0.224, 0.225]) of the ImageNet dataset as a baseline. The specific formula is as follows:

[0008] I = Image / 255

[0009] I m = (I-mean) / std

[0010] Input = DataAug(I m )

[0011] Where Image is the original image, DataAug(·) is the data augmentation strategy, and Input is the augmented image;

[0012] Step 2: Utilize the segmentation network structure S θ The specific formula for initializing the network for medical images is as follows: Among them, the segmentation network S θ The encoder extracts CFE from complementary features. θ1 Module, Global Feature Extraction (GCR) θ2 Module, Pyramid Attention Feature Extraction Module PAFE θ3 Mutual Attention (MA) θ4 Module composition;

[0013] Step 3: Sample medical images and corresponding ground truth values ​​from the training set, and feed the sampled images into the complementary feature extraction (CFE) function of the segmentation network. θ1 This module extracts global features Z using both a global branch based on a Transformer architecture and a local branch based on a CNN architecture. g and multi-scale local features {E i |1≤i≤5};

[0014] Step 4: Convert the global feature Z g The high-level semantic features E5 are fed into the global feature optimization GCR. θ2 The module enhances the ability to represent global information and adaptively generates a set of global features {G}. i |1≤i≤5}, i.e., G i =GCR θ2 (Z g ,E5);

[0015] Step 5: To address the issue of large scale variations in segmented targets, pyramid attention feature extraction (PAFE) is introduced. θ3 The module mines multi-scale contextual information and generates optimized feature E. p Formalized as Ep =PAFE θ3 (Z g );

[0016] Step Six: G i and E p Send to Mutual Attention Module MA θ4 Foreground features F are obtained through a reverse attention mechanism. i and background feature B i Then, F1 and B1 are aggregated using a dual-branch fusion block to obtain the final predicted medical image S;

[0017] Step 7: Calculate the total loss function and update the model parameters through backpropagation.

[0018] Step 8: Determine if the preset maximum number of iterations (max_iteration) has been reached. If not, return to Step 3.

[0019] Step 9: Input the medical image to be segmented into the segmentation model, perform forward propagation, and obtain the final segmentation result.

[0020] The Global Feature Optimization Module (GCR) described in step four θ2 With E5 and Z g As input, first, axial attention is applied to enhance global features on E5. Then, convolution and pooling operations are performed on G to adaptively generate a set of global features, formally described as follows:

[0021] G=AA(E5)⊙MaxP(Z g )

[0022] G i =G×AP(Act(Conv(G)))

[0023] Where AA(·) represents axial attention, MaxP(·) represents max pooling, ⊙ represents concatenation operation, Conv(·) represents a convolutional layer with 5 output channels, Act(·) is the activation function, and AP(·) is average pooling operation.

[0024] The mutual attention module MA mentioned in step six θ4 With G i and E i As input, through its cascaded dual attention block DAB i Construct foreground and background attention maps ( and The system employs a top-down feature fusion approach to obtain foreground-background contrast features for enhanced medical images. The specific formula is as follows:

[0025]

[0026] Among them, A i For DAB i The internally generated attention map uses Sig(·) to represent the Sigmoid function, f(·) to represent a combination of convolution, batch normalization, and non-linear activation functions, and U(·) to represent the bilinear upsampling function. This is used in the foreground and background attention maps. and Guided by ), the dual attention block DAB i Capable of obtaining contrast features: and When i = 5, E in the above formula i For E p Furthermore, a residual connection structure is adopted to progressively optimize foreground and background features, and F1 and B1 are aggregated through a dual-branch fusion block to obtain the prediction map S, which is described by the formula:

[0027]

[0028] S = F1 + F1 × Sig(1 - |F1 - B1|)

[0029] The total loss function described in step seven affects the final output segmented image S and global features Z. g Mutual Attention Module (MA) θ4 The foreground feature F1 and background feature B1 are supervised, and their formal expression is as follows:

[0030] L total =αL(Z) g G f )+βL(B1,G b )+γL(F1,G f )+ηL(S,G f )

[0031] G b =1-G f

[0032] Where α, β, γ and η are hyperparameters, G f It is a truth value, G b Representing the truth value of the background, L(·) = L wiou (·)+L wbce L(·) is the sum of weighted IoU and weighted binary cross-entropy loss.

[0033] The beneficial effects of this invention are as follows: This invention proposes a medical image segmentation method based on complementary and contrastive features. This method constructs a novel fully supervised medical image segmentation architecture. In the encoding stage, a complementary feature extraction module is built to effectively extract global-local features. Furthermore, a global feature optimization module is designed to mine global information from high-level semantic features using axial attention. In the decoding stage, a mutual attention module is added, which decomposes features into foreground-background contrastive features through cascaded dual attention blocks, improving the model's ability to resist complex background interference and ultimately achieving excellent segmentation performance. Attached Figure Description

[0034] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate the invention and are used to explain it, but do not constitute an undue limitation of the invention.

[0035] Figure 1 This is a flowchart of the medical image segmentation method based on complementary and contrastive features of the present invention;

[0036] Figure 2 This is a schematic diagram of the medical image segmentation model based on complementary and contrastive features of the present invention;

[0037] Figure 3 This is a schematic diagram of the global feature optimization module of the present invention;

[0038] Figure 4 This is a schematic diagram of the dual attention module of the present invention;

[0039] Figure 5 These are the polyp segmentation results for each model in this invention. Detailed Implementation

[0040] The technical solution will now be described in more detail with reference to the schematic diagrams in the embodiments of the present invention. It should be noted that the embodiments described represent only a portion of specific examples of the present invention and do not cover all possible implementations. Other implementation methods that can be derived by those skilled in the art based on these embodiments without creative effort should be considered within the scope of protection of the present invention.

[0041] Example 1:

[0042] To address the problems existing in the prior art, this embodiment provides a medical image segmentation method based on complementary and contrastive features. This method utilizes a trained model to segment medical images and specifically includes the following steps:

[0043] Step 1: Acquire medical image data;

[0044] Step 2 involves preprocessing the medical image data. First, the medical images are normalized using the mean and standard deviation of the ImageNet dataset as a benchmark. Second, data augmentation strategies such as color jittering and random rotation are applied. The medical images selected are polyp images, and the data comes from the publicly available CVC-ClinicDB and Kvasir datasets.

[0045] Step 3: Divide the medical image data into training and testing sets according to a predetermined ratio;

[0046] Step 4: Construct a medical image segmentation model based on complementary and contrastive features, and feed the training set into the model to train and optimize the model parameters;

[0047] Step 5: After training is complete, use the model loaded with the optimal weights to process the test set and obtain the image segmentation results.

[0048] Specifically, the structure of the medical image segmentation model based on complementary and contrastive features is as follows: Figure 2 As shown, specifically:

[0049] ① Construct a complementary feature extraction module. This module uses DeiT-S from the Transformer architecture as the global branch, and Res2Net-50 from the CNN architecture as the local branch, aiming to extract complementary information from both global and local dimensions.

[0050] ②Build the Global Feature Optimization (GCR) module, see [link / reference] Figure 3 The global feature optimization module enhances the model's global representation by performing axial attention on local features and generates a set of global information to compensate for the semantic dilution problem caused during the decoding stage.

[0051] ③ The bottleneck layer introduces the PAFE module to extract multi-scale information and obtain optimized features, represented as E. p =PAFE θ3 (E5)

[0052] ④ Construct a mutual attention module. This module first mines foreground and background features through five dual attention blocks, and then uses a dual-branch fusion module to achieve effective feature aggregation. Its internal structure is described in [link to internal structure]. Figure 4 .

[0053] The following explanation uses a practical application as an example, including the following steps:

[0054] Step 1: Taking the polyp segmentation task as an example, select the CVC-ClinicDB and Kvasir datasets, and divide them into training set, validation set and test set according to the ratio [0.8, 0.1, 0.1].

[0055] Step 2 involves normalizing and data augmentation preprocessing the input images. Simultaneously, the dimensions of all images are uniformly adjusted to 352×352.

[0056] Step 3: Create a medical image segmentation model based on complementary and contrastive features, train the model using the training set, and evaluate its performance using the validation set. The network uses the Adam optimizer with an initial learning rate of 1e-4, and the batch size and total number of iterations are set to 8 and 150, respectively.

[0057] Step 4: Feed the test set into the segmentation model with optimal parameters to obtain the segmentation results. The segmentation performance of the model is evaluated using the Dice coefficient and the Intersection over Union (IoU) metric.

[0058] Multiple experimental results show that the Mean Dice of this invention reaches 93.5% and 91.3% on the CVC-ClinicDB and Kvasir datasets, respectively, as shown in Table 1. Compared with FCN, U-Net, UNet++, DCRNet, and EU-Net, the proposed method improves the Mean IoU index by 11%, 9.3%, 8.9%, 3.9%, and 3.3% on the CVC-ClinicDB dataset, respectively. The segmentation results of each model are shown in... Figure 5 middle.

[0059] Table 1 shows the experimental results on the CVC-ClinicDB and Kvasir datasets.

[0060]

[0061] Example 2:

[0062] This example provides a fully supervised medical image segmentation system, including:

[0063] The acquisition module is designed to extract medical image data to be segmented.

[0064] Data preprocessing includes color jittering and random flipping. The data in this example is cell nucleus images, sourced from the 2018 Data Science Bowl.

[0065] The segmentation model building module includes a complementary feature extraction module, a global feature optimization module, a pyramid attention feature extraction module, and a mutual attention module;

[0066] The complementary feature extraction module constitutes the encoder of the model, which combines CNN and Transformer structures to extract complementary global-local features.

[0067] Global feature optimization module, such as Figure 3As shown, it utilizes axial attention to enhance the model's global representation capabilities and generates a set of global features to compensate for the semantic dilution problem in the decoding stage.

[0068] like Figure 2 As shown, the mutual attention module consists of cascaded dual attention blocks (DABs). i Composition. DAB i The input features are decomposed into contrasting foreground-background features using a reverse attention mechanism to improve the model's ability to withstand complex background interference.

[0069] During training, the medical image dataset is fed into the segmentation model, and the model parameters are updated through backpropagation.

[0070] The model's total loss function is:

[0071] L total =0.1×L(Z) g G f )+0.1×L(B1,G b )+0.1×L(F1,G f )+0.7×L(S,G f )

[0072] Where, L(·)=L wiou (·)+L wbce (·)

[0073] As shown in Table 2, this invention demonstrates advanced segmentation performance in the cell nucleus segmentation task, achieving F1-Score, Sensitivity, and Accuracy of 92.4%, 86.2%, and 93.0%, respectively. Compared to FCN, U-Net, SegNet, and PraNet, the proposed method improves the F1-Score by 3.1%, 1.5%, 1.7%, and 2.1%, respectively.

[0074] Table 2 Results of cell nucleus segmentation experiment

[0075]

[0076] In summary, this invention provides a novel fully supervised medical image segmentation method. This method captures complementary global-local features and contrasting foreground-background features to enhance the model's resilience against complex background interference. Specifically, this invention designs a complementary feature extraction module that fully leverages the local information representation capabilities of CNNs and the advantages of Transformers in global information modeling to obtain complementary global-local features. Furthermore, a global context optimization module further strengthens the model's global information representation capabilities. Finally, a mutual attention module mines contrasting features from a foreground-background perspective, enhancing the model's resilience against complex background interference.

[0077] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the invention. Various adjustments and modifications can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements to the present invention should be considered within the scope of protection of the present invention.

Claims

1. A medical image segmentation method based on complementary and contrastive features, characterized in that: Includes the following steps: Step 1: Normalize and augment medical images using the mean (mean=[0.485,0.456,0.406]) and variance (std=[0.229,0.224,0.225]) of the ImageNet dataset as a baseline. The specific formula is as follows: I = Image / 255 I m =(I-mean) / std Input=DataAug(I m ) Where Image is the original image, DataAug(·) is the data augmentation strategy, and Input is the augmented image; Step 2: Utilize the segmentation network structure The specific formula for initializing the network for medical images is as follows: Among them, segmented network The encoder is extracted from complementary features. Module, global feature extraction Module, Pyramid Attention Feature Extraction Module and mutual attention Module composition; Step 3: Sample medical images and corresponding ground truth values ​​from the training set, and feed the sampled images into the complementary feature extraction of the segmentation network. This module extracts global features by utilizing global branches based on a Transformer architecture and local branches based on a CNN architecture. and multi-scale local features ; Step 4: Global Features and high-level semantic features Feed into global feature optimization The module enhances the ability to represent global information and adaptively generates a set of global features. ,Right now ; Step 5: To address the issue of large scale variations in segmented targets, pyramid attention feature extraction is introduced. The module mines multi-scale contextual information and generates optimized features. Formal representation as ; Step Six: and Send to mutual attention module Through its cascaded dual attention block DAB i Construct foreground and background attention maps ( and ): in, For DAB i Internally generated attention map Represents the Sigmoid function. This represents a combination of convolution, batch normalization, and nonlinear activation functions. The function is a bilinear upsampling function; then, foreground features are generated. and background features Then, use the dual-branch fusion block to aggregate. and To obtain the final predictive medical image. ; Step 7: Calculate the total loss function and update the model parameters through backpropagation. Step 8: Determine if the preset maximum number of iterations (max_iteration) has been reached. If not, return to Step 3. Step 9: Input the medical image to be segmented into the segmentation model, perform forward propagation, and obtain the final segmentation result.

2. The medical image segmentation method based on complementary and contrastive features according to claim 1, characterized in that: The global feature optimization module described in step four by and As input, firstly Execute axial attention to enhance global features, and secondly, for Performing operations such as convolution and pooling, an adaptive set of global features is generated, which can be formally described as: in, For axial attention, Represents max pooling. This indicates a splicing operation. This represents a convolutional layer with 5 output channels. For activation function, It is an average pooling operation.

3. The medical image segmentation method based on complementary and contrastive features according to claim 1, characterized in that: The mutual attention module described in step six Through its cascaded dual attention block DAB i Construct foreground and background attention maps ( and It employs a top-down feature fusion approach to obtain foreground-background contrast features for enhanced medical images. and ,in When, in the above formula for Furthermore, a residual connection structure is adopted to progressively optimize foreground and background features, and these features are aggregated through a dual-branch fusion block. and The predicted image is obtained. The formula is described as follows: 。 4. The medical image segmentation method based on complementary and contrastive features according to claim 1, characterized in that: The total loss function described in step seven affects the final output segmented image. Global features Mutual Attention Module Foreground features and background features Supervision, in its formal form, is as follows: in, , , and For hyperparameters, It is a truth value. Indicates the truth value of the background. ,in It is the sum of weighted IoU and weighted binary cross-entropy loss.

Images