Transform medical image segmentation method based on local enhancement

Abstract

The invention relates to a Transform medical image segmentation method based on local enhancement, and belongs to the field of medical image processing. The method aims at improving the small target segmentation performance, and a brand-new fully-supervised medical image segmentation framework is provided. In a coding stage, a Pyramid Vision Transform is used to extract global information of different scales, and a local enhancement module with aligned features is additionally arranged at a jump joint, so that the local information representation capability of the model is improved. And in a decoding stage, a progressive local induction decoder is designed, so that the spatial details of the features are effectively recovered, and the recognition capability of a small target is improved. In addition, a mutual information loss function is introduced, so that the model pays more attention to task-related small target information in a training stage, and the influence of background noise is suppressed. According to the method, the global and local representation capability of the model is improved, the challenge of difficult feature extraction caused by limited information amount of the small target is effectively overcome, and the precision of small target segmentation in the medical image is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a Transformer medical image segmentation method based on local enhancement. Background Art

[0002] Medical image segmentation is a key technology for computer-aided diagnosis. It can accurately separate the target area (such as tumors, organs, etc.) in medical images from the background, improving the accuracy and efficiency of diagnosis. In clinical practice, early screening and detection of tumors play a vital role in improving patient survival rates, reducing treatment costs, and improving treatment outcomes. However, early tumors are often small in size and have a poor contrast with surrounding tissues, which often leads to missed diagnosis and increases the difficulty of early disease diagnosis. Therefore, in the medical community, there is an urgent need for a technology that can accurately segment small targets in order to provide doctors with more accurate diagnosis and treatment support.

[0003] In recent years, medical image segmentation technology based on Convolutional Neural Network (CNN) has made significant progress. However, CNN has an inherent limitation, that is, it is difficult to capture global information in the image. Compared with CNN, Transformer has obvious advantages in understanding complex spatial relationships in medical images due to its high efficiency in processing long-distance dependencies. However, when faced with small targets of small size and diverse shapes, Transformer-based medical image segmentation methods often find it difficult to accurately capture subtle features, affecting image segmentation performance. Therefore, it is urgent to develop a method to enhance the Transformer-based method's attention to local details, thereby improving the ability to capture small targets. Summary of the invention

[0004] The purpose of this invention is to provide a Transformer medical image segmentation method based on local enhancement to address the problem that existing methods have insufficient recognition capabilities for small targets. This method can effectively extract global features, enhance local representations, and adaptively restore spatial information to enhance the perception of small targets and effectively improve segmentation performance.

[0005] The above-mentioned object of the present invention is achieved by the following technical solutions:

[0006] A Transformer medical image segmentation method based on local enhancement includes the following steps:

[0007] Step 1: Normalize and enhance the medical images based on the mean and standard deviation of the ImageNet dataset to achieve standardized processing of image data. The data enhancement process includes random rotation, color jitter, cropping and other technologies. The formula is as follows:

[0008] Img_mean=[0.485,0.456,0.406]

[0009] Img_std = [0.229, 0.224, 0.225]

[0010] Img_data = img / 255

[0011] Img_transform=(Img_data-Img_mean) / Img_std

[0012] I=DataAug(Img_transform)

[0013] Where Img is the original image, Img_mean is the mean, Img_std is the standard deviation, DataAug(·) is the data enhancement strategy, and I is the enhanced image;

[0014] Step 2: Use the segmentation network structure Model θ Initialize the network for the enhanced image, where the encoder PVTE θ1 Using the PyramidVision Transformer loaded with pre-trained weights, the feature alignment local enhancement module FLE θ2 and progressive local induction decoder PLD θ3 The convolution operation uses Kaiming initialization operation;

[0015] Step 3: Sample medical images and their corresponding true values ​​from the training data set, and send the sampled images to the segmentation network model in step 2 θ Encoder PVTE θ1 Perform forward propagation to obtain features of four different scales, expressed as E i =PTVE θ1 (I)1≤i≤4;

[0016] Step 4: Combine the two adjacent layers of features E i and E i-1 Feed into the feature alignment local enhancement module FLE θ2 , where the local enhancement module FLE for feature alignment is configured θ2 First, align E through the semantic field flow i and E i-1, and then achieve local enhancement through dense convolution to obtain the optimized feature F i , the formal description is as follows:

[0017] Δ i-1 =Conv(E i-1 )⊙Up(Conv(E i ))

[0018]

[0019] Among them, Conv(·) is the convolution operation, Up(·) is the upsampling operation, ⊙ is the concatenation of features in the channel dimension, △ is the semantic flow field, and Warp(·) represents the deformation mapping function. is the high-resolution feature for feature alignment, and DeConv(·) represents the dense convolution operation;

[0020] Step 5: E 4 and F i Feed into the progressive local induction decoder PLD θ3 , adaptively restore the spatial information of the features and obtain the prediction map;

[0021] Step 6: Construct a loss function and update the parameters of the segmentation network through back propagation;

[0022] Step 7: Repeat steps 3 to 6 until the maximum number of iterations (max_iteration) is reached.

[0023] Step 8: Input the medical image data to be processed into the segmentation model, and obtain the final segmentation map through the forward propagation process.

[0024] The progressive local induction decoder in step 5 is based on E 4 and F i As input, three cascaded local reconstruction and refinement modules LRR are used to adaptively restore the spatial details of the features. LRR is based on the extracted reconstruction kernel Guide E 4 The upsampling process is formally described as:

[0025] κ=Soft(Conv(Up(E 4 ))⊙Conv(F 3 ))

[0026]

[0027] Among them, k represents the size of the reconstructed local feature, Soft(·) is the softmax activation function, and SA(·) represents the refinement block based on split attention, which contains spatial and channel attention. Further, in the next LRR module, in F 2 Under the guidance of 3 The spatial information generation D 2 In the same way, the final LRR module generates D with the same dimension as the original input space 1 .

[0028] The loss function described in step 6 is the mutual information loss function, which is used to guide the learning process of the model. The specific expression is as follows:

[0029] IB(Y,Z)=MaxI(Z;Y)-I(Z;X)

[0030] Where I represents mutual information, X is the input medical image, its corresponding true value is represented by Y, Z represents the deep features extracted from the input X, I(Z;Y) is estimated using the pixel position aware loss function, and I(Z;X) is estimated using the variational self-distillation loss.

[0031] The beneficial effect of the present invention is that: the present invention proposes a Transformer medical image segmentation method based on local enhancement, which constructs a new fully supervised medical image segmentation framework. In the encoding stage, the Pyramid Vision Transformer is introduced to extract global information of different scales. In view of the problem of insufficient feature representation of small targets, a local enhancement module for feature alignment is designed at the jump connection, and the discriminative local features are further mined on the basis of aligning adjacent global features with the semantic field flow. The decoder constructs a progressive local induced decoder, uses low-level features to guide the upsampling process of high-level features, and then optimizes the feature representation capability by splitting attention. In addition, the mutual information loss is introduced to enable the model to focus on task-related information and suppress useless background noise. This method enhances the representation capability of small targets and improves the segmentation performance of small targets by capturing global and local features. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide further understanding of the present invention and constitute a part of this application. The illustrative examples of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0033] Figure 1 It is a flow chart of the medical image segmentation method based on local enhancement Transformer of the present invention;

[0034] Figure 2It is a schematic diagram of the Transformer medical image segmentation model based on local enhancement of the present invention;

[0035] Figure 3 A schematic diagram of a local enhancement module for feature alignment of the present invention;

[0036] Figure 4 It is a schematic diagram of the local reconstruction and refinement module of the present invention;

[0037] Figure 5 The polyp segmentation results of each model;

[0038] Figure 6 Shown are the breast lesion segmentation results of each model. DETAILED DESCRIPTION

[0039] The technical solution will be described in more detail below in conjunction with the schematic diagram in the embodiment of the present invention. It should be noted that the embodiments only represent a part of the specific examples of the present invention and do not cover all possible implementations. Based on these embodiments, other implementation methods that can be derived by those skilled in the art without performing creative work should all be regarded as the scope of protection of the present invention.

[0040] Embodiment 1:

[0041] In view of the problems existing in the above-mentioned prior art, this embodiment provides a Transformer medical image segmentation method based on local enhancement, which is trained on a labeled data set and applies its optimal weight to achieve accurate segmentation of medical images. Specifically, the method includes the following steps:

[0042] Step 1, determining the medical image data to be segmented, wherein the medical image is selected as a colon polyp image, and the data comes from the public CVC-ClinicDB, Kvasir and CVC-ColonDB data sets;

[0043] Step 2: Normalize and enhance the medical image data. Normalization uses the mean and standard deviation of the ImageNet dataset to adjust the input image to achieve data standardization. Data enhancement includes random rotation, color jitter, and cropping.

[0044] Step 3: Divide the data set into training set and test set according to the established ratio;

[0045] Step 4: Build a Transformer medical image segmentation model based on local enhancement and use the training set to optimize the model parameters;

[0046] Step 5: After training, the model loads its optimal weights and verifies the segmentation performance of the model on the test set.

[0047] Specifically, the structure of the Transformer medical image segmentation model based on local enhancement is as follows: Figure 2 As shown,

[0048] ① Build an encoder based on the Pyramid Vision Transformer, use the Transformer's self-attention mechanism to capture global features, and introduce a spatial reduction mechanism to achieve feature extraction at different scales.

[0049] ②Construct the local enhancement module FLE for feature alignment θ2 ,like Figure 3 As shown in the figure, the local enhancement module of feature alignment mines discriminative features based on the alignment of adjacent features in the semantic field flow, and further uses dense convolution to strengthen local information.

[0050] ③Construct a progressive local induction decoder, such as Figure 2 As shown in Figure 2, the decoder consists of three cascaded local reconstruction and refinement modules LRR, whose internal structure is shown in Figure 4 LRR uses low-level features to guide the upsampling process of high-level features and uses the split attention mechanism to optimize feature representation and effectively restore the spatial details of the features.

[0051] The following is an example of actual application, which includes the following steps:

[0052] Step 1, obtain the medical image dataset and divide the dataset. Taking polyp segmentation as an example, the CVC-ClinicDB and Kvasir datasets are split into training set, validation set and test set according to the ratio of [0.8, 0.1, 0.1];

[0053] Step 2, normalize and enhance the input data, and adjust the image to 352×352;

[0054] Step 3, generate a Transformer medical image segmentation model based on local enhancement, and use the training set to optimize the model parameters. At the same time, the model performance is evaluated through the validation set. The model uses the AdamW optimizer, the initial learning rate is set to 1e-4, the batch size is 16, and the total number of iterations is 200;

[0055] Step 4: Send the test set data to the segmentation model loaded with the best training weights to obtain the segmentation results and performance. The Dice coefficient and intersection over union (IoU) are used as evaluation indicators for evaluating model performance.

[0056] The experimental results in Table 1 show that the proposed method achieves the best segmentation performance on the visible CVC-ClinicDB and Kvasir datasets. Specifically, on the CVC-ClinicDB dataset, the Mean Dice reaches 94.5% and the Mean IoU reaches 89.9%. On the Kvasir dataset, compared with U-Net, UNet++, ACSNet, SETR-PUP and TransUnet, the Mean IoU index of the proposed method is improved by 10.8%, 10.5%, 1.9%, 1.5% and 1.3% respectively.

[0057] Table 1 Experimental results on CVC-ClinicDB and Kvasir datasets

[0058]

[0059] As shown in Table 2, on the unseen CVC-ColonDB dataset, the Mean Dice of the proposed method reaches 79.5%, which is 21.7%, 19.6%, 8.6%, 2.2% and 1.4% higher than U-Net, UNet++, ACSNet, SETR-PUP and TransUnet. At the same time, the proposed method still achieves the best result in the Mean IoU evaluation indicator. Figure 5 The segmentation results of each model are shown.

[0060] Table 2 Experimental results on the CVC-ColonDB dataset

[0061]

[0062] Embodiment 2:

[0063] This example provides a fully supervised paradigm medical image segmentation system, including:

[0064] A data acquisition module, used for selecting medical image data to be segmented;

[0065] Data preprocessing, that is, using conventional methods such as normalization and data enhancement. The data of this embodiment is breast lesion images under ultrasound examination, and the data comes from BUSIS;

[0066] Segmentation network building module, including PVT-based encoder, feature-aligned local enhancement module, and progressive local induction decoder;

[0067] Based on the PVT encoder, the Transformer self-attention and space reduction mechanism are used to extract global features of different scales.

[0068] The local enhancement module for feature alignment is as follows Figure 3As shown in the figure, the module uses the semantic flow field to align adjacent features, mine discriminative features and strengthen local information to improve the model's ability to recognize small objects.

[0069] Progressive local induction decoder Figure 2 As shown in Figure 1, it includes three cascaded local reconstruction and refinement modules. The internal structure of this module can be found in Figure 4 , which uses low-level features to build a reconstruction kernel, guides the upsampling process of high-level features, and uses split attention to optimize feature representation capabilities in channel and spatial dimensions, effectively recovering the spatial detailed information of the features.

[0070] Implement model training and parameter optimization. Mutual information loss is used to supervise the model learning process, which is formally expressed as:

[0071] IB(Y,Z)=MaxI(Z;Y)-I(Z;X)

[0072] Where I represents mutual information. Specifically, I(Z;Y) uses the pixel position-aware loss function (L PPA ) is used to estimate I(Z;X) using the variational self-distillation loss (L VSD ) is estimated. That is, L total =L PPA +L VSD .

[0073] After multiple rounds of experiments, the present invention showed advanced segmentation performance in breast lesion segmentation tasks compared with other methods. On benign lesions, the Accuracy, Jaccard, Precision and Dice of the present invention reached 95.4%, 67.8%, 80.4% and 77.1%, as shown in Table 3. Compared with U-Net, AttentionU-Net, SegNet, PraNet and CPF-Net, the Jaccard of the proposed method increased by 11.5%, 9.7%, 4.1%, 6.4% and 2.6%, and the Dice increased by 10.1%, 8.9%, 6.4%, 4% and 3.3%. The specific breast lesion segmentation results are shown in the figure. Figure 6 shown.

[0074] Table 3 Breast lesion experimental results

[0075]

[0076] In summary, the present invention provides a new Transformer-based fully supervised medical image segmentation method. This method effectively captures global and local information and improves the model's ability to recognize small targets. Specifically, the present invention introduces a PVT-based encoder to extract multi-scale global information; at the jump connection, a feature-aligned local enhancement module is added to improve the model's local information representation capability; a progressive local induced decoder is designed to adaptively restore spatial detailed information. In addition, the present invention uses mutual information loss to guide the model to learn small target information related to the task and suppress background noise. The introduction of the above novel modules improves the segmentation model's ability to perceive small targets and improves the accuracy of medical image segmentation.

[0077] The above contents are only preferred embodiments of the present invention and are not intended to limit the scope of the present invention. For those skilled in the art, the present invention can be adjusted and modified in various ways. Any modification, equivalent substitution or improvement of the present invention should be considered to be within the scope of protection of the present invention.

Claims

1. A Transformer medical image segmentation method based on local enhancement, characterized in that: The steps include: Step 1: Normalize and enhance the medical images based on the mean and standard deviation of the ImageNet dataset to achieve standardized processing of image data. The data enhancement process includes random rotation, color jitter, cropping and other technologies. The formula is as follows: Img_mean=[0.485,0.456,0.406] Img_std = [0.229, 0.224, 0.225] Img_data = img / 255 Img_transform=(Img_data-Img_mean) / Img_std I=DataAug(Img_transform) Where Img is the original image, Img_mean is the mean, Img_std is the standard deviation, DataAug(·) is the data enhancement strategy, and I is the enhanced image; Step 2: Use the segmentation network structure Model θ Initialize the network for the enhanced image, where the encoder PVTE θ1 Using the Pyramid Vision Transformer loaded with pre-trained weights, the feature alignment local enhancement module FLE θ2 and progressive local induction decoder PLD θ3 The convolution operation uses Kaiming initialization operation; Step 3: Sample medical images and their corresponding true values ​​from the training data set, and send the sampled images to the segmentation network model in step 2 θ Encoder PVTE θ1 Perform forward propagation to obtain features of four different scales, expressed as E i =PTVE θ1 (I)1≤i≤4; Step 4: Combine the two adjacent layers of features E i and E i-1 Feed into the feature alignment local enhancement module FLE θ2 , where the local enhancement module FLE for feature alignment is configured θ2 First, align E through the semantic field flow i and E i-1 , and then achieve local enhancement through dense convolution to obtain the optimized feature F i , the formal description is as follows: Δ i-1 =Conv(E i-1 )⊙Up(Conv(And i )) Among them, Conv(·) is the convolution operation, Up(·) is the upsampling operation, ⊙ is the concatenation of features in the channel dimension, △ is the semantic flow field, and Warp(·) represents the deformation mapping function. is the high-resolution feature for feature alignment, and DeConv(·) represents the dense convolution operation; Step 5: E4 and F i Feed into the progressive local induction decoder PLD θ3 , adaptively restore the spatial information of the features and obtain the prediction map; Step 6: Construct a loss function and update the parameters of the segmentation network through back propagation; Step 7: Repeat steps 3 to 6 until the maximum number of iterations (max_iteration) is reached. Step 8: Input the medical image data to be processed into the segmentation model, and obtain the final segmentation map through the forward propagation process.

2. The Transformer medical image segmentation method based on local enhancement according to claim 1, characterized in that: The progressive local induction decoder in step 5, with E4 and F i As input, three cascaded local reconstruction and refinement modules LRR are used to adaptively restore the spatial details of the features. LRR is based on the extracted reconstruction kernel The upsampling process guiding E4 is formally described as: κ=Soft(Conv(Up(E4))⊙Conv(F3)) Among them, k represents the size of the reconstructed local feature, Soft(·) is the softmax activation function, SA(·) represents the refinement block based on split attention, which contains spatial and channel attention. Further, in the next LRR module, under the guidance of F2, D2 is generated by recovering the spatial information of the high-level feature D3. In the same way, the final LRR module generates D1 with the same spatial dimension as the original input.

3. The Transformer medical image segmentation method based on local enhancement according to claim 1, characterized in that: The loss function described in step 6 is the mutual information loss function, which is used to guide the learning process of the model. The specific expression is as follows: IB(Y,Z)=MaxI(Z;Y)-I(Z;X) Where I represents mutual information, X is the input medical image, its corresponding true value is represented by Y, Z represents the deep features extracted from the input X, I(Z;Y) is estimated using the pixel position aware loss function, and I(Z;X) is estimated using the variational self-distillation loss.

Images