A colonoscopy polyp recognition method based on improved U-Net

Abstract

This invention discloses a colonoscopy polyp identification method based on an improved U-Net, belonging to the field of medical image processing technology. The method includes: acquiring a colonoscopy polyp image dataset; constructing an improved U-Net network, introducing an attention module in the deep encoder that adaptively gates and weights window attention and axial attention, and introducing an adaptive feature enhancement module that uses channel-gated weighted fusion in skip connections; training, validating, and testing the improved U-Net network using the colonoscopy polyp image dataset; and inputting the colonoscopy polyp image data to be identified into the trained improved U-Net network to obtain the identification result. This invention improves the model's ability to extract polyp features through adaptive feature-level gated weighted fusion. The adaptive feature enhancement module using channel-gated weighted fusion selectively enhances effective polyp features, improving the ability to focus on small targets.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a colonoscopy polyp identification method based on an improved U-Net. Background Technology

[0002] With the continuous development of medical imaging technology, colonoscopy images can clearly reflect the morphology and structure of the intestinal mucosa and rich pathological information. Automatic identification technology for colorectal cancer polyps plays an important role in the early screening, diagnosis and treatment, prognosis monitoring and public health prevention and control of colorectal cancer.

[0003] However, colonoscopy images present challenges in polyp identification due to the diverse shapes and colors of polyps that resemble the intestines. Furthermore, issues such as intestinal mucosal folds and occlusion between polyps and secretions further complicate the process. Additionally, polyps are characterized by their small size, multi-scale nature, and irregular shape. Pooling operations during feature extraction can reduce image resolution when expanding the receptive field, easily leading to the loss of crucial features such as polyp edges and small polyps. While U-Net can extract differential polyp features to some extent, its limited network depth and complexity prevent it from extracting deeper pathological features. Skip connections can easily overlook or confuse features of small polyps, resulting in insufficient learning of some polyp features and low accuracy in extracting features of small polyps. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a colonoscopy polyp identification method based on an improved U-Net network, which solves the technical problems of insufficient learning of some target features and low accuracy of feature extraction for small polyps when the existing U-Net is applied to polyp identification.

[0005] To achieve the above objectives, this invention proposes a colonoscopy polyp identification method based on an improved U-Net, comprising the following steps: (1) Obtain a dataset of colonoscopy polyp images; (2) Construct an improved U-Net network, introduce an attention module that adaptively gates and weights window attention and axial attention in the deep encoder, and introduce an adaptive feature enhancement module that fused channel gates in the skip connections; (3) The improved U-Net network was trained, validated, and tested using the colonoscopy polyp image dataset; (4) Input the colonoscopy polyp image data to be identified into the trained improved U-Net network to obtain the identification result.

[0006] Furthermore, the attention module specifically comprises: First, polyp features of different scales are captured by applying axial attention and window attention to the input features respectively; Then, the outputs of axial attention and window attention are concatenated in the channel dimension, and the attention weights of the two channels are obtained by sequentially passing 1×1 convolution, layer normalization, GELU activation function, 1×1 convolution and Softmax normalization. The attention weights of the two channels are then weighted and fused with the outputs of axial attention and window attention to obtain the preliminary fused features. in: Indicates preliminary fusion characteristics; The output represents the window's attention; This indicates the output of axial attention; , For dual-channel attention weights; This indicates element-wise multiplication; Then, a 1×1 convolutional layer is used to compress the number of channels of the preliminary fused features back to the number of channels of the initial input, resulting in compressed features; Finally, feature-level gating weights are calculated for the compressed features. The feature-level gating weights are then multiplied element-wise with the compressed features to perform feature filtering. Finally, the features are mapped through a linear layer to obtain the final output features of the attention module.

[0007] Furthermore, the adaptive feature enhancement module specifically comprises: First, the spatial context module performs global spatial dependency modeling on the input features to obtain context-enhanced features; then, the feature refinement module extracts deep semantic features from the input features to finally output refined features. Then, the obtained context enhancement features and refinement features are concatenated along the channel dimension, and the concatenated features are fused and linearly transformed through a 1×1 convolutional layer. After ReLU activation, the original number of channels is restored to obtain the basic enhancement features. Then, global average pooling, 1×1 convolution dimensionality reduction, ReLU activation, and 1×1 convolution dimensionality increase are sequentially performed on the basic enhanced features, and finally, Sigmoid activation is used to generate gating weights that are consistent with the number of input feature channels. Finally, the basic enhanced features are combined with the original input features through element-wise weighted fusion to further enhance the features and obtain the final output of the adaptive feature enhancement module; in: This represents the final output of the adaptive feature enhancement module; Indicates the gating weight; Indicates basic augmentation features; Represents the original input features; This represents the Sigmoid activation function; Represents a 1×1 convolution; Represents the ReLU activation function; This indicates global average pooling.

[0008] Furthermore, the improved U-Net network introduces a composite loss function, which includes Dice loss, cross-entropy loss, and Boundary-DoU loss. in: This is the total composite loss function; Cross-entropy loss; For Dice's loss; Boundary-DoU loss; These are the weighting coefficients; Category weights; Pixels in a colonoscopy polyp image Category The true label; To predict the category to which this pixel belongs The probability of; Total number of categories; This represents the total number of pixels in a single colonoscopy image of a polyp. This is used to avoid extremely small numbers with a denominator of zero; and These are the corresponding pixel values ​​of the actual polyp mask and the predicted polyp probability map in the colonoscopy polyp image, respectively; This represents the pixel intersection between the predicted polyp boundary and the actual polyp boundary in the colonoscopy image; For adaptive boundary weights.

[0009] The present invention also provides a colonoscopy polyp identification device based on an improved U-Net, comprising: Data acquisition unit: used to acquire colonoscopy polyp image datasets; The modeling module is used to build an improved U-Net network. An attention module that adaptively gates and weights window attention and axial attention is introduced in the deep encoder, and an adaptive feature enhancement module that integrates channel-gated weighted fusion is introduced in the skip connections. The training module is used to train, validate, and test the improved U-Net network using the colonoscopy polyp image dataset. The recognition module is used to input the colonoscopy polyp image data to be recognized into the trained improved U-Net network to obtain the recognition result.

[0010] The beneficial effects of this invention are: 1) Window attention and axial attention are fused through adaptive feature-level gating weighting. By establishing feature associations between local and global features, the model’s ability to extract polyp features is improved.

[0011] 2) The adaptive feature enhancement module with channel-gated weighted fusion is used to filter redundant information while retaining key features and introduce them into skip connections to selectively enhance effective polyp features and improve the ability to focus on small targets.

[0012] 3) Design a composite loss function with enhanced boundaries to alleviate the problem of missing small lesions and strengthen the learning of polyp edges to solve the problem of blurred segmentation boundaries, thereby further improving the model's ability to identify polyps that are difficult to segment. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the colonoscopy polyp identification method based on the improved U-Net according to an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of the improved U-Net network structure according to an embodiment of the present invention.

[0015] Figure 3 This is a schematic diagram of the WAFA fusion attention module structure in an embodiment of the present invention.

[0016] Figure 4 This is a schematic diagram of the adaptive feature enhancement module structure for channel-gated weighted fusion in an embodiment of the present invention.

[0017] Figure 5 This is a visualization result of the colonoscopy polyp identification method based on the improved U-Net in an embodiment of the present invention. Detailed Implementation

[0018] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings and preferred embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0019] like Figure 1 As shown, this embodiment of the invention provides a colonoscopy polyp identification method based on an improved U-Net, the specific steps of which include: S101. Obtain the colonoscopy polyp image dataset.

[0020] This invention employs three public polyp image datasets—CVC-ClinicDB, Kvasir-SEG, and CVC-300—to train and validate the algorithm. The CVC-ClinicDB dataset contains 612 polyp-annotated images with a resolution of 288×368 pixels, derived from 23 video sequences from 13 different patients, acquired through standard optical colonoscopy. The Kvasir-SEG dataset consists of 1000 polyp images, collected and validated by experienced gastroenterologists, with image resolutions ranging from 332×487 to 1920×1072 pixels.

[0021] S102. Construct an improved U-Net network, introducing an attention module in the encoder that adaptively gates and weights window attention and axial attention, and an adaptive feature enhancement module that utilizes channel-gated weighted fusion in the skip connections.

[0022] like Figure 2 As shown, the improved U-Net network (WAFA-UNet) is based on the U-Net network and introduces an attention module in its deep encoder that adaptively gates and weights window attention and axial attention. That is, an adaptive feature-level gated attention module is added after the last two encoder layers and the bottleneck layer. Figure 2 (The part marked with blue arrows in the middle); an adaptive feature enhancement module utilizing channel-gated weighted fusion is introduced into the skip connections ( Figure 2 (The part with the purple arrow in the middle); and a composite loss function is introduced at the same time.

[0023] like Figure 3 As shown, the attention module (WAFA) specifically works as follows: First, it processes the input features... X Features of polyps at different scales were captured using axial attention and window attention, respectively. X Aattn and X Wattn Then, the outputs of axial attention and window attention are concatenated along the channel dimension, and the attention weights for both channels are obtained by sequentially applying 1×1 convolution, layer normalization, GELU activation function, 1×1 convolution, and Softmax normalization. w 1 、w 2. The attention weights obtained from the two channels are used to weight and fuse the outputs of axial attention and window attention to obtain preliminary fusion features. X fused .

[0024] in: This indicates element-wise multiplication.

[0025] Then, a 1×1 convolutional layer is used to compress the number of channels of the initially fused features back to the number of channels C of the initial input feature X, resulting in a dimension of... Compression features X attn_fusion Finally, feature-level gating weights are calculated for the compressed features. These weights are then multiplied element-wise by the compressed features for feature filtering. Finally, the features are mapped through a linear layer to obtain the final output features of the attention module. X final .

[0026] like Figure 4 As shown, the adaptive feature enhancement module of channel-gated weighted fusion specifically works as follows: First, the input features are processed by the spatial context module. X Perform global spatial dependency modeling to obtain context-enhanced features. X ctx The input features are refined using the feature refinement module. X Extract deep semantic features and finally output refined features. X refine Then, the obtained context-enhanced features X ctx With refined features X refine The concatenated features are stitched along the channel dimension, and the stitched features are fused and linearly transformed through a 1×1 convolutional layer, and then restored to the original number of channels after ReLU activation. C To obtain basic enhanced features X 1. Further enhance the basic features. X 1. Perform global average pooling to compress the spatial dimension and obtain X pool and make X pool The input features are sequentially processed through 1×1 convolution for dimensionality reduction, ReLU activation, and 1×1 convolution for dimensionality increase, and finally activated by Sigmoid to generate gating weights with the same number of input feature channels. G Finally, the basic enhanced features are integrated through element-wise weighted fusion. X 1. Features of the original input X By combining these features, the final output of the adaptive feature enhancement module is obtained. X AFE .

[0027] in: This represents the final output of the adaptive feature enhancement module; Indicates the gating weight; Indicates basic augmentation features; Represents the original input features; This represents the Sigmoid activation function; Represents a 1×1 convolution; Represents the ReLU activation function; This indicates global average pooling.

[0028] An improved U-Net network was designed with a composite loss function. Dice loss was used to mitigate the problem of missing small lesions, while cross-entropy and Boundary-DoU loss were combined to enhance polyp edge learning and address the issue of blurred segmentation boundaries. CE loss, Dice loss, and Boundary-DoU loss were fused as the model's base loss function to improve the completeness of polyp region segmentation and the accuracy of edge segmentation, thus resolving the problem of similar gray levels and blurred edges between polyps and intestinal mucosa.

[0029] in: This is the total composite loss function; Cross-entropy loss; For Dice's loss; Boundary-DoU loss; These are the weighting coefficients for each loss component; Category weights; Pixels in a colonoscopy polyp image Category The true label; To predict the category to which this pixel belongs The probability of; Total number of categories; This represents the total number of pixels in a single colonoscopy image of a polyp. (10) -5 () is used to avoid extremely small numbers with a denominator of zero; and These are the corresponding pixel values ​​of the actual polyp mask and the predicted polyp probability map in the colonoscopy polyp image, respectively; This represents the pixel intersection between the predicted polyp boundary and the actual polyp boundary in the colonoscopy image; For adaptive boundary weights.

[0030] S103. The improved U-Net network is trained, validated, and tested using the colonoscopy polyp image dataset.

[0031] The experimental equipment, parameter settings, and evaluation indicators used in the training, verification, and testing processes of this invention are as follows: 1) Experimental equipment The experiment was implemented based on the deep learning framework PyTorch. The experimental platform was configured as follows: an Intel(R) Xeon(R) Gold 6530 processor with 32GB of RAM; an Nvidia A4090 dedicated graphics card with 45GB of video memory; Python packages such as Numpy and torchvision for data processing; and Matplotlib library for visualization of the recognition results.

[0032] 2) Parameter settings The model training parameters were set as follows: the dataset was divided into training, testing and validation sets in a 9:1:1 ratio; the batch size was set to 4; the number of training epochs was set to 500; the initial learning rate was set to 0.0001, and the learning rate was reduced to 0.00001 after about 300 epochs; the Adam optimizer was used to update the network training weights.

[0033] 3) Evaluation Indicators The proposed method, WAFA-Unet, when applied to polyp segmentation and recognition, uses the Dice similarity coefficient (DSC) and intersection-over-union ratio (IoU) as two core metrics, and recall (Sen), specificity (Spe), and precision as three auxiliary evaluation metrics. Sen measures the proportion of real-world polyp-positive targets successfully identified by the model; the closer the value is to 1, the lower the false negative rate. Spe measures the proportion of all pixels that are actually background that are correctly predicted by the model as background, and is generally used together with Sen to measure the model's segmentation performance. If both Sen and Spe are high, it indicates that the model neither misses nor falsely detects, indicating good performance; if Sen is high but Spe is low, it indicates a high number of false positives. Both together demonstrate the reliability of the recognition results. The calculation formula is shown below: Wherein: TP, FP, FN, and TN represent true positive, false positive, false negative, and true negative results in the segmentation results, respectively.

[0034] 4) Result Evaluation Experimental results are as follows Figure 5As shown in Table 1, the proposed method achieves experimental data on three publicly available datasets. Table 1 reveals that WAFA-UNet achieves mDSC and mIoU of 95.2% and 92.6% on the CVC-ClinicDB dataset, 97.9% and 93.8% on the CVC-300 dataset, and 92.5% and 85.0% on the Kvasir-SEG dataset. Across the three datasets, the average mIoU and average mDSC are improved by 1.9% and 2.5% compared to PraNet, and by 5.7% and 4.7% compared to CCLDNet. Therefore, the experimental results demonstrate that the proposed model exhibits good generalization ability and high segmentation performance across different datasets. In comparison, the method proposed in this embodiment can effectively identify polyps in colonoscopy images, achieving high polyp identification accuracy with low false positive and false negative rates.

[0035] Table 1 S104. Input the colonoscopy polyp image data to be identified into the trained improved U-Net network to obtain the identification result.

[0036] This invention provides a colonoscopy polyp identification device based on an improved U-Net, comprising: Data acquisition unit: used to acquire colonoscopy polyp image datasets and label text data.

[0037] The modeling module is used to build an improved U-Net network. It introduces an attention module in the deep encoder that adaptively gates and weights window attention and axial attention, and introduces an adaptive feature enhancement module that uses channel-gated weighted fusion in the skip connections.

[0038] The training module is used to train, validate, and test the improved U-Net network using the colonoscopy polyp image dataset.

[0039] The recognition module is used to input the colonoscopy polyp image data to be recognized into the trained improved U-Net network to obtain the recognition result.

[0040] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A colonoscopy polyp identification method based on an improved U-Net, characterized in that, Includes the following steps: (1) Obtain a dataset of colonoscopy polyp images; (2) Construct an improved U-Net network, introduce an attention module that adaptively gates and weights window attention and axial attention in the deep encoder, and introduce an adaptive feature enhancement module that fused channel gates in the skip connections; (3) The improved U-Net network was trained, validated, and tested using the colonoscopy polyp image dataset; (4) Input the colonoscopy polyp image data to be identified into the trained improved U-Net network to obtain the identification result.

2. The colonoscopy polyp identification method based on improved U-Net according to claim 1, characterized in that, The attention module is specifically: First, polyp features of different scales are captured by applying axial attention and window attention to the input features respectively; Then, the outputs of axial attention and window attention are concatenated in the channel dimension, and the attention weights of the two channels are obtained by sequentially passing 1×1 convolution, layer normalization, GELU activation function, 1×1 convolution and Softmax normalization. The attention weights of the two channels are then weighted and fused with the outputs of axial attention and window attention to obtain the preliminary fused features. in: Indicates preliminary fusion characteristics; The output represents the window's attention; This indicates the output of axial attention; , For dual-channel attention weights; This indicates element-wise multiplication; Then, a 1×1 convolutional layer is used to compress the number of channels of the preliminary fused features back to the number of channels of the initial input, resulting in compressed features; Finally, feature-level gating weights are calculated for the compressed features. The feature-level gating weights are then multiplied element-wise with the compressed features to perform feature filtering. Finally, the features are mapped through a linear layer to obtain the final output features of the attention module.

3. The colonoscopy polyp identification method based on improved U-Net according to claim 1, characterized in that, The adaptive feature enhancement module is specifically as follows: First, the spatial context module performs global spatial dependency modeling on the input features to obtain context-enhanced features; then, the feature refinement module extracts deep semantic features from the input features to finally output refined features. Then, the obtained context enhancement features and refinement features are concatenated along the channel dimension, and the concatenated features are fused and linearly transformed through a 1×1 convolutional layer. After ReLU activation, the original number of channels is restored to obtain the basic enhancement features. Then, global average pooling, 1×1 convolution dimensionality reduction, ReLU activation, and 1×1 convolution dimensionality increase are sequentially performed on the basic enhanced features, and finally, Sigmoid activation is used to generate gating weights that are consistent with the number of input feature channels. Finally, the basic enhanced features are combined with the original input features through element-wise weighted fusion to further enhance the features and obtain the final output of the adaptive feature enhancement module; in: This represents the final output of the adaptive feature enhancement module; Indicates the gating weight; Indicates basic augmentation features; Represents the original input features; This represents the Sigmoid activation function; Represents a 1×1 convolution; Represents the ReLU activation function; This indicates global average pooling.

4. The colonoscopy polyp identification method based on improved U-Net according to claim 1, characterized in that: The improved U-Net network also introduces a composite loss function, which includes Dice loss, cross-entropy loss, and Boundary-DoU loss. in: This is the total composite loss function; Cross-entropy loss; For Dice's loss; Boundary-DoU loss; These are the weighting coefficients; Category weights; Pixels in a colonoscopy polyp image Category The true label; To predict the category to which this pixel belongs The probability of; Total number of categories; This represents the total number of pixels in a single colonoscopy image of a polyp. This is used to avoid extremely small numbers with a denominator of zero; and These are the corresponding pixel values ​​of the actual polyp mask and the predicted polyp probability map in the colonoscopy polyp image, respectively; This represents the pixel intersection between the predicted polyp boundary and the actual polyp boundary in the colonoscopy image; For adaptive boundary weights.

5. A colonoscopy polyp identification device based on an improved U-Net, characterized in that, include: Data acquisition unit: used to acquire colonoscopy polyp image datasets; The modeling module is used to build an improved U-Net network. An attention module that adaptively gates and weights window attention and axial attention is introduced in the deep encoder, and an adaptive feature enhancement module that integrates channel-gated weighted fusion is introduced in the skip connections. The training module is used to train, validate, and test the improved U-Net network using the colonoscopy polyp image dataset. The recognition module is used to input the colonoscopy polyp image data to be recognized into the trained improved U-Net network to obtain the recognition result.

Images