Polyp image segmentation system with uncertainty enhanced contextual attention network

Abstract

The present application provides a polyp image segmentation system based on uncertainty enhanced context attention network, comprising: constructing a polyp image dataset; constructing a polyp image segmentation model based on the UACANet network model; training and testing the polyp image segmentation model based on the dataset; and performing polyp image segmentation based on the tested polyp image segmentation model. The present application can solve the problems of the existing polyp segmentation algorithm, such as the unclear polyp boundary segmentation of the backbone network, the difficulty in detection and high missing rate caused by the difference in appearance size, the insufficient expression ability of the model, and even the weak generalization ability of the model, the deepening of the model depth, and the increasing of the parameter quantity.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image segmentation of computer vision, and particularly relates to a polyp image segmentation system of uncertainty-enhanced context attention network. BACKGROUND

[0002] The most common cancer sites in the digestive system are esophagus, stomach and colon, especially the colon. Scientific research shows that timely detection and treatment can effectively control the occurrence of cancer before the lesion occurs in the colon. Colorectal cancer is developed from normal mucosa to adenoma through long-term accumulation of mutations, and the tumor is generally polypoid, and there are also some sessile or flat. In some cases, adenoma will invade the submucosa and develop into malignant tumor. Although the time for adenoma to develop into cancer is uncertain, it generally takes 10 years, and doctors can perform colon polyp examination during this period to assess the degree of polyp development by observing the growth of adenoma in polyps.

[0003] Colonoscopy is an effective technique for detecting colon polyps, and colon polyps are highly related to colorectal cancer. In clinical experiments, it is very important to segment polyps from colonoscopy images because it provides valuable information for diagnosis and surgery. Accurate polyp segmentation is a very challenging task for two reasons: (I) the same type of polyp has different sizes, colors and textures; and (II) the boundary between the polyp and its surrounding mucosa is not sharp. Therefore, some polyps may be missed during the colonoscopy detection process in the clinic, resulting in a high polyp miss rate of up to 6%. The size of the polyp and the severity of the lesion also affect the polyp miss rate, and it is difficult for doctors to find small polyps with a diameter of 6-9 mm or tiny polyps with a diameter less than or equal to 5 mm, and some intestinal complications also affect polyp detection. Using computer vision technology and deep learning for polyp segmentation can assist doctors in diagnosis, which has certain social value.

[0004] With the development of human deep learning and artificial intelligence, deep learning methods gradually appear on the stage of medical image segmentation and gradually become the current hot spot. In the early polyp segmentation method, it mainly depends on the hand-crafted features of polyps, such as color, size, texture, appearance or the combination of these features. These methods are usually trained by a classifier to distinguish polyps from their surroundings. However, these methods have a high false negative rate. The polyp segmentation method based on deep learning can well solve the problems brought by traditional methods, but the existing polyp segmentation method based on deep learning has a complex model, too many parameters and not obvious effect. The contrast between polyps and their surrounding mucosa is low, making it very difficult to identify / segment the target. For the polyp segmentation task, the latest convolutional neural network method is to generate a probability map using a deconvolution layer, replace the fully connected layer with a deconvolution layer, and use the information of the previous layer to improve the segmentation accuracy. In this category, fully convolutional networks and U-Net networks are two main methods, but due to the unclear boundary of polyps and their surrounding environment and the complex structure, the segmentation effect of the U-Net network is not very good. Variants of U-Net, namely U-Net++ and ResUNet++, are also used for polyp segmentation to improve performance. With the deepening of the neural network model, the number of neurons also increases, resulting in too many parameters and the problem of being unable to clearly segment the boundary information. Subsequently, many scholars have proposed some polyp segmentation algorithms, such as PraNet proposed by Fan et al., which uses a parallel partial decoder and a reverse attention mechanism to segment polyps, but the segmentation accuracy is not high enough, which increases the complexity of the model and has a negative impact on the generalization of the model. SUMMARY

[0005] To solve the above technical problems, the polyp image segmentation system of the uncertainty enhanced context attention network is proposed, which solves the problem that the backbone network used in the existing polyp segmentation algorithm cannot clearly segment the boundary of the polyp, the size of the appearance is different, which makes it difficult to detect and has a high false negative rate, the expression ability of the model is insufficient, and even the generalization ability of the model is weak, the depth of the model is deep, and the parameter quantity is large.

[0006] To achieve the above purpose, the polyp image segmentation system of the uncertainty enhanced context attention network is provided, which comprises a first construction module, a second construction module, a training module and an output module.

[0007] The first construction module is used to construct a data set of polyp images.

[0008] The second construction module is used to construct a polyp image segmentation model based on the UACANet network model.

[0009] The training module is used to train and test the polyp image segmentation model based on the data set.

[0010] The output module is used to perform polyp image segmentation based on the tested polyp image segmentation model.

[0011] Optionally, constructing the polyp image segmentation model includes:

[0012] Based on the UACANet network model, the output of the UACANet network model is used as an attention map to guide the high-level fusion of features. A multi-scale residual inference module is designed to obtain information of the fused features and segment polyp targets.

[0013] Optionally, the polyp image segmentation model includes: a backbone network, a parallel axial attention encoder module, a parallel axial attention decoder module, an uncertainty-enhanced contextual attention module, an RFB module, and a multi-scale residual inference module;

[0014] Multi-scale features are extracted based on the backbone network, wherein the multi-scale features include: low-level features. and advanced features ; Advanced features The parallel axial attention encoder module obtains the encoded feature information from... The feature maps from both parallel axial attention encoder modules are used for the lateral fusion path, parallel axial attention decoder, and uncertainty-enhanced context attention. The two output feature maps from the parallel axial attention decoder and parallel axial attention encoder modules are concatenated and, after passing through the parallel axial attention decoder, obtain the initial saliency map of the polyp. Then, the feature maps from the parallel axial attention encoder and parallel axial attention decoder are concatenated for the first uncertainty-enhanced context attention module, and the output saliency map from the second uncertainty-enhanced context attention module is used for context guidance. The resulting output of the second uncertainty-enhanced context attention module... After upsampling, they are compared with low-level features. Perform convolution, then pass it through the RFB module to expand the receptive field and obtain the output. ; After upsampling and Perform a concatenation operation, then a 3×3 convolution, and then... The concatenation operation is performed, and the output is then subjected to a 3×3 convolution before being fed into the multi-scale residual inference module to finally obtain the segmentation result.

[0015] Optionally, the parallel axial attention encoder module includes: a first encoder unit, a second encoder unit, and a third encoder unit;

[0016] The first, second and third encoder units are respectively connected with high-level features in the backbone network, the backbone network uses Res2Net50 which is composed of 50 convolutional layers and is divided into 5 stages, namely Five stages, each stage contains a group of convolutional layers and pooling layers, three encoder units are respectively connected with high-level features ; and the outputs of the three encoder units are all connected to the parallel axial attention decoder module.

[0017] Optionally, in the parallel axial attention decoder module:

[0018] The outputs of the parallel axial attention encoder module are spliced to obtain a splicing result;

[0019] The splicing result is sequentially subjected to multiple convolutional processing to obtain the output of the parallel axial attention decoder module.

[0020] Optionally, the uncertainty-enhanced context attention module comprises:

[0021] The uncertainty-enhanced context attention module comprises two parts: uncertainty enhancement and context attention; wherein the uncertainty enhancement is used to estimate the uncertainty of the input image, so as to adjust the weight of the attention mechanism; the context attention part is used to generate an attention map for guiding the model to pay attention to which part of the image in the classification task; the uncertainty-enhanced context attention module improves the generalization ability of segmentation by combining uncertainty and context attention mechanisms.

[0022] Optionally, the RFB module comprises a first branch, a second branch, a third branch and a fourth branch;

[0023] The RFB first branch comprises a 1x1 convolutional layer;

[0024] The RFB second branch comprises a 1x1 convolutional layer, a 1x3 convolutional layer, a 3x1 convolutional layer and a first 3x3 convolutional layer connected in sequence;

[0025] The RFB third branch comprises a 1x1 convolutional layer, a 1x5 convolutional layer, a 5x1 convolutional layer and a second 3x3 convolutional layer connected in sequence;

[0026] The RFB fourth branch comprises a 1x1 convolutional layer, a 1x7 convolutional layer, a 7x1 convolutional layer and a third 3x3 convolutional layer connected in sequence.

[0027] The above four branches are connected through a 3x3 convolutional layer, and then connected with the original input to obtain the output.

[0028] Optionally, the inference module comprises: 1*1 convolution layers connected in sequence, a multi-scale residual block, 1*1 convolution layers, a multi-scale residual block and 1*1 convolution layers.

[0029] Optionally, the multi-scale residual inference module comprises: 1*1 convolution layers, a first unit MRB, 1*1 convolution layers, a second unit MRB and 1*1 convolution layers.

[0030] The first unit MRB comprises: 3*3 convolution layers connected in sequence, a regular normalization layer, a ReLU activation function layer, 3*3 deconvolution layers, a regular normalization layer and a ReLU activation function layer.

[0031] The second unit MRB comprises: 5*5 convolution layers connected in sequence, a regular normalization layer, a ReLU activation function layer, 5*5 deconvolution layers, a regular normalization layer and a ReLU activation function layer.

[0032] Optionally, the data set of polyp images comprises:

[0033] Data augmentation is performed on the polyp images.

[0034] Additional random dilation and erosion are added to the true value labels in the data augmented polyp images.

[0035] The added data is processed by flipping to obtain the data set.

[0036] Compared with the prior art, the present application has the following advantages and technical effects:

[0037] The present application constructs a polyp segmentation network model based on UACANet, and the result of high-level feature fusion prediction is used as an attention map after up-sampling to refine low-level features, and then an RFB module is used to expand the receptive field to obtain more rich features and reduce calculation. Finally, the fused features are input into an inference module to generate the final detection result. The model can use the information features in the low-level features refined by the coarse prediction map through the multi-scale strategy, and the detection effect is greatly improved on the problem of missed detection caused by low contrast between polyps and their surrounding environment and different appearances and sizes of polyps. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application and their

[0039] Figure 1 is a whole framework diagram of the embodiment of the present application;

[0040] Figure 2 is a parallel axial attention (PAA) module schematic diagram of the embodiment of the present application;

[0041] Figure 3 A parallel axial attention encoder (PAA-e) of an embodiment of the application;

[0042] Figure 4 A decoder (PAA-d) module schematic diagram of an embodiment of the application;

[0043] Figure 5 An uncertainty augmented context attention (UACA) module schematic diagram of an embodiment of the application;

[0044] Figure 6 A receptive field blocking (RFB) module structure schematic diagram of an embodiment of the application;

[0045] Figure 7 A reasoning (MSFR) module schematic diagram of an embodiment of the application;

[0046] Figure 8 A residual module (MRB) module schematic diagram in a reasoning (MSFR) module of an embodiment of the application. DETAILED DESCRIPTION

[0047] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0048] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0049] The present application proposes a polyp image segmentation system of an uncertainty augmented context attention network, comprising a first construction module, a second construction module, a training module and an output module;

[0050] The first construction module is configured to construct a polyp image dataset;

[0051] The second construction module is configured to construct a polyp image segmentation model based on a UACANet network model;

[0052] The training module is configured to train and test the polyp image segmentation model based on the dataset;

[0053] The output module is configured to perform polyp image segmentation based on the tested polyp image segmentation model.

[0054] Further, the polyp image segmentation model comprises:

[0055] Based on the UACANet network model, the output of the UACANet network model is used as an attention map to guide the high-level fusion of features, thereby achieving polyp segmentation.

[0056] To address the problems of unclear polyp boundary segmentation, difficulty in detection due to differences in appearance size, and high false negative rates in existing polyp segmentation algorithms, this invention, based on the UACANet network model, uses the output of the UACANet network model as an attention map to guide high-level feature fusion. A multi-scale residual inference module is designed to obtain information from the fused features and segment the polyp target.

[0057] Furthermore, the polyp image segmentation model includes: a backbone network, a parallel axial attention encoder module, a parallel axial attention decoder module, an uncertainty-enhanced contextual attention module, an RFB module, and an inference module;

[0058] Multi-scale features are extracted based on the backbone network, including: low-level features. and advanced features ; Advanced features A parallel axial attention encoder acquires feature maps and performs top-down and side-out fusion paths; from The feature maps from both PAA-e modules are used for the lateral fusion path, parallel axial attention decoder, and uncertainty-enhanced contextual attention; the two feature maps from PAA-d are concatenated to predict the initial saliency map of the polyp; then the feature maps from PAA-e and PAA-d are concatenated for UACA, and the output saliency map from UACA is used for contextual guidance; the resulting... After upsampling, they are compared with low-level features. Perform convolution, then pass it through the RFB module to expand the receptive field and obtain the output. ; After upsampling and Perform a concatenation operation, then a 3×3 convolution, and then... The concatenation operation is performed, and the output is then subjected to a 3×3 convolution before being fed into the inference module to obtain the final segmentation result.

[0059] Furthermore, the parallel axial attention encoder module includes: a first encoder unit, a second encoder unit, and a third encoder unit;

[0060] The first encoder unit, the second encoder unit, and the third encoder unit are respectively connected to the high-level features in the backbone network. The backbone network uses Res2Net50, which consists of 50 convolutional layers and is divided into 5 stages. Five stages, each stage contains a set of convolutional layers and pooling layers, three encoder units are respectively connected with high-level features, i.e. high-level features The outputs of the three encoder units are connected to the parallel axial attention decoder module.

[0061] Further, in the parallel axial attention decoder module:

[0062] The outputs of the parallel axial attention encoder module are spliced to obtain a splicing result;

[0063] The splicing result is sequentially subjected to multiple convolutional processing to obtain the output of the parallel axial attention decoder module.

[0064] Further, the uncertainty-enhanced context attention module comprises:

[0065] Further, the RFB module comprises: a first RFB unit, a second RFB unit, a third RFB unit and a fourth RFB unit;

[0066] The first RFB unit comprises: a 1x1 convolutional layer;

[0067] The second RFB unit comprises: a 1x1 convolutional layer, a 1x3 convolutional layer, a 3x1 convolutional layer and a first 3x3 convolutional layer connected in sequence;

[0068] The third RFB unit comprises: a 1x1 convolutional layer, a 1x5 convolutional layer, a 5x1 convolutional layer and a second 3x3 convolutional layer connected in sequence;

[0069] The fourth RFB unit comprises: a 1x1 convolutional layer, a 1x7 convolutional layer, a 7x1 convolutional layer and a third 3x3 convolutional layer connected in sequence.

[0070] Further, the inference module comprises: a 1x1 convolutional layer, a multi-scale residual block, a 1x1 convolutional layer, a multi-scale residual block and a 1x1 convolutional layer connected in sequence.

[0071] Further, the multi-scale residual block comprises: a first MRB unit and a second MRB unit;

[0072] The first MRB unit comprises: a 3x3 convolutional layer, a regular normalization layer, a ReLU activation function layer, a 3x3 deconvolutional layer, a regular normalization layer and a ReLU activation function layer connected in sequence;

[0073] The second MRB unit comprises: a 5x5 convolutional layer, a regular normalization layer, a ReLU activation function layer, a 5x5 deconvolutional layer, a regular normalization layer and a ReLU activation function layer connected in sequence.

[0074] Further, the data set of polyp images comprises:

[0075] data augmentation is performed on the polyp image;

[0076] additional random dilation and erosion is added to the true value label in the data augmented polyp image;

[0077] the added data is flipped;

[0078] the flipped data is trained.

[0079] The polyp image segmentation system based on uncertainty enhanced context attention network, the embodiment further proposes: a polyp image segmentation method based on uncertainty enhanced context attention network, comprising:

[0080] constructing a polyp image dataset;

[0081] constructing a polyp image segmentation model based on the UACANet network model;

[0082] training and testing the polyp image segmentation model based on the dataset;

[0083] segmenting the polyp image based on the tested polyp image segmentation model.

[0084] As shown in Figure 1 The polyp image segmentation algorithm based on uncertainty enhanced context attention network proposed in the embodiment mainly includes the following steps:

[0085] Step 1: construction and arrangement of the dataset;

[0086] Step 2: design of the polyp segmentation network model;

[0087] Step 3: training of the network model;

[0088] Step 4: testing of the network model;

[0089] Step 5: evaluation of the network model;

[0090] In step 1, five famous polyp segmentation datasets are selected to verify the designed model, which are Kvasir, CVC-ClinicDB, ETIS, CVC-ColonDB and CVC-300 datasets. The arranged dataset is shown in Table 1 and Table 2:

[0091] Table 1 Five polyp segmentation datasets

[0092]

[0093] Table 2 Division of training set and test set

[0094]

[0095] In step 2, a new network model is designed based on the UACANet network model. The main feature of the UACANet network model is to use PAA-e encoding, and the encoded features are forwarded to PAA-d for the initial saliency map, which is used as an initial guide map, so that the UACA module learns the remaining saliency map in addition to the initial map. PAA-e is used for top-down outflow and side fusion path. However, the overall framework of UACANet only fuses high-level features, and does not use bottom-level features. The bottom-level features contain more texture detail parts. This design uses the output results of the UACANet model as an attention map to guide the high-level fusion features, so as to better achieve accurate polyp segmentation. In order to effectively utilize multi-scale features, the output of the second UACA module (denoted as ) is used to refine the low-level features (i.e., ). The low-level features are expanded by the modified RFB module to expand the receptive field and expand the receptive field to obtain richer features and reduce the calculation. Then the features are cascaded and fed to the camouflage reasoning module (MSFR) to generate the final detection result. The module can utilize low-level features through a multi-scale strategy to obtain the final segmentation result.

[0096] In step 3, the model is trained using the training set divided from the current most representative five polyp segmentation datasets, and the model training weight is obtained.

[0097] In step 4, the model is tested using the test set of the above five datasets respectively.

[0098] In step 5, the model is evaluated using the five evaluation indexes of meanDic, meanIoU, mae, maxDic, and maxIoU.

[0099] The polyp image segmentation algorithm model structure of the above uncertainty enhanced context attention network is as follows:

[0100] Res2Net-50 is used as the backbone to extract multi-scale features, denoted as In the model of this paper, the extracted features are divided into two groups, i.e. low-level features and high-level features An additional encoder, parallel axial attention encoder (PAA-e), is used for top-down stream and lateral fusion path. Feature maps from both PAA-e modules are used for lateral fusion path, parallel axial attention decoder (PAA-d) and uncertainty augmented context attention (UACA). The two feature maps of PAA-e of PAA-d are concatenated and predict the initial saliency map of polyp. Then the feature maps from PAA-e and PAA-d are concatenated for UACA, the output saliency map from UACA is used for context guidance. The resulting After one up-sampling, the low-level feature is convolved and then passed through the RFB module to expand the receptive field output . After up-sampling, the low-level feature is concatenated, then passed through 3x3 convolution, and then concatenated with The output is then passed through 3x3 convolution and sent to the inference module (MSFR) to obtain the final segmentation result. The overall framework diagram is shown in Figure 1 .

[0101] The structures of the above PAA-e and PAA-d modules are as follows:

[0102] As shown in Figure 2 , PAA:Input: HxWxC

[0103] Vertical axial:

[0104] Branch 1: Input→1x1conv→Reshape & Transpose→WCxH;

[0105] Branch 2: Input→1x1conv→Reshape→HxWC;

[0106] (HxWC) (WCxH)→HxH (denoted as Z1);

[0107] Branch 3: Input→1x1conv→Reshape→HxWC;

[0108] Z1 (HxWC)→Reshape→HxWxC

[0109] Input⊕(HxWxC)→Z2

[0110] Horizontal axial:

[0111] Branch 4: Input→1x1conv→Reshape & Transpose→HCxW;

[0112] Branch 5: Input → 1×1conv → Reshape → W×HC;

[0113] (W×HC) (HC×W)→W×W (represented by R1);

[0114] Branch 6: Input → 1×1conv → Reshape → W×HC;

[0115] R1 (W×HC)→Reshape→H×W×C

[0116] Input⊕(H×W×C)→R2

[0117] Summing corresponding elements in the horizontal and vertical directions:

[0118] Z2⊕R2→H×W×C

[0119] Output: H×W×C

[0120] Note: ⊕ denotes matrix multiplication, and ⊕ denotes summation of corresponding elements.

[0121] like Figure 3 As shown, PAA-e:

[0122] Branch 1: Input → 1×1 convolution;

[0123] Branch 2: Input → 1×3 convolution → 3×1 convolution → 3×3 convolution (rate=3) → Axial Attention (PAA);

[0124] Branch 3: Input → 1×5 convolution → 5×1 convolution → 3×3 convolution (rate=5) → Axial Attention (PAA);

[0125] Branch 4: Input → 1×7 convolution → 7×1 convolution → 3×3 convolution (rate=7) → Axial Attention (PAA);

[0126] The results of the four branches are concatenated, and then subjected to 3×3 convolution and 1×1 convolution.

[0127] like Figure 4 As shown, PAA-d:

[0128] High-level features The results encoded by the PAA-e module were then subjected to 4× upsampling, 2× upsampling, and... The result is obtained by concatenating the outputs after PAA-e. .

[0129] → 3x3 convolution → axial attention (PAA) → 3x3 convolution → 3x3 convolution → 3x3 convolution → 1x1 convolution → output:

[0130] As Figure 5 shown, the above UACA module structure:

[0131] The computed input saliency map is denoted as m and the corresponding foreground map m f , background map m b and uncertain map m u are generated.

[0132] (1)

[0133] The foreground and background maps are computed using max operation so as to separate not only from each other but also from the uncertain region. The representative vector of the foreground map, background map and uncertain region map is first computed by aggregating the pixel representation from each region of the input feature map x as follows:

[0134] (2)

[0135] where denotes the pixel in spatial dimension. Equation 2 is implemented with matrix multiplication as shown in Figure 5 , each vector represents the most representative feature vector, thus denotes the foreground feature vector, denotes the uncertain region. Then the vector ) and each pixel from the input feature map is computed as follows:

[0136]

[0137]

[0138] where, .(3)

[0139] The context feature map is computed by the weighted sum of the three representative vectors with similarity score as follows:

[0140] .(4)

[0141] Note and are point-wise convolutions. Each pixel in the context feature map, , can be understood as three vectors a weighted average of the outputs of the four branches. The context feature map t and the input feature map x are concatenated along the channel axis and fed forward to a point-wise convolution of the final output feature map, as shown in Figure 5

[0142] As shown in Figure 6 the receptive field blocking (RFB) module structure described above:

[0143] Branch 1: input -> 1x1 convolution;

[0144] Branch 2: input -> 1x1 convolution -> 1x3 convolution -> 3x1 convolution -> 3x3 convolution (dilation = 3);

[0145] Branch 3: input -> 1x1 convolution -> 1x5 convolution -> 5x1 convolution -> 3x3 convolution (dilation = 5);

[0146] Branch 4: input -> 1x1 convolution -> 1x7 convolution -> 7x1 convolution -> 3x3 convolution (dilation = 7);

[0147] The outputs of the above four branches are spliced, and then a 3x3 convolution is performed, and the result is spliced with the 1x1 convolution result of the input x before being fed into an activation function to obtain the output result.

[0148] As shown in Figure 7 the MSFR module structure described above:

[0149] input -> 1x1 convolution -> multi-scale residual block (MRB) -> 1x1 convolution -> multi-scale residual block (MRB) -> 1x1 convolution;

[0150] In the MSFR, in order to make full use of multi-scale information, a multi-scale residual block (MRB) is added to detect local and multi-scale features. Specifically, a double-stream network is constructed, and each stream uses different convolution kernels. As shown in Figure 8

[0151] Multi-scale residual block (MRB):

[0152] Branch 1: input feature X -> 3x3 convolution -> regular normalization -> ReLU activation function -> 3x3 deconvolution -> regular normalization -> ReLU activation function;

[0153] Branch 2: input feature X -> 5x5 convolution -> regular normalization -> ReLU activation function -> 5x5 deconvolution -> regular normalization -> ReLU activation function;

[0154] The outputs of the above two branches are spliced, and then fed into a 3x3 convolution after being concatenated with the input feature X to obtain a fused multi-scale feature representation.

[0155] ​​In step 2, when training the polyp segmentation model, the following sub-steps are included:

[0156] Step 2-1) Data augmentation, using popular data augmentation techniques, including random inversion on the horizontal and vertical axes, and random picture scaling from 0.75 to 1.25.

[0157] Step 2-2) Additional random dilation and erosion are added to the true value label to enhance the model's generalization ability;

[0158] Step 2-3) The picture is also randomly flipped by 0-359 degrees, because the picture obtained during colonoscopy detection may be rotated.

[0159] Step 2-4) Train the data, use Tesla T4 GPU to train the model, set the number of single training samples to 8, that is, batchsize=8, the input picture size is 352x352, and the model input channel number is 256.

[0160] The embodiment has the following advantages:

[0161] The embodiment constructs a polyp segmentation network model based on UACANet, and the predicted result after fusing high-level features is used as an attention map to refine low-level features Q l Then, the RFB module is used to expand the receptive field to obtain more rich features and reduce the calculation. Finally, the fused features are input into the inference module to generate the final detection result. The module can use the information features in the low-level features refined by the coarse prediction map through a multi-scale strategy, and the detection effect is greatly improved on the problem of missed detection caused by low contrast between polyps and their surrounding environment and different appearances and sizes of polyps.

[0162] To prove that the model constructed in this embodiment has good segmentation effect, this embodiment compares with several polyp segmentation network models such as U-Net, UNet++, ResUNet, SFA, PraNet and UACANet, to test the segmentation effect of the model algorithm in this embodiment. The model algorithm in this embodiment is trained on the training set of Kvasir and CVC-ClinicDB two data sets. The test set of Kvasir and CVC-ClinicDB is tested and evaluated, and Table 3 shows the evaluation results of the present application. Compared with the most current method, mDice is improved by 1.0%, and mIoU is improved by 1.5%. The model is tested and evaluated by using another three data sets unrelated to the training set, and in Table 4, compared with UACANet on the most challenging data set ETIS, mDice is improved by 4.9%, and mIoU is improved by 5.4%. The segmentation effect on the unknown data set of the non-training set shows that the model of the present application has good generalization ability and good ability to segment small polyps.

[0163] Table 3 Evaluation results of the most advanced method and the present method on Kvasir and CVC-ClinicDB data sets

[0164]

[0165] Note: ↑ indicates the higher the better, and ↓ indicates the lower the better.

[0166] Table 4 Comparison of the most advanced method and the present method on ETIS, CVC-ColonDB and CVC-300 data sets

[0167]

[0168] Note: ↑ indicates the higher the better, and ↓ indicates the lower the better.

[0169] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be limited by the protection scope of the claims.

Claims

1. A polyp image segmentation system of uncertainty-augmented contextual attention network, characterized in that, include: The first building module, the second building module, the training module, and the output module; The first building module is used to build a dataset of polyp images; The second building module is used to construct a polyp image segmentation model based on the UACANet network model; The training module is used to train and test the polyp image segmentation model based on the dataset. The output module is used to perform polyp image segmentation based on the tested polyp image segmentation model; Constructing the polyp image segmentation model includes: Based on the UACANet network model, the output of the UACANet network model is used as an attention map to guide the high-level fusion of features. A multi-scale residual inference module is designed to obtain information of the fused features and segment polyp targets. The polyp image segmentation model includes: a backbone network, a parallel axial attention encoder module, a parallel axial attention decoder module, an uncertainty-enhanced contextual attention module, an RFB module, and a multi-scale residual inference module; extracting multi-scale features based on the backbone network, wherein the multi-scale features include low-level features and high-level features ; the high-level features and the parallel axial attention encoder module obtain encoded feature information, the feature maps from the two parallel axial attention encoder modules are both used for the lateral fusion path, the parallel axial attention decoder and the uncertainty enhanced context attention; the two output feature maps of the parallel axial attention decoder and the parallel axial attention encoder module are connected, and the initial saliency map of the polyp is obtained after the parallel axial attention decoder; then the feature maps from the parallel axial attention encoder module and the parallel axial attention decoder are connected for the first uncertainty enhanced context attention module, and the output saliency map from the second uncertainty enhanced context attention module is used for context guidance; the output of the second uncertainty enhanced context attention module is upsampled and then convolved with the low-level features , and then the receptive field is expanded by the RFB module to obtain ; After upsampling, the output is spliced with , and then 3x3 convolution is performed, and then spliced with , and then 3x3 convolution is performed on the output result, and then sent to the multi-scale residual reasoning module to obtain the final segmentation result.

2. The polyp image segmentation system of uncertainly augmented contextual attention network according to claim 1, characterized in that, The parallel axial attention encoder module includes: a first encoder unit, a second encoder unit, and a third encoder unit; The first, second and third encoder units are respectively connected with high-level features in the backbone network, the backbone network uses Res2Net50 which is composed of 50 convolutional layers and is divided into 5 stages, i.e. Five stages, each stage contains a group of convolutional layers and pooling layers, three encoder units are respectively connected with high-level features ; and the outputs of the three encoder units are all connected with the parallel axial attention decoder module.

3. The polyp image segmentation system of uncertainly augmented contextual attention network according to claim 1, characterized in that, In the parallel axial attention decoder module: The outputs of the parallel axial attention encoder module are spliced ​​together to obtain the splicing result; The splicing result is subjected to multiple convolution processes in sequence to obtain the output of the parallel axial attention decoder module.

4. The polyp image segmentation system of uncertainly augmented contextual attention network according to claim 1, characterized in that, The uncertainty-enhanced context attention module includes: The uncertainty-enhanced contextual attention module consists of two parts: uncertainty enhancement and contextual attention. Uncertainty enhancement is used to estimate the uncertainty of the input image, thereby adjusting the weights of the attention mechanism. The contextual attention part is used to generate an attention map to guide the model on which parts of the image to focus on in the classification task. The uncertainty-enhanced contextual attention module improves the generalization ability of segmentation by combining uncertainty and contextual attention mechanisms.

5. The polyp image segmentation system of uncertainly augmented contextual attention network according to claim 1, characterized in that, The RFB module includes: a first branch, a second branch, a third branch, and a fourth branch; The first branch of the RFB includes: a 1×1 convolutional layer; The second branch of the RFB includes: a 1×1 convolutional layer, a 1×3 convolutional layer, a 3×1 convolutional layer and a first 3×3 convolutional layer connected in sequence; The third branch of the RFB includes: a 1×1 convolutional layer, a 1×5 convolutional layer, a 5×1 convolutional layer, and a second 3×3 convolutional layer connected in sequence; The fourth branch of the RFB includes: a 1×1 convolutional layer, a 1×7 convolutional layer, a 7×1 convolutional layer and a third 3×3 convolutional layer connected in sequence; The four branches are connected and passed through a 3×3 convolutional layer, and then connected to the original input to obtain the output.

6. The polyp image segmentation system of uncertainly augmented contextual attention network according to claim 1, characterized in that, The inference module includes: a 1×1 convolutional layer, a multi-scale residual block, a 1×1 convolutional layer, a multi-scale residual block, and a 1×1 convolutional layer connected in sequence.

7. The polyp image segmentation system of uncertainly augmented contextual attention network according to claim 6, characterized in that, The multi-scale residual inference module includes: a 1×1 convolutional layer, a first unit MRB, a 1×1 convolutional layer, a second unit MRB, and a 1×1 convolutional layer; The first unit MRB comprises: 3*3 convolution layers, regular normalization layers, ReLU activation function layers, 3*3 deconvolution layers, regular normalization layers and ReLU activation function layers connected in sequence; The second unit MRB comprises: 5*5 convolution layers, regular normalization layers, ReLU activation function layers, 5*5 deconvolution layers, regular normalization layers and ReLU activation function layers connected in sequence.

8. The polyp image segmentation system of uncertainly augmented contextual attention network according to claim 1, characterized in that, The data set of the polyp image comprises: Data enhancement is performed on the polyp image; An additional random dilation and erosion is added to the true value label in the polyp image after data enhancement; The data set is obtained by performing a flipping processing on the added data.

Images