Colon polyp image segmentation method based on shape perception and feature enhancement

Abstract

The present invention discloses a method for segmenting colon polyp images based on shape perception and feature enhancement, comprising the following steps: acquiring and dividing colon polyp images, obtaining training sets and test sets, and performing image enhancement preprocessing operations on the training sets and test sets; constructing an initial segmentation model based on a PVTv2 backbone extraction network, a shape perception module, a feature enhancement module, a parallel partial decoder, and a multimodal attention module; training the initial segmentation model through the training set after the preprocessing operation, and verifying it through the test set after the preprocessing operation, to obtain a segmentation model with adjusted weight parameters; performing image segmentation on the image to be detected through the segmentation model, and obtaining a colon polyp image. The present invention can solve the problems of under-segmentation and over-segmentation of polyps caused by the complexity of polyp size, shape, and color and the unclear boundary between polyps and mucosa, realizes the automatic segmentation of polyp images, and has good application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing based on deep learning, and in particular to a colon polyp image segmentation method based on shape perception and feature enhancement. Background Art

[0002] Polyp segmentation in colonoscopic images is a complex challenge due to various factors: (1) colon polyps vary greatly in shape, size, color, and quantity; and (2) the appearance and color of polyps and the background are very similar, so the boundary between polyps and the background is usually difficult to distinguish.

[0003] To overcome the challenge of polyp segmentation, many scholars have tried a lot of efforts and have developed various techniques for the polyp segmentation task, which can be mainly divided into two categories: (1) manual feature based methods and (2) deep learning based methods.

[0004] Early traditional methods used handcrafted features to segment polyps. These methods mainly relied on features such as color, shape, texture, and edge, and trained classifiers to distinguish polyps from surrounding normal mucosa. However, these methods can hardly capture the global contextual information of polyps and are not robust to complex scenes, so they are inefficient and inaccurate. The main reason is that polyps vary greatly in shape, size, and color, and the ability of manual feature extraction is quite limited. Therefore, it is still a long way from clinical application.

[0005] Compared with traditional methods, the colon polyp segmentation method based on deep learning can save more time and improve segmentation accuracy. Therefore, more and more people are committed to using convolutional neural networks (CNNs) for polyp segmentation tasks. Among them, the network based on encoder and decoder has shown excellent performance in the semantic segmentation of medical images, such as UNet, UNet++, ResUNet, ResUNet++, etc. The encoder is used to extract feature representations, and the decoder is used to generate segmentation masks. The jump connection can pass the complex and rich feature information in the encoder to the decoder sub-network, which not only promotes the propagation of high-level semantic features, but also enables the network to retain low-level detail information. In order to further improve the performance of the polyp segmentation network, a specific network specifically designed for polyp segmentation is also designed, such as PraNet, ACSNet, MSNet, etc. These networks introduce ideas such as reverse attention and multi-scale subtraction into the polyp segmentation problem, and the effect is very good. Although convolutional neural networks (CNNs) have made great progress in polyp segmentation, there is still a big gap from clinical application. These models have good performance for the prediction of most simple polyps, but the prediction resolution of complex polyps is not ideal, resulting in rough segmentation results and inaccurate boundaries. In addition, due to the locality of convolution operations, the network can only obtain sufficient receptive field by stacking convolution layers. The global modeling capability is limited and CNN cannot model long-distance dependencies. Summary of the invention

[0006] In order to solve the problems existing in the above-mentioned prior art, the present invention provides the following technical solutions:

[0007] A colon polyp image segmentation method based on shape perception and feature enhancement comprises the following steps:

[0008] Acquire and segment colon polyp images to obtain a training set and a test set, and perform image enhancement preprocessing operations on the training set and the test set;

[0009] Construct an initial segmentation model based on the PVTv2 backbone extraction network, shape perception module, feature enhancement module, parallel partial decoder and multimodal attention module;

[0010] The initial segmentation model is trained by the training set after the preprocessing operation, and verified by the test set after the preprocessing operation, so as to obtain the segmentation model with adjusted weight parameters;

[0011] The image to be detected is segmented by using the segmentation model to obtain a colon polyp image.

[0012] Preferably, the method for acquiring colon polyp images comprises: extracting endoscopic images of colon polyps from Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, CVC-T and ETIS.

[0013] Preferably, the image enhancement preprocessing method includes: adjusting the resolution of the images in the colon polyp image set to 352×352, and performing data enhancement operations, wherein the data enhancement operations include horizontal flipping, vertical flipping, 90-degree rotation, and random cropping with a size of 224×224.

[0014] Preferably, the method for training the initial segmentation model by using the training set after the preprocessing operation includes: extracting the multi-scale features of the colon polyp images in the training set by the PVTv2 backbone extraction network, generating four levels of features, wherein the four levels of features are respectively E 1 、E 2 、E 3 、E 4 , the feature E 1 、E 2 、E 3 Input into the shape perception module to generate the segmentation map S 1 , the feature E 2 、E 3 、E 4 Input them into the feature enhancement module respectively to obtain the enhanced feature F 2 、F 3 、F 4 , F 2 、F 3 、F 4 Input into the parallel partial decoder to generate the initial segmentation map S 5 ;

[0015] F 4 and the initial segmentation map S 5 Input to the multimodal attention module to generate feature map M 4 , the feature map M 4 With the initial segmentation map S 5 Add to generate segmentation map S 4 , F 3 and S 4 Input to the multimodal attention module to generate feature map M 3 , the feature map M 3 With the segmentation map S 4 Add to generate segmentation map S 3 , F 2 and segmentation map S 3 Input to the multimodal attention module to generate feature map M 2 , segmentation map S 1 With the feature map M2 and segmentation map S 3 Perform addition operation to generate segmentation map S 2 , the segmentation graph S 2 The final segmentation map is obtained by upsampling to the same size as the input polyp image and performing a Sigmoid operation.

[0016] Preferably, the segmentation map S is generated 1 The method includes: 1 、E 2 、E 3 The convolutional features are input into the 3×3 convolutional layer respectively to obtain the convolutional features and perform feature subtraction operations between adjacent features in sequence to obtain the feature differences of different scales at adjacent levels in the encoder. Then, based on the feature differences, the important channel features are highlighted through channel attention to obtain E 1_2 , E 2_3 , for E 1_2 and E 2_3 Perform 3×3 convolutional layers, feature subtraction, and channel attention operations to obtain E 1_3 , the E 1_2 and E 1_3 Add them together to get E 1_2_3 , and finally the feature map E 2_3 The result of 3×3 convolution and E 1_2_3 Perform channel splicing and convolution layer operations to obtain complementary enhanced features, and obtain the segmentation map S through spatial attention based on the complementary enhanced features. 1 .

[0017] Preferably, the feature enhancement module includes a first branch, a second branch, a third branch, a fourth branch and a residual branch;

[0018] The first branch includes a 1×1 convolutional layer and a cross-semantic attention module connected sequentially;

[0019] The second branch includes a 1×1 convolutional layer, a 1×3 convolutional layer, a 3×1 convolutional layer, a dilated convolution with a dilation rate of 3, and a cross-semantic attention module connected in sequence;

[0020] The third branch includes a 1×1 convolutional layer, a 1×5 convolutional layer, a 5×1 convolutional layer, a dilated convolution with a dilation rate of 5, and a cross-semantic attention module connected in sequence;

[0021] The fourth branch includes a 1×1 convolutional layer, a 1×7 convolutional layer, a 7×1 convolutional layer, a dilated convolution with a dilation rate of 7, and a cross-semantic attention module connected in sequence;

[0022] The residual strip branch includes sequentially connected 1×1 convolutional layers and non-local operations;

[0023] The first branch, the second branch, the third branch and the fourth branch are spliced, and a feature enhancement module is obtained by performing channel splicing through a 3×3 convolutional layer and a residual branch.

[0024] Preferably, the process of generating a feature map by a multimodal attention module comprises:

[0025] Calculate the previous level segmentation map S respectively i+1 , foreground attention, background attention, and boundary attention of i=2,3,4, the calculation formula is as follows:

[0026]

[0027] Where pred∈R H×W×1 represents the previous level segmentation map, S foreground Indicates foreground attention, S background represents background attention, S boundary Indicates boundary attention;

[0028] The foreground attention, background attention and boundary attention are respectively associated with the feature F i Multiply them to get three result branches, perform 1×1 and 3×3 convolution operations on the result branches respectively, and then add them to the segmentation map S i+1 Add, connect the three result branches through channels, perform squeeze excitation and 3×3 convolution operations, and finally combine them with the feature F i Add to generate feature map M i , where i=2,3,4.

[0029] Preferably, deep supervision is performed during the training process through a loss function, and the expression of the loss function is:

[0030]

[0031] in, represents the weighted IoU loss, represents the weighted BCE loss;

[0032] The total loss expression is:

[0033]

[0034] Where G represents the true label, Represents the upsampled prediction map.

[0035] The present invention has the following technical effects:

[0036] 1. The shape perception module SAM proposed in the present invention utilizes the differences between the backbone features at each level to distinguish polyps from the background, and removes background noise to extract clearer shape, boundary, texture and other detail information.

[0037] 2. The feature enhancement module FEM proposed in the present invention is used to improve the segmentation of polyps of different sizes and shapes and to select and refine features.

[0038] 3. The multimodal attention module MMAM proposed in the invention not only focuses on background information, but also expands its focus to the three areas of background, foreground and boundary, so as to cross-consider the role of these three areas in local area segmentation and achieve weighted balance, thereby improving the network's discrimination ability.

[0039] 4. The present invention adopts a deep supervision method to calculate the loss of the segmentation maps generated at each stage, accelerate the convergence of the model and improve the training efficiency of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 A schematic diagram of the flow of the colon polyp segmentation method based on shape perception and feature enhancement proposed by the present invention;

[0042] Figure 2 This is a schematic diagram of the framework structure of the colon polyp segmentation method based on shape perception and feature enhancement proposed in the present invention;

[0043] Figure 3 A schematic diagram of a shape perception module SAM of the colon polyp segmentation method based on shape perception and feature enhancement proposed in the present invention;

[0044] Figure 4 A schematic diagram of a feature enhancement module FEM of the colon polyp segmentation method based on shape perception and feature enhancement proposed by the present invention;

[0045] Figure 5 A schematic diagram of a cross-semantic attention module CSA of a colon polyp segmentation method based on shape perception and feature enhancement proposed in the present invention;

[0046] Figure 6 A schematic diagram of a parallel partial decoder PD of the colon polyp segmentation method based on shape perception and feature enhancement proposed in the present invention;

[0047] Figure 7 Schematic diagram of the multimodal attention module MMAM for the colon polyp segmentation method based on shape perception and feature enhancement proposed in the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0050] Embodiment 1

[0051] like Figure 1 The figure is a flow chart of a method for segmenting colon polyps based on shape perception and feature enhancement according to an embodiment of the present invention. The method for segmenting colon polyps based on shape perception and feature enhancement comprises the following steps:

[0052] Step S1, obtaining a public colon polyp dataset, dividing the dataset into a training set and a test set, and performing data enhancement operations on the colon polyp data;

[0053] Step S2, the model consists of a PVTv2 backbone extraction network, a shape perception module SAM, a feature enhancement module FEM, a parallel partial decoder PD, and a multimodal attention module MMAM;

[0054] Step S3, input the training set after the enhancement operation into the colon polyp segmentation method based on shape perception and feature enhancement for multi-scale training to improve the reliability of the training, set the training data set size scaling ratio to {0.75, 1.0, 1.25}, use the loss function for deep supervision, calculate the loss between the segmentation map generated in each stage and the true label, adjust the parameters, and save the weight parameter file of the epoch with the best performance when the model converges;

[0055] Step S4: import the saved weight parameter file into the model, then input the test image into the model, calculate the model performance and save the segmentation result image.

[0056] The following are detailed descriptions:

[0057] The specific process of step S1 is as follows:

[0058] Among them, the collected public colon polyp datasets include Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, ETIS and CVC-T. For the division of the dataset, 900 images in Kvasir-SEG and 550 images in CVC-ClinicDB, a total of 1450 images, are used as training sets, and the remaining 100 images in Kvasir-SEG and 62 images in CVC-ClinicDB as well as images in ETIS, CVC-ColonDB, and CVC-T are used as test datasets. The resolution of the colon polyp dataset is uniformly adjusted to 352×352, and the data enhancement operations include horizontal flipping, vertical flipping, 90-degree rotation, and random cropping of size 224×224.

[0059] The following table gives detailed information of each colon polyp dataset in this embodiment:

[0060] Dataset Resolution quantity Training samples Test samples Kvasir-SEG 332×487～1920×1072 1000 900 100 CVC-ClinicDB 384×288 612 550 62 CVC-ColonDB 574×500 380 n / a 380 ETIS 1225×966 196 n / a 196 CVC-T 574×500 60 n / a 60

[0061] The specific process of step S2 is as follows:

[0062] like Figure 2 The figure shows a schematic diagram of the framework structure of a colon polyp segmentation method based on shape perception and feature enhancement described in an embodiment of the present invention, wherein the PVTv2 backbone extraction network is used to extract multi-scale features of polyps; the shape perception module SAM utilizes the differences in features at each level of the backbone to remove noise in the colonoscopy image, extracts clearer shape features and generates a segmentation map; the feature enhancement module FEM is used to enhance and enrich the encoder feature semantic information, capture polyps of different shapes and sizes, select and refine features, and then input them into the parallel partial decoder PD to generate an initial segmentation map; the designed multimodal attention module MMAM is used to use the global feature map formed by the parallel partial decoder PD or the high-level segmentation map generated by the previous decoder as a guide map to achieve attention to the background, foreground, and boundary, so that the network pays more attention to suspicious and complex polyp areas.

[0063] The specific steps are as follows:

[0064] The PVTv2 backbone extraction network extracts multi-scale features of colon polyps and generates four levels of features, namely, E 1 、E 2 、E 3 、E 4 , low-level features contain more detailed information, so in order to capture more detailed information such as shape, texture, and boundary of polyps, the backbone features E at all levels are 1 、E 2 、E 3 Input into the shape perception module SAM to generate a segmentation map S 1 , E2 、E 3 、E 4 Input feature enhancement module FEM to generate enhanced features F 2 、F 3 、F 4 , F 2 、F 3 、F 4 It is fed into the parallel partial decoder PD to aggregate high-level features and capture global context information to generate a coarsely localized global segmentation map S 5 Next, F 4 and the initial segmentation map S 5 Input to the multimodal attention module MMAM to generate feature M 4 , M 4 Then with the initial segmentation map S 5 Add to generate segmentation map S 4 Then, use the same operation to change F 3 and S 4 Input to the multimodal attention module MMAM to generate feature map M 3 , and then with the previous level segmentation map S 4 Add to generate segmentation map S 3 Finally, the segmentation map S generated by the shape perception module SAM 1 With the feature map M 2 and segmentation map S 3 Perform addition operation to generate segmentation map S 2 , where the feature map M 2 Yes F 2 and S 3 Input to the multimodal attention module MMAM to generate the segmentation map S 2 Upsample to the same size as the input polyp image and perform a Sigmoid operation to obtain the final probability map.

[0065] Furthermore, the PVTv2 backbone extraction network is a PVTv2 pre-trained on ImageNet, where in order to adapt PVTv2 to the polyp segmentation task, the last classification layer is removed to obtain the backbone features E at each level. 1 、E 2 、E 3 、E 4 , E 1 As the lowest level feature, E 4 As the highest level feature, it contains rich semantic information.

[0066] Further, such as Figure 3FIG. 1 is a schematic diagram of a shape perception module SAM of a method for segmenting colon polyps based on shape perception and feature enhancement according to an embodiment of the present invention. The input of the shape perception module SAM is the main feature E 1 、E 2 、E 3 , respectively input into the 3×3 convolutional layer, and perform feature subtraction operations between adjacent features in sequence to obtain feature differences of different scales at adjacent levels in the encoder. Then, the difference feature maps are respectively used to highlight important channel features through channel attention CA to obtain E 1_2 , E 2_3 Next, for E 1_2 and E 2_3 Perform 3×3 convolution, feature subtraction, and channel attention CA operations to obtain E 1_3 , the obtained E 1_2 and E 1_3 Add them together to get E 1_2_3 , and finally the feature map E 2_3 The result of 3×3 convolution and E 1_2_3 Channel splicing and convolution operations are performed to generate complementary enhanced features, and spatial attention SA is applied to obtain a feature segmentation map S with detailed location information of polyps 1 .

[0067] Further, such as Figure 4 FIG. 1 is a schematic diagram of a feature enhancement module FEM of a colon polyp segmentation method based on shape perception and feature enhancement according to an embodiment of the present invention, wherein the feature enhancement module FEM includes: a first branch, a second branch, a third branch, a fourth branch, and a residual branch;

[0068] The first branch of the feature enhancement module FEM includes: sequentially connecting a 1×1 convolutional layer and a cross-semantic attention module CSA;

[0069] The second branch of the feature enhancement module FEM includes: sequentially connecting a 1×1 convolution layer, a 1×3 convolution layer, a 3×1 convolution layer, a dilated convolution with a dilation rate of 3, and a cross-semantic attention module CSA;

[0070] The third branch of the feature enhancement module FEM includes: sequentially connecting a 1×1 convolution layer, a 1×5 convolution layer, a 5×1 convolution layer, a dilated convolution with a dilation rate of 5, and a cross-semantic attention module CSA;

[0071] The fourth branch of the feature enhancement module FEM includes: sequentially connecting a 1×1 convolution layer, a 1×7 convolution layer, a 7×1 convolution layer, a dilated convolution with a dilation rate of 7, and a cross-semantic attention module CSA;

[0072] The feature enhancement module FEM residual strip branch includes: sequentially connecting a 1×1 convolutional layer and a non-local operation Non-Local;

[0073] The above four branches are concatenated, then passed through a 3×3 convolutional layer, and then channel-concatenated with the residual branch to obtain the output features.

[0074] Further, such as Figure 5 FIG. 1 is a schematic diagram of a cross-semantic attention module CSA of a method for segmenting colon polyps based on shape perception and feature enhancement according to an embodiment of the present invention. The input features of the cross-semantic attention module CSA are divided into E ic and E is , E ic Used to extract one-dimensional channel global information M ic , where E ic Perform global average pooling, then convert the resulting 2D tensor to 1D, then perform a one-dimensional convolution operation, and finally convert the resulting vector back to 2D space and perform a Sigmoid operation; E is Used to extract two-dimensional spatial information M is , for E is Perform 3×3 convolution and Sigmoid operations to combine these two parts M ic and M is Combined in an interactive way, the specific operation of the interaction is two branches. The first branch combines M is and E ic Perform matrix multiplication operation; the second branch performs matrix multiplication on E is Perform a 3×3 convolution operation and then add it to M ic Perform matrix multiplication operation; finally, splice the two branches on the channel.

[0075] Further, such as Figure 6 FIG. 1 is a schematic diagram of a parallel partial decoder PD of a method for segmenting colon polyps based on shape perception and feature enhancement according to an embodiment of the present invention. The input of the schematic diagram of the parallel partial decoder PD is the output result from the feature enhancement module FEM, which are F 2 、F 3 、F 4 , one of the branches will be F 4 Upsample by 2 to make F 4 Size and F 3 Same, F 4 The upsampling result is similar to F 3 After multiplication, add F 4 The up-sampling results are spliced ​​on the channel to get f 3_4 ; Another branch will F 3 Upsample by 2 and convert F4 The result of upsampling 4 times is similar to F 2 Multiply, then upsample and perform a 3×3 convolution operation with f 3_4 Perform channel splicing to obtain f 2_3_4 Next, f 2_3_4 Perform two 3×3 convolutions and one 1×1 convolution, and upsample the obtained feature map to the size of the input image to obtain the initial segmentation map S 5

[0076] Further, such as Figure 7 FIG. 1 is a schematic diagram of a modal attention module MMAM of a method for segmenting colon polyps based on shape perception and feature enhancement according to an embodiment of the present invention. The multimodal attention module MMAM calculates the previous level segmentation map S i+1 , foreground attention, background attention, and boundary attention of i=3,4,5, the calculation formula is as follows:

[0077]

[0078] where pred∈R H×W×1 represents the previous level segmentation map, S foreground Indicates foreground attention, S background represents background attention, S boundary Indicates boundary attention;

[0079] These three notes are respectively related to the feature F i Multiply, then perform 1×1 and 3×3 convolution operations on the resulting branches, and then add them to the segmentation map S i+1 Add, connect the three branches through channels, perform squeeze excitation SE and 3×3 convolution operations, and finally combine with feature F i Add to generate feature map M i , where i=2,3,4.

[0080] The specific process of step S3 is as follows:

[0081] The preprocessed training data set is input into the colon polyp segmentation method based on shape perception and feature enhancement for training. The initial learning rate in the hyperparameters is 0.0001, and the weight decay is also adjusted to 0.0001. The AdamW optimization algorithm is used, the batch size is 8, and the loss function is used for deep supervision. The loss between the segmentation map generated in each stage and the true label is calculated. The loss function is defined as:

[0082]

[0083] in represents the weighted IoU loss, represents the weighted BCE loss;

[0084] The total loss is:

[0085]

[0086] Where G represents the true label, represents the upsampled segmentation map;

[0087] Finally, save the weight parameter file of the epoch with the best performance until the model converges.

[0088] The specific process of step S4 is as follows:

[0089] Input the test image into the trained model, test the performance of the model with the saved weight parameters, and output and save the segmentation map.

[0090] In order to verify the effectiveness of the present invention compared with the prior art, we conducted comparative experiments with UNet, UNet++, SFA, PraNet, ACSNet, EU-Net, SANet, and MSNet technologies in the test data set. The experimental results are shown in the following table:

[0091]

[0092] From the comparison results, it can be seen that our invention has good learning and generalization capabilities compared with the above 8 existing classic methods, and has the ability to accurately segment polyp images.

[0093] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have several changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for segmenting colon polyp images based on shape perception and feature enhancement, characterized in that: The following steps are involved: Acquire and segment colon polyp images to obtain a training set and a test set, and perform image enhancement preprocessing operations on the training set and the test set; Construct an initial segmentation model based on the PVTv2 backbone extraction network, shape perception module, feature enhancement module, parallel partial decoder and multimodal attention module; The initial segmentation model is trained by the training set after the preprocessing operation, and verified by the test set after the preprocessing operation, so as to obtain the segmentation model with adjusted weight parameters; Performing image segmentation on the image to be detected by using the segmentation model to obtain a colon polyp image; The method for training the initial segmentation model by the training set after the preprocessing operation includes: extracting multi-scale features of the colon polyp image in the training set by the PVTv2 backbone extraction network, generating 4 levels of features, wherein the 4 levels of features are E1, E2, E3, and E4, inputting the features E1, E2, and E3 into the shape perception module to generate a segmentation map S1, inputting the features E2, E3, and E4 into the feature enhancement module to obtain enhanced features F2, F3, and F4, inputting F2, F3, and F4 into the parallel partial decoder to generate an initial segmentation map S5; Input F4 and the initial segmentation map S5 into the multimodal attention module to generate the feature map M4, add the feature map M4 to the initial segmentation map S5 to generate the segmentation map S4, input F3 and S4 into the multimodal attention module to generate the feature map M3, add the feature map M3 to the segmentation map S4 to generate the segmentation map S3, input F2 and the segmentation map S3 into the multimodal attention module to generate the feature map M2, add the segmentation map S1 to the feature map M2 and the segmentation map S3 to generate the segmentation map S2, upsample the segmentation map S2 to the same size as the input polyp image and perform a Sigmoid operation to obtain the final segmentation map; The process of generating feature maps through the multimodal attention module includes: Calculate the previous level segmentation map S respectively i+1 , foreground attention, background attention, and boundary attention of i=2,3,4, the calculation formula is as follows: Where pred∈R H×W×1 represents the previous level segmentation map, S foreground Indicates foreground attention, S background represents background attention, S boundary Indicates boundary attention; The foreground attention, background attention and boundary attention are respectively associated with the feature F i Multiply them to get three result branches, perform 1×1 and 3×3 convolution operations on the result branches respectively, and then add them to the segmentation map S i+1 Add, connect the three result branches through channels, perform squeeze excitation and 3×3 convolution operations, and finally combine them with the feature F i Add to generate feature map M i , where i=2,3,4.

2. The method for segmenting colon polyp images based on shape perception and feature enhancement according to claim 1, characterized in that: The method for obtaining colon polyp images includes: extracting endoscopic images of colon polyps from Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, CVC-T and ETIS.

3. The method for segmenting colon polyp images based on shape perception and feature enhancement according to claim 1, characterized in that: The image enhancement preprocessing method includes: adjusting the resolution of the images in the colon polyp image set to 352×352, and performing a data enhancement operation, wherein the data enhancement operation includes horizontal flipping, vertical flipping, 90-degree rotation, and random cropping with a size of 224×224.

4. The method for segmenting colon polyp images based on shape perception and feature enhancement according to claim 1, characterized in that: The method for generating the segmentation map S1 comprises: inputting features E1, E2, and E3 into a 3×3 convolution layer respectively, obtaining convolution features and performing feature subtraction operations between adjacent features in sequence, obtaining feature differences of different scales at adjacent levels in the encoder, and then highlighting important channel features through channel attention based on the feature differences, obtaining E 1_2 , E 2_3 , for E 1_2 and E 2_3 Perform 3×3 convolutional layers, feature subtraction, and channel attention operations to obtain E 1_3 , the E 1_2 and E 1_3 Add them together to get E 1_2_3 , and finally the feature map E 2_3 The result of 3×3 convolution and E 1_2_3 Channel splicing and convolution layer operations are performed to obtain complementary enhanced features, and segmentation map S1 is obtained through spatial attention based on the complementary enhanced features.

5. The method for segmenting colon polyp images based on shape perception and feature enhancement according to claim 1, characterized in that: The feature enhancement module includes a first branch, a second branch, a third branch, a fourth branch and a residual branch; The first branch includes a 1×1 convolutional layer and a cross-semantic attention module connected sequentially; The second branch includes a 1×1 convolutional layer, a 1×3 convolutional layer, a 3×1 convolutional layer, a dilated convolution with a dilation rate of 3, and a cross-semantic attention module connected in sequence; The third branch includes a 1×1 convolutional layer, a 1×5 convolutional layer, a 5×1 convolutional layer, a dilated convolution with a dilation rate of 5, and a cross-semantic attention module connected in sequence; The fourth branch includes a 1×1 convolutional layer, a 1×7 convolutional layer, a 7×1 convolutional layer, a dilated convolution with a dilation rate of 7, and a cross-semantic attention module connected in sequence; The residual strip branch includes sequentially connected 1×1 convolutional layers and non-local operations; The first branch, the second branch, the third branch and the fourth branch are spliced, and a feature enhancement module is obtained by performing channel splicing through a 3×3 convolutional layer and a residual branch.

6. The method for segmenting colon polyp images based on shape perception and feature enhancement according to claim 1, characterized in that: During the training process, deep supervision is performed through a loss function, and the expression of the loss function is: in, represents the weighted IoU loss, represents the weighted BCE loss; The total loss expression is: Where G represents the true label, Represents the upsampled prediction map.

Images