Gastrointestinal polyp segmentation method based on Mix Transform model

Abstract

A gastrointestinal polyp segmentation method based on a Mix Transform model relates to the technical field of image processing, and introduces an edge guidance graph in the self-attention calculation process of a Mix Transform block, thereby effectively improving the identification capability of a lesion boundary and a fine structure, and significantly improving the problem that the boundary of a traditional Transform model is fuzzy. And through explicit shunting and difference enhancement of foreground and background features, the discrimination capability of the network is significantly improved, and the segmentation stability of the model in a low-contrast region is enhanced. And through the attention gating module, the contribution ratio of different layers of features can be adaptively adjusted, redundant background information can be suppressed, and the feature response of a target area can be enhanced, so that the accuracy and robustness of a segmentation result can be effectively improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method for segmenting gastrointestinal polyps based on the Mix Transformer model. Background Technology

[0002] Achieving high-precision medical image segmentation remains a challenging task due to the prevalence of problems such as blurred boundaries, heterogeneous lesion scale, and low-contrast regions in medical images. To address this issue, a gastrointestinal polyp segmentation method based on MixTransformer (MiT) is proposed. This method uses MiT as the backbone network and introduces an edge-guided modulation mechanism into its structure to enhance the model's ability to perceive structural details and boundary information. Traditional MixTransformer relies solely on the content similarity between tokens (Query–Key similarity) when calculating self-attention, resulting in insufficient attention to image structural information. Summary of the Invention

[0003] In order to overcome the shortcomings of the above technologies, this invention provides a gastrointestinal polyp segmentation method based on the Mix Transformer model that effectively improves the model's ability to distinguish in regions with blurred boundaries and enhances the overall segmentation accuracy.

[0004] The technical solution adopted by this invention to overcome its technical problems is: A method for segmenting gastrointestinal polyps based on the Mix Transformer model, comprising: a) Obtain Zhang's gastrointestinal endoscopy images and masked images were used to obtain the dataset of original gastrointestinal endoscopy images. and mask image set The training set constituted ,in For the first Zhang's gastrointestinal images, , , For the first Zhang's gastrointestinal images The corresponding masked image is marked with gastrointestinal polyps; b) Preprocess the endoscopy and gastroscopy images and mask images in the training set by mirroring to obtain the standardized and enhanced training dataset; c) Construct an image segmentation network consisting of encoding units and decoding units; d) Input the standardized and enhanced gastrointestinal endoscopy images from the training dataset into the encoding unit of the image segmentation network, and output the feature map. ; e) Feature map The input is fed into the decoding unit of the image segmentation network, and the output is a segmented image; f) Calculate the total loss Total loss Train the image segmentation network to obtain the optimized image segmentation network; g) Obtain The test set consists of 10 gastrointestinal images. The images of gastrointestinal endoscopy are input into the optimized image segmentation network, and the output is a segmented image.

[0005] Furthermore, in step a), the Kvasir-SEG medical image dataset is obtained. Zhang's gastrointestinal images and Zhang masked images were obtained from the CVC-ClinicDB medical image dataset. Zhang's gastrointestinal images and Zhang masked image, .

[0006] Preferred, The value is 900. The value is 550.

[0007] Furthermore, step b) includes the following steps: b-1) The training set of the first Zhang's gastrointestinal images The images from the gastrointestinal endoscopy are obtained by converting them to RGB format using the cv2.cvtColor() function in the OpenCV library. The training set will be the first Zhang masked image Convert to a single-channel grayscale image using the cv2.imread(mask_path, cv2.IMREAD_GRAYSCALE) function from the OpenCV library. ; b-2) Use the cv2.resize() function in the OpenCV library to resize the gastrointestinal endoscopic images respectively. and single-channel grayscale images Gastrointestinal images scaled to 256×256 and single-channel grayscale images The torchvision.transforms.Normalize() function is used to transform gastrointestinal endoscopy images. The pixels were normalized to zero mean and unit variance to obtain gastrointestinal images. ; b-3) Gastrointestinal endoscopic images and single-channel grayscale images The gastrointestinal images were obtained by randomly flipping them vertically with a 50% probability using the transforms.RandomVerticalFlip() function in the PyTorch library. and single-channel grayscale images ; b-4) Gastrointestinal endoscopic images and single-channel grayscale images The gastrointestinal images were obtained by randomly flipping the images horizontally with a 50% probability using the transforms.RandomHorizontalFlip() function in the PyTorch library. and single-channel grayscale images ; b-5) Gastrointestinal endoscopic images and single-channel grayscale images Random noise was added using the `torch.randn_like()` function in the PyTorch library to obtain standardized and enhanced gastrointestinal images. and the normalized and enhanced masked images The standardized and enhanced colonoscopy image dataset is as follows: , The standardized and enhanced masked image set is , Standardized and enhanced colonoscopy image dataset With the normalized and enhanced masked image set as This constitutes the standardized and enhanced training dataset.

[0008] Furthermore, step d) includes the following steps: d-1) The encoding unit of the image segmentation network consists of a first improved Mix Transformer block, a second improved Mix Transformer block, a third improved Mix Transformer block, and a fourth improved Mix Transformer block; d-2) The first element in the standardized and augmented training dataset Zhang Standardized and Enhanced Endoscopic Images The input is fed into the first improved Mix Transformer block of the encoding unit. This first improved Mix Transformer block is composed of a Mix Transformer model, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula For learnable parameters, For the dot product operation, the first improved Mix Transformer block outputs the feature map. ; d-3) Feature map The input is fed into the second improved Mix Transformer block of the encoding unit. This second improved Mix Transformer block is composed of a Mix Transformer model, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula The second improved MixTransformer block outputs feature maps as learnable parameters. ; d-4) Feature map The input is fed into the third improved Mix Transformer block of the encoding unit. This third improved Mix Transformer block is composed of Mix Transformer models, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula The feature map is obtained from the output of the third improved MixTransformer block, which contains learnable parameters. ; d-5) Feature map The input is fed into the fourth improved Mix Transformer block of the encoding unit. This fourth improved Mix Transformer block is composed of Mix Transformer models, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula The fourth improved MixTransformer block outputs feature maps as learnable parameters. .

[0009] Furthermore, step e) includes the following steps: The encoding unit of the image segmentation network (e-1) consists of a first feature extraction module, a second feature extraction module, a third feature extraction module, a first attention gating module, a second attention gating module, and a feature fusion module. The first, second, and third feature extraction modules of the encoding unit each consist of a first convolutional layer, a first batch normalization (BN) layer, a first ReLU activation function, a second convolutional layer, an average pooling layer, a max pooling layer, a third convolutional layer, a second BN layer, a sigmoid activation function, a first branch, a second branch, a fourth convolutional layer, a fifth convolutional layer, a third BN layer, and a second ReLU activation function. The first, second, and third feature extraction modules... The first branch of each module consists of a convolutional layer, a batch normalization layer, and a ReLU activation function. The second branches of the first, second, and third feature extraction modules also consist of convolutional layers, batch normalization layers, and ReLU activation functions. The first and second attention gating modules of the encoding unit consist of a first branch, a second branch, a ReLU activation function, a convolutional layer, a batch normalization layer, and a Sigmoid activation function. The first branches of the first and second attention gating modules also consist of convolutional layers, batch normalization layers, and PReLU activation functions. The second branches of the first and second attention gating modules also consist of convolutional layers and batch normalization layers. e-2) Feature map The inputs are sequentially fed into the first convolutional layer, the first batch normalization (BN) layer, the first ReLU activation function, and the second convolutional layer of the first feature extraction module, and the output is the compressed feature. Compression features The input is fed into the average pooling layer of the first feature extraction module, and the output is the feature. Compression features The input is fed into the max pooling layer of the first feature extraction module, and the output is the feature. , will feature With features After the addition operation, the data is sequentially fed into the third convolutional layer, the second batch normalization (BN) layer, and the sigmoid activation function of the first feature extraction module, and the output is the foreground attention map. Feature map Foreground attention map The area outside is the background attention map. , feature map With Foreground Attention Map Perform element-wise multiplication to obtain the foreground branch. , feature map Background attention map Perform element-wise multiplication to obtain the background branch. , foreground branch The input is fed into the first branch of the first feature extraction module, and the output is the feature. , background branch The input is fed into the second branch of the first feature extraction module, and the output is the feature. , will feature With features After performing element-wise subtraction along the channel dimension, the input is fed into the fourth convolutional layer of the first feature extraction module, and the output is the feature. , will feature With features After concatenation along the channel dimension, the data is sequentially input into the fifth convolutional layer, the third batch normalization (BN) layer, and the second ReLU activation function of the first feature extraction module, and the output is the feature map. , feature map With feature map Perform residual connections to obtain feature maps. ; e-3) Feature map The inputs are sequentially fed into the first convolutional layer, the first batch normalization (BN) layer, the first ReLU activation function, and the second convolutional layer of the second feature extraction module, and the output is the compressed feature. Compression features The input is fed into the average pooling layer of the second feature extraction module, and the output is the feature. Compression features The input is fed into the max pooling layer of the second feature extraction module, and the output is the feature. , will feature With features After the addition operation, the data is sequentially fed into the third convolutional layer, the second batch normalization (BN) layer, and the sigmoid activation function of the second feature extraction module, and the output is the foreground attention map. Feature map Foreground attention map The area outside is the background attention map. , feature map With Foreground Attention Map Perform element-wise multiplication to obtain the foreground branch. , feature map Background attention map Perform element-wise multiplication to obtain the background branch. , foreground branch The input is fed into the first branch of the second feature extraction module, and the output is the feature. , background branch The input is fed into the second branch of the second feature extraction module, and the output is the feature. , will feature With features After performing element-wise subtraction along the channel dimension, the input is fed into the fourth convolutional layer of the second feature extraction module, and the output is the feature. , will feature With features After concatenation along the channel dimension, the data is sequentially fed into the fifth convolutional layer, the third batch normalization (BN) layer, and the second ReLU activation function of the second feature extraction module, and the output is the feature map. , feature map With feature map Perform residual connections to obtain feature maps. ; e-4) Feature Map The inputs are sequentially fed into the first convolutional layer, the first batch normalization (BN) layer, the first ReLU activation function, and the second convolutional layer of the third feature extraction module, and the output is the compressed feature. Compression features The input is fed into the average pooling layer of the third feature extraction module, and the output is the feature. Compression features The input is fed into the max pooling layer of the third feature extraction module, and the output is the feature. , will feature With features After the addition operation, the data is sequentially fed into the third convolutional layer, the second batch normalization (BN) layer, and the sigmoid activation function of the third feature extraction module, and the output is the foreground attention map. Feature map Foreground attention map The area outside is the background attention map. , feature map With Foreground Attention Map Perform element-wise multiplication to obtain the foreground branch. , feature map Background attention map Perform element-wise multiplication to obtain the background branch. , foreground branch The input is fed into the first branch of the third feature extraction module, and the output is the feature. , background branch The input is fed into the second branch of the third feature extraction module, and the output is the feature. , will feature With features After performing element-wise subtraction along the channel dimension, the input is fed into the fourth convolutional layer of the third feature extraction module, and the output is the feature. , will feature With features After concatenation along the channel dimension, the data is sequentially fed into the fifth convolutional layer, the third batch normalization (BN) layer, and the second ReLU activation function of the third feature extraction module, and the output is the feature map. , feature map With feature map Perform residual connections to obtain feature maps. ; e-5) Feature Map Feature maps are obtained after bilinear interpolation upsampling. , feature map The input is fed into the first branch of the first attention gating module, and the output is the feature map. , feature map The input is fed into the second branch of the first attention gating module, and the output is the feature map. , feature map With feature map After performing element-wise addition along the channel dimension, the input is fed into the ReLU activation function of the first attention-gated module, and the output is the fused feature. , will integrate features The inputs are sequentially fed into the convolutional layer, batch normalization layer, and sigmoid activation function of the first attention gating module, and the output is a single-channel gating weight map. The single-channel gating weight map With feature map Perform element-wise multiplication to obtain the weighted output features. ; e-6) Feature Map Feature maps are obtained after bilinear interpolation upsampling. , feature map The input is fed into the first branch of the second attention gating module, and the output is the feature map. , feature map The input is fed into the second branch of the second attention gating module, and the output is the feature map. , feature map With feature map After performing element-wise addition along the channel dimension, the input is fed into the ReLU activation function of the second attention-gated module, and the output is the fused feature. , will integrate features The inputs are sequentially fed into the convolutional layer, batch normalization layer, and sigmoid activation function of the second attention gating module, and the output is a single-channel gating weight map. The single-channel gating weight map With feature map Perform element-wise multiplication to obtain the weighted output features. ; The feature fusion module of the e-7 coding unit consists of a first branch, a second branch, a third branch, a convolutional layer, a batch normalization (BN) layer, and a ReLU activation function. The first, second, and third branches of the fusion module are each composed of a convolutional layer, a first BN layer, a first ReLU activation function, a depthwise separable convolutional layer, a second BN layer, and a second ReLU activation function, respectively, to fuse the feature map. Weighted output features Weighted output features Perform a stitching operation to obtain the feature map. , feature map The input is fed into the first branch of the feature fusion module, and the output is the feature map. , feature map After performing bilinear interpolation upsampling, the feature map is obtained. , feature map The input is fed into the second branch of the feature fusion module, and the output is the feature map. , feature map After performing bilinear interpolation upsampling, the feature map is obtained. , feature map The input is fed into the third branch of the feature fusion module, and the output is the feature map. , feature map After performing bilinear interpolation upsampling, the feature map is obtained. , feature map Feature map Feature map After the concatenation operation, the data is sequentially fed into the convolutional layer, batch normalization layer, and ReLU activation function of the fusion module, and the output is the feature map. , feature map With feature map The segmented image is obtained by performing residual connections.

[0010] Preferably, in step e-1), the kernel size of the first convolutional layer of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3, the kernel size of the second convolutional layer is 1×1, the kernel size of the third convolutional layer is 1×1, the kernel size of the fourth convolutional layer is 1×1, and the kernel size of the fifth convolutional layer is 3×3; the kernel size of the convolutional layer of the first branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the second branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the first branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the first attention gating module, the second attention gating module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the second branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the first attention gating module, the second attention gating module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the first branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the second branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the first branch ... The kernel size of the convolutional layer of the block is 1×1; the kernel size of the convolutional layer of the first branch of the first attention gating module and the second attention gating module is 3×3; the kernel size of the convolutional layer of the second branch of the first attention gating module and the second attention gating module is 3×3; the average pooling layer of the first branch of the feature fusion module has a scale of 2×2 and a kernel size of 1×1; the average pooling layer of the second branch of the feature fusion module has a scale of 4×4 and a kernel size of 1×1; the average pooling layer of the third branch of the feature fusion module has a scale of 6×6 and a kernel size of 1×1; and the convolutional layer size of the feature fusion module is 1×1.

[0011] Furthermore, in step f), through the first Zhang's gastrointestinal images The segmented image and the first Zhang masked image IoU loss was calculated Through formula The total loss was calculated. In the formula , All are weighting coefficients. , This represents the cross-entropy loss.

[0012] Furthermore, in step f), the SGD optimizer is used to utilize the total loss. Train the image segmentation network to obtain the optimized image segmentation network. During training, set weight_decay to 0.0001 and learning rate lr to the base learning rate of 0.01.

[0013] Furthermore, in step g), the CVC-ColonDB medical image dataset, ETIS-LaribPolypDB medical image dataset, and EndoScene medical image dataset are obtained. The test set consists of several gastrointestinal and intestinal endoscopy images.

[0014] The beneficial effects of this invention are: (1) An edge-guided modulation mechanism is introduced to enhance the model's ability to perceive structural details. This invention introduces an edge-guided map during the self-attention calculation process of the MixTransformer block. The Laplacian operator is used to extract the image edge structure, and an edge saliency mask is generated to perform weighted modulation on the Query and Key features. This mechanism enables the model to adaptively enhance the response of edge regions during the attention calculation stage, thereby effectively improving the ability to identify lesion boundaries and fine structures, and significantly improving the problem of blurred boundaries in traditional Transformer models.

[0015] (2) A semantic compression and contrastive attention mechanism is designed to achieve explicit splitting and difference enhancement of foreground and background features. A semantic compression layer is introduced in the decoding stage, extracting local and global semantic features through two-level convolution, and using foreground attention maps and background attention maps for feature splitting. Furthermore, a semantic contrastive mapping mechanism is combined to calculate the semantic difference mapping between the foreground and background, enabling the network to explicitly distinguish between target and non-target regions. This design significantly improves the network's discriminative ability and enhances the model's segmentation stability in low-contrast regions.

[0016] (3) Design an attention gating module to achieve adaptive fusion of multi-level features. An attention gating mechanism is introduced in the multi-level feature fusion stage. High-level semantic and low-level structural features are extracted through a dual-branch structure, and a single-channel gating weight map is generated to achieve dynamic weighting. This mechanism can adaptively adjust the contribution ratio of features at different levels, suppress redundant background information, and enhance the feature response of the target region, thereby effectively improving the accuracy and robustness of the segmentation results. Detailed Implementation

[0017] The present invention will be further described below.

[0018] A method for segmenting gastrointestinal polyps based on the Mix Transformer model, comprising: a) Obtain Zhang's gastrointestinal endoscopy images and masked images were used to obtain the dataset of original gastrointestinal endoscopy images. and mask image set The training set constituted ,in For the first Zhang's gastrointestinal images, , , For the first Zhang's gastrointestinal images The corresponding masked image is marked with gastrointestinal polyps.

[0019] b) Preprocess the endoscopy images and masked images in the training set by mirroring them to obtain the standardized and enhanced training dataset.

[0020] c) Construct an image segmentation network consisting of encoding units and decoding units.

[0021] d) Input the standardized and enhanced gastrointestinal endoscopy images from the training dataset into the encoding unit of the image segmentation network, and output the feature map. .

[0022] e) Feature map The input is fed into the decoding unit of the image segmentation network, and the output is a segmented image.

[0023] f) Calculate the total loss Total loss Train the image segmentation network to obtain the optimized image segmentation network.

[0024] g) Obtain The test set consists of 10 gastrointestinal images. The images of gastrointestinal endoscopy are input into the optimized image segmentation network, and the output is a segmented image.

[0025] It effectively improves the ability to identify lesion boundaries and fine structures, significantly improving the boundary blurring problem of traditional Transformer models. It enhances the model's segmentation stability in low-contrast regions. It effectively improves the accuracy and robustness of segmentation results.

[0026] In one embodiment of the present invention, step a) involves obtaining data from the Kvasir-SEG medical image dataset. Zhang's gastrointestinal images and Zhang masked images were obtained from the CVC-ClinicDB medical image dataset. Zhang's gastrointestinal images and Zhang masked image, In this embodiment, preferably, The value is 900. The value is 550.

[0027] In one embodiment of the present invention, step b) includes the following steps: b-1) The training set of the first Zhang's gastrointestinal images The images from the gastrointestinal endoscopy are obtained by converting them to RGB format using the cv2.cvtColor() function in the OpenCV library. The training set will be the first Zhang masked image Convert to a single-channel grayscale image using the cv2.imread(mask_path, cv2.IMREAD_GRAYSCALE) function from the OpenCV library. .

[0028] b-2) Use the cv2.resize() function in the OpenCV library to resize the gastrointestinal endoscopic images respectively. and single-channel grayscale images Gastrointestinal images scaled to 256×256 and single-channel grayscale images The torchvision.transforms.Normalize() function is used to transform gastrointestinal endoscopy images. The pixels were normalized to zero mean and unit variance to obtain gastrointestinal images. .

[0029] b-3) Gastrointestinal endoscopic images and single-channel grayscale images The gastrointestinal images were obtained by randomly flipping them vertically with a 50% probability using the transforms.RandomVerticalFlip() function in the PyTorch library. and single-channel grayscale images .

[0030] b-4) Gastrointestinal endoscopic images and single-channel grayscale images The gastrointestinal images were obtained by randomly flipping the images horizontally with a 50% probability using the transforms.RandomHorizontalFlip() function in the PyTorch library. and single-channel grayscale images .

[0031] b-5) Gastrointestinal endoscopic images and single-channel grayscale images Random noise was added using the `torch.randn_like()` function in the PyTorch library to obtain standardized and enhanced gastrointestinal images. and the normalized and enhanced masked images The standardized and enhanced colonoscopy image dataset is as follows: , The standardized and enhanced masked image set is , Standardized and enhanced colonoscopy image dataset With the normalized and enhanced masked image set as This constitutes the standardized and enhanced training dataset.

[0032] In one embodiment of the present invention, step d) includes the following steps: d-1) The encoding unit of the image segmentation network consists of a first improved Mix Transformer block, a second improved Mix Transformer block, a third improved Mix Transformer block, and a fourth improved Mix Transformer block.

[0033] d-2) The first element in the standardized and augmented training dataset Zhang Standardized and Enhanced Endoscopic Images The input is fed into the first improved Mix Transformer block of the encoding unit. This first improved Mix Transformer block is composed of a Mix Transformer model, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula For learnable parameters, For the dot product operation, the first improved Mix Transformer block outputs the feature map. Amplify the response intensity of the query / key in edge regions to give it more attention during attention calculation. Feature map The size is H / 4 W / 4 C and H are feature maps. Height, W is the feature map The width of the feature map, C is the width of the feature map. The dimension of C is 64 by default.

[0034] d-3) Feature map The input is fed into the second improved Mix Transformer block of the encoding unit. This second improved Mix Transformer block is composed of a Mix Transformer model, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula The second improved MixTransformer block outputs feature maps as learnable parameters. Feature map The size is H / 8 W / 8 2C.

[0035] d-4) Feature map The input is fed into the third improved Mix Transformer block of the encoding unit. This third improved Mix Transformer block is composed of Mix Transformer models, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula The feature map is obtained from the output of the third improved MixTransformer block, which contains learnable parameters. Feature map The size is H / 16 W / 16 4C.

[0036] d-5) Feature map The input is fed into the fourth improved Mix Transformer block of the encoding unit. This fourth improved Mix Transformer block is composed of Mix Transformer models, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula The fourth improved MixTransformer block outputs feature maps as learnable parameters. Feature map The size is H / 32 W / 32 8C.

[0037] In one embodiment of the present invention, step e) includes the following steps: The encoding unit of the image segmentation network (e-1) consists of a first feature extraction module, a second feature extraction module, a third feature extraction module, a first attention gating module, a second attention gating module, and a feature fusion module. The first, second, and third feature extraction modules of the encoding unit each consist of a first convolutional layer, a first batch normalization (BN) layer, a first ReLU activation function, a second convolutional layer, an average pooling layer, a max pooling layer, a third convolutional layer, a second BN layer, a sigmoid activation function, a first branch, a second branch, a fourth convolutional layer, a fifth convolutional layer, a third BN layer, and a second ReLU activation function. The first, second, and third feature extraction modules... The first branch of each module consists of a convolutional layer, a batch normalization layer, and a ReLU activation function. The second branches of the first, second, and third feature extraction modules also consist of a convolutional layer, a batch normalization layer, and a ReLU activation function. The first and second attention gating modules of the encoding unit consist of a first branch, a second branch, a ReLU activation function, a convolutional layer, a batch normalization layer, and a Sigmoid activation function. The first branches of the first and second attention gating modules also consist of a convolutional layer, a batch normalization layer, and a PReLU activation function. The second branches of the first and second attention gating modules also consist of a convolutional layer and a batch normalization layer.

[0038] e-2) Feature map The inputs are sequentially fed into the first convolutional layer, the first batch normalization (BN) layer, the first ReLU activation function, and the second convolutional layer of the first feature extraction module, and the output is the compressed feature. The first convolutional layer of the first feature extraction module is used to extract local contextual features, and the second convolutional layer is used to achieve channel dimensionality reduction and semantic compression. To semantically separate foreground and background regions, compressed features are... The input is fed into the average pooling layer of the first feature extraction module, and the output is the feature. Compression features The input is fed into the max pooling layer of the first feature extraction module, and the output is the feature. , will feature With features After the addition operation, the data is sequentially fed into the third convolutional layer, the second batch normalization (BN) layer, and the sigmoid activation function of the first feature extraction module, and the output is the foreground attention map. Feature map Foreground attention map The area outside is the background attention map. , feature map With Foreground Attention Map Perform element-wise multiplication to obtain the foreground branch. , feature map Background attention map Perform element-wise multiplication to obtain the background branch. This enables explicit foreground and background separation, with foreground branches. Capture semantic features of the target topic region, background branches Then retain the noise information of the non-foreground structural domains. The foreground branch... The input is fed into the first branch of the first feature extraction module, and the output is the feature. , background branch The input is fed into the second branch of the first feature extraction module, and the output is the feature. To further enhance the difference between foreground and background, a semantic contrast mapping mechanism is introduced. This is done after obtaining features. and characteristics Next, we further compute the difference feature mapping between the two to explicitly represent the response gap between the foreground and background in the high-level semantic space. Specifically, we map the features... With features After performing element-wise subtraction along the channel dimension, the input is fed into the fourth convolutional layer of the first feature extraction module, and the output is the feature. , will feature With features After concatenation along the channel dimension, the data is sequentially input into the fifth convolutional layer, the third batch normalization (BN) layer, and the second ReLU activation function of the first feature extraction module, and the output is the feature map. , feature map With feature map Perform residual connections to obtain feature maps. .

[0039] e-3) Feature map The inputs are sequentially fed into the first convolutional layer, the first batch normalization (BN) layer, the first ReLU activation function, and the second convolutional layer of the second feature extraction module, and the output is the compressed feature. The first convolutional layer of the second feature extraction module is used to extract local contextual features, and the second convolutional layer is used to achieve channel dimensionality reduction and semantic compression. To semantically separate foreground and background regions, compressed features are... The input is fed into the average pooling layer of the second feature extraction module, and the output is the feature. Compression features The input is fed into the max pooling layer of the second feature extraction module, and the output is the feature. , will feature With features After the addition operation, the data is sequentially fed into the third convolutional layer, the second batch normalization (BN) layer, and the sigmoid activation function of the second feature extraction module, and the output is the foreground attention map. Feature map Foreground attention map The area outside is the background attention map. , feature map With Foreground Attention Map Perform element-wise multiplication to obtain the foreground branch. , feature map Background attention map Perform element-wise multiplication to obtain the background branch. This enables explicit foreground and background separation, with foreground branches. Capture semantic features of the target topic region, background branches Then retain the noise information of the non-foreground structural domains. The foreground branch... The input is fed into the first branch of the second feature extraction module, and the output is the feature. , background branch The input is fed into the second branch of the second feature extraction module, and the output is the feature. To further enhance the difference between foreground and background, a semantic contrast mapping mechanism is introduced. This is done after obtaining features. and characteristics Next, we further compute the difference feature mapping between the two to explicitly represent the response gap between the foreground and background in the high-level semantic space. Specifically, we map the features... With features After performing element-wise subtraction along the channel dimension, the input is fed into the fourth convolutional layer of the second feature extraction module, and the output is the feature. , will feature With features After concatenation along the channel dimension, the data is sequentially fed into the fifth convolutional layer, the third batch normalization (BN) layer, and the second ReLU activation function of the second feature extraction module, and the output is the feature map. , feature map With feature map Perform residual connections to obtain feature maps. .

[0040] e-4) Feature Map The inputs are sequentially fed into the first convolutional layer, the first batch normalization (BN) layer, the first ReLU activation function, and the second convolutional layer of the third feature extraction module, and the output is the compressed feature. The first convolutional layer of the third feature extraction module is used to extract local contextual features, and the second convolutional layer is used to achieve channel dimensionality reduction and semantic compression. To semantically separate foreground and background regions, compressed features are... The input is fed into the average pooling layer of the third feature extraction module, and the output is the feature. Compression features The input is fed into the max pooling layer of the third feature extraction module, and the output is the feature. , will feature With features After the addition operation, the data is sequentially fed into the third convolutional layer, the second batch normalization (BN) layer, and the sigmoid activation function of the third feature extraction module, and the output is the foreground attention map. Feature map Foreground attention map The area outside is the background attention map. , feature map With Foreground Attention Map Perform element-wise multiplication to obtain the foreground branch. , feature map Background attention map Perform element-wise multiplication to obtain the background branch. Achieve explicit foreground and background separation, foreground branching Capture semantic features of the target topic region, background branches Then retain the noise information of the non-foreground structural domains. The foreground branch... The input is fed into the first branch of the third feature extraction module, and the output is the feature. , background branch The input is fed into the second branch of the third feature extraction module, and the output is the feature. To further enhance the difference between foreground and background, a semantic contrast mapping mechanism is introduced. This is done after obtaining features. and characteristics Next, we further compute the difference feature mapping between the two to explicitly represent the response gap between the foreground and background in the high-level semantic space. Specifically, we map the features... With features After performing element-wise subtraction along the channel dimension, the input is fed into the fourth convolutional layer of the third feature extraction module, and the output is the feature. , will feature With features After concatenation along the channel dimension, the data is sequentially fed into the fifth convolutional layer, the third batch normalization (BN) layer, and the second ReLU activation function of the third feature extraction module, and the output is the feature map. , feature map With feature map Perform residual connections to obtain feature maps. .

[0041] e-5) Feature Map Feature maps are obtained after bilinear interpolation upsampling. , feature map The input is fed into the first branch of the first attention gating module, and the output is the feature map. The first branch pairs the feature map. Row convolution operations are performed to extract semantic guidance information, followed by batch normalization layers and PreLU activation functions to achieve feature normalization and non-linear enhancement. The feature maps are then processed... The input is fed into the second branch of the first attention gating module, and the output is the feature map. The second branch is used to extract local structure and texture features. The feature map... With feature map After performing element-wise addition along the channel dimension, the input is fed into the ReLU activation function of the first attention-gated module to achieve non-linear interaction between semantic and spatial information, and the output is the fused feature. , will integrate features The inputs are sequentially fed into the convolutional layer, batch normalization layer, and sigmoid activation function of the first attention gating module, and the output is a single-channel gating weight map. The single-channel gating weight map With feature map Element-wise multiplication is performed to selectively enhance or suppress low-level features, resulting in weighted output features. .

[0042] e-6) Feature Map Feature maps are obtained after bilinear interpolation upsampling. , feature map The input is fed into the first branch of the second attention gating module, and the output is the feature map. The first branch pairs the feature map. Row convolution operations are performed to extract semantic guidance information, followed by batch normalization layers and PreLU activation functions to achieve feature normalization and non-linear enhancement. The feature maps are then processed... The input is fed into the second branch of the second attention gating module, and the output is the feature map. The second branch is used to extract local structure and texture features. The feature map... With feature map After performing element-wise addition along the channel dimension, the input is fed into the ReLU activation function of the second attention-gating module to achieve non-linear interaction between semantic and spatial information, and the output is the fused feature. , will integrate features The inputs are sequentially fed into the convolutional layer, batch normalization layer, and sigmoid activation function of the second attention gating module, and the output is a single-channel gating weight map. The single-channel gating weight map With feature map Element-wise multiplication is performed to selectively enhance or suppress low-level features, resulting in weighted output features. .

[0043] The feature fusion module of the e-7 coding unit consists of a first branch, a second branch, a third branch, a convolutional layer, a batch normalization (BN) layer, and a ReLU activation function. The first, second, and third branches of the fusion module are each composed of a convolutional layer, a first BN layer, a first ReLU activation function, a depthwise separable convolutional layer, a second BN layer, and a second ReLU activation function, respectively, to fuse the feature map. Weighted output features Weighted output features Perform a stitching operation to obtain the feature map. , feature map The input is fed into the first branch of the feature fusion module, and the output is the feature map. , feature map After performing bilinear interpolation upsampling, the feature map is obtained. , feature map The input is fed into the second branch of the feature fusion module, and the output is the feature map. , feature map After performing bilinear interpolation upsampling, the feature map is obtained. , feature map The input is fed into the third branch of the feature fusion module, and the output is the feature map. , feature map After performing bilinear interpolation upsampling, the feature map is obtained. , feature map Feature map Feature map After the concatenation operation, the data is sequentially fed into the convolutional layer, batch normalization layer, and ReLU activation function of the fusion module, and the output is the feature map. , feature map With feature map The segmented image is obtained by performing residual connections.

[0044] In this embodiment, preferably, in step e-1), the kernel size of the first convolutional layer of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3, the kernel size of the second convolutional layer is 1×1, the kernel size of the third convolutional layer is 1×1, the kernel size of the fourth convolutional layer is 1×1, and the kernel size of the fifth convolutional layer is 3×3; the kernel size of the convolutional layer of the first branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the second branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the first attention gating module, the second attention gating module, and the third attention gating module is 3×3; the kernel size of the convolutional layer of the second branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the second branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the first attention gating module, the second attention gating module, and the third attention gating module is 3×3; the kernel size of the convolutional layer of the first branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the second branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layer of the first branch ... The kernel size of the convolutional layer in the gating module is 1×1; the kernel size of the convolutional layer in the first branch of the first attention gating module and the second attention gating module is 3×3; the kernel size of the convolutional layer in the second branch of the first attention gating module and the second attention gating module is 3×3; the average pooling layer of the first branch of the feature fusion module has a scale of 2×2 and a kernel size of 1×1; the average pooling layer of the second branch of the feature fusion module has a scale of 4×4 and a kernel size of 1×1; the average pooling layer of the third branch of the feature fusion module has a scale of 6×6 and a kernel size of 1×1; and the convolutional layer size of the feature fusion module is 1×1.

[0045] In one embodiment of the present invention, step f) is performed by the first Zhang's gastrointestinal images The segmented image and the first Zhang masked image IoU loss was calculated Through formula The total loss was calculated. In the formula , All are weighting coefficients. , This is the cross-entropy loss. Preferably, The value is 0.5. The value is 0.5. Used to improve pixel-level classification accuracy and enhance the model's sensitivity to detailed regions. It is used to measure the degree of overlap between the predicted result and the true mask, thereby optimizing the overall region matching effect.

[0046] In one embodiment of the invention, step f) uses the SGD optimizer to utilize the total loss. Train the image segmentation network to obtain the optimized image segmentation network. During training, set weight_decay to 0.0001 and learning rate lr to the base learning rate of 0.01.

[0047] In one embodiment of the present invention, step g) involves obtaining data from the CVC-ColonDB medical image dataset, the ETIS-LaribPolypDB medical image dataset, and the EndoScene medical image dataset. The test set consists of several gastrointestinal and intestinal endoscopy images.

[0048] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for segmenting gastrointestinal polyps based on the Mix Transformer model, characterized in that, include: a) Obtain Zhang's gastrointestinal endoscopy images and masked images were used to obtain the dataset of original gastrointestinal endoscopy images. and mask image set The training set constituted ,in For the first Zhang's gastrointestinal images, , , For the first Zhang's gastrointestinal images The corresponding masked image is marked with gastrointestinal polyps; b) Preprocess the endoscopy and gastroscopy images and mask images in the training set by mirroring to obtain the standardized and enhanced training dataset; c) Construct an image segmentation network consisting of encoding units and decoding units; d) Input the standardized and enhanced gastrointestinal endoscopy images from the training dataset into the encoding unit of the image segmentation network, and output the feature map. ; e) Feature map The input is fed into the decoding unit of the image segmentation network, and the output is a segmented image; f) Calculate the total loss Total loss Train the image segmentation network to obtain the optimized image segmentation network; g) Obtain The test set consists of 10 gastrointestinal images. The images of gastrointestinal endoscopy are input into the optimized image segmentation network, and the output is a segmented image.

2. The gastrointestinal polyp segmentation method based on the Mix Transformer model according to claim 1, characterized in that: Step a) involves obtaining images from the Kvasir-SEG medical image dataset. Zhang's gastrointestinal images and Zhang masked images were obtained from the CVC-ClinicDB medical image dataset. Zhang's gastrointestinal images and Zhang masked image, .

3. The gastrointestinal polyp segmentation method based on the Mix Transformer model according to claim 2, characterized in that: The value is 900. The value is 550.

4. The gastrointestinal polyp segmentation method based on the Mix Transformer model according to claim 1, characterized in that, Step b) includes the following steps: b-1) The training set of the first Zhang's gastrointestinal images The images from the gastrointestinal endoscopy are obtained by converting them to RGB format using the cv2.cvtColor() function in the OpenCV library. The training set will be the first Zhang masked image Convert to a single-channel grayscale image using the cv2.imread(mask_path, cv2.IMREAD_GRAYSCALE) function from the OpenCV library. ; b-2) Use the cv2.resize() function in the OpenCV library to resize the gastrointestinal endoscopic images respectively. and single-channel grayscale images Gastrointestinal images scaled to 256×256 and single-channel grayscale images The torchvision.transforms.Normalize() function is used to transform gastrointestinal endoscopy images. The pixels were normalized to zero mean and unit variance to obtain gastrointestinal images. ; b-3) Gastrointestinal endoscopic images and single-channel grayscale images The gastrointestinal images were obtained by randomly flipping them vertically with a 50% probability using the transforms.RandomVerticalFlip() function in the PyTorch library. and single-channel grayscale images ; b-4) Gastrointestinal endoscopic images and single-channel grayscale images The gastrointestinal images were obtained by randomly flipping the images horizontally with a 50% probability using the transforms.RandomHorizontalFlip() function in the PyTorch library. and single-channel grayscale images ; b-5) Gastrointestinal endoscopic images and single-channel grayscale images Random noise was added using the `torch.randn_like()` function in the PyTorch library to obtain standardized and enhanced gastrointestinal images. and the normalized and enhanced masked images The standardized and enhanced colonoscopy image dataset is as follows: , The standardized and enhanced masked image set is , Standardized and enhanced colonoscopy image dataset With the normalized and enhanced masked image set as This constitutes the standardized and enhanced training dataset.

5. The gastrointestinal polyp segmentation method based on the Mix Transformer model according to claim 1, characterized in that, Step d) includes the following steps: d-1) The encoding unit of the image segmentation network consists of a first improved Mix Transformer block, a second improved Mix Transformer block, a third improved Mix Transformer block, and a fourth improved Mix Transformer block; d-2) The first element in the standardized and augmented training dataset Zhang Standardized and Enhanced Endoscopic Images The input is fed into the first improved Mix Transformer block of the encoding unit. This first improved Mix Transformer block is composed of a Mix Transformer model, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula For learnable parameters, For the dot product operation, the first improved Mix Transformer block outputs the feature map. ; d-3) Feature map The input is fed into the second improved Mix Transformer block of the encoding unit. This second improved Mix Transformer block is composed of a Mix Transformer model, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula The second improved MixTransformer block outputs feature maps as learnable parameters. ; d-4) Feature map The input is fed into the third improved Mix Transformer block of the encoding unit. This third improved Mix Transformer block is composed of a Mix Transformer model, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula The feature map is obtained from the output of the third improved MixTransformer block, which contains learnable parameters. ; d-5) Feature map The input is fed into the fourth improved Mix Transformer block of the encoding unit. This fourth improved Mix Transformer block is composed of a Mix Transformer model, and it converts the query vector from the attention mechanism in the Mix Transformer model into... Change to , In the formula The fourth improved MixTransformer block outputs feature maps as learnable parameters. .

6. The gastrointestinal polyp segmentation method based on the Mix Transformer model according to claim 5, characterized in that, Step e) includes the following steps: The encoding unit of the image segmentation network (e-1) consists of a first feature extraction module, a second feature extraction module, a third feature extraction module, a first attention gating module, a second attention gating module, and a feature fusion module. The first, second, and third feature extraction modules of the encoding unit each consist of a first convolutional layer, a first batch normalization (BN) layer, a first ReLU activation function, a second convolutional layer, an average pooling layer, a max pooling layer, a third convolutional layer, a second BN layer, a sigmoid activation function, a first branch, a second branch, a fourth convolutional layer, a fifth convolutional layer, a third BN layer, and a second ReLU activation function. The first, second, and third feature extraction modules... The first branch of each module consists of a convolutional layer, a batch normalization layer, and a ReLU activation function. The second branches of the first, second, and third feature extraction modules also consist of convolutional layers, batch normalization layers, and ReLU activation functions. The first and second attention gating modules of the encoding unit consist of a first branch, a second branch, a ReLU activation function, a convolutional layer, a batch normalization layer, and a Sigmoid activation function. The first branches of the first and second attention gating modules also consist of convolutional layers, batch normalization layers, and PReLU activation functions. The second branches of the first and second attention gating modules also consist of convolutional layers and batch normalization layers. e-2) Feature map The inputs are sequentially fed into the first convolutional layer, the first batch normalization (BN) layer, the first ReLU activation function, and the second convolutional layer of the first feature extraction module, and the output is the compressed feature. Compression features The input is fed into the average pooling layer of the first feature extraction module, and the output is the feature. Compression features The input is fed into the max pooling layer of the first feature extraction module, and the output is the feature. , will feature With features After the addition operation, the data is sequentially fed into the third convolutional layer, the second batch normalization (BN) layer, and the sigmoid activation function of the first feature extraction module, and the output is the foreground attention map. Feature map Foreground attention map The area outside is the background attention map. , feature map With Foreground Attention Map Perform element-wise multiplication to obtain the foreground branch. , feature map Background attention map Perform element-wise multiplication to obtain the background branch. , foreground branch The input is fed into the first branch of the first feature extraction module, and the output is the feature. , background branch The input is fed into the second branch of the first feature extraction module, and the output is the feature. , will feature With features After performing element-wise subtraction along the channel dimension, the input is fed into the fourth convolutional layer of the first feature extraction module, and the output is the feature. , will feature With features After concatenation along the channel dimension, the data is sequentially input into the fifth convolutional layer, the third batch normalization (BN) layer, and the second ReLU activation function of the first feature extraction module, and the output is the feature map. , feature map With feature map Perform residual connections to obtain feature maps. ; e-3) Feature map The inputs are sequentially fed into the first convolutional layer, the first batch normalization (BN) layer, the first ReLU activation function, and the second convolutional layer of the second feature extraction module, and the output is the compressed feature. Compression features The input is fed into the average pooling layer of the second feature extraction module, and the output is the feature. Compression features The input is fed into the max pooling layer of the second feature extraction module, and the output is the feature. , will feature With features After the addition operation, the data is sequentially fed into the third convolutional layer, the second batch normalization (BN) layer, and the sigmoid activation function of the second feature extraction module, and the output is the foreground attention map. Feature map Foreground attention map The area outside is the background attention map. , feature map With Foreground Attention Map Perform element-wise multiplication to obtain the foreground branch. , feature map Background attention map Perform element-wise multiplication to obtain the background branch. , foreground branch The input is fed into the first branch of the second feature extraction module, and the output is the feature. , background branch The input is fed into the second branch of the second feature extraction module, and the output is the feature. , will feature With features After performing element-wise subtraction along the channel dimension, the input is fed into the fourth convolutional layer of the second feature extraction module, and the output is the feature. , will feature With features After concatenation along the channel dimension, the data is sequentially fed into the fifth convolutional layer, the third batch normalization (BN) layer, and the second ReLU activation function of the second feature extraction module, and the output is the feature map. , feature map With feature map Perform residual connections to obtain feature maps. ; e-4) Feature Map The inputs are sequentially fed into the first convolutional layer, the first batch normalization (BN) layer, the first ReLU activation function, and the second convolutional layer of the third feature extraction module, and the output is the compressed feature. Compression features The input is fed into the average pooling layer of the third feature extraction module, and the output is the feature. Compression features The input is fed into the max pooling layer of the third feature extraction module, and the output is the feature. , will feature With features After the addition operation, the data is sequentially fed into the third convolutional layer, the second batch normalization (BN) layer, and the sigmoid activation function of the third feature extraction module, and the output is the foreground attention map. Feature map Foreground attention map The area outside is the background attention map. , feature map With Foreground Attention Map Perform element-wise multiplication to obtain the foreground branch. , feature map Background attention map Perform element-wise multiplication to obtain the background branch. , foreground branch The input is fed into the first branch of the third feature extraction module, and the output is the feature. , background branch The input is fed into the second branch of the third feature extraction module, and the output is the feature. , will feature With features After performing element-wise subtraction along the channel dimension, the input is fed into the fourth convolutional layer of the third feature extraction module, and the output is the feature. , will feature With features After concatenation along the channel dimension, the data is sequentially fed into the fifth convolutional layer, the third batch normalization (BN) layer, and the second ReLU activation function of the third feature extraction module, and the output is the feature map. , feature map With feature map Perform residual connections to obtain feature maps. ; e-5) Feature Map Feature maps are obtained after bilinear interpolation upsampling. , feature map The input is fed into the first branch of the first attention gating module, and the output is the feature map. , feature map The input is fed into the second branch of the first attention gating module, and the output is the feature map. , feature map With feature map After performing element-wise addition along the channel dimension, the input is fed into the ReLU activation function of the first attention-gated module, and the output is the fused feature. , will integrate features The inputs are sequentially fed into the convolutional layer, batch normalization layer, and sigmoid activation function of the first attention gating module, and the output is a single-channel gating weight map. The single-channel gating weight map With feature map Perform element-wise multiplication to obtain the weighted output features. ; e-6) Feature Map Feature maps are obtained after bilinear interpolation upsampling. , feature map The input is fed into the first branch of the second attention gating module, and the output is the feature map. , feature map The input is fed into the second branch of the second attention gating module, and the output is the feature map. , feature map With feature map After performing element-wise addition along the channel dimension, the input is fed into the ReLU activation function of the second attention-gated module, and the output is the fused feature. , will integrate features The inputs are sequentially fed into the convolutional layer, batch normalization layer, and sigmoid activation function of the second attention gating module, and the output is a single-channel gating weight map. The single-channel gating weight map With feature map Perform element-wise multiplication to obtain the weighted output features. ; The feature fusion module of the e-7 coding unit consists of a first branch, a second branch, a third branch, a convolutional layer, a batch normalization (BN) layer, and a ReLU activation function. The first, second, and third branches of the fusion module are each composed of a convolutional layer, a first BN layer, a first ReLU activation function, a depthwise separable convolutional layer, a second BN layer, and a second ReLU activation function, respectively, to fuse the feature map. Weighted output features Weighted output features Perform a stitching operation to obtain the feature map. , feature map The input is fed into the first branch of the feature fusion module, and the output is the feature map. , feature map After performing bilinear interpolation upsampling, the feature map is obtained. , feature map The input is fed into the second branch of the feature fusion module, and the output is the feature map. , feature map After performing bilinear interpolation upsampling, the feature map is obtained. , feature map The input is fed into the third branch of the feature fusion module, and the output is the feature map. , feature map After performing bilinear interpolation upsampling, the feature map is obtained. , feature map Feature map Feature map After the concatenation operation, the data is sequentially fed into the convolutional layer, batch normalization layer, and ReLU activation function of the fusion module, and the output is the feature map. , feature map With feature map The segmented image is obtained by performing residual connections.

7. The gastrointestinal polyp segmentation method based on the Mix Transformer model according to claim 6, characterized in that, In step e-1), the kernel size of the first convolutional layer of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3, the kernel size of the second convolutional layer is 1×1, the kernel size of the third convolutional layer is 1×1, the kernel size of the fourth convolutional layer is 1×1, and the kernel size of the fifth convolutional layer is 3×3. The kernel size of the convolutional layer of the first branch of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3. The kernel size of the convolutional layers in the second branches of the first feature extraction module, the second feature extraction module, and the third feature extraction module is 3×3; the kernel size of the convolutional layers in the first attention gating module and the second attention gating module is 1×1; and the kernel size of the convolutional layers in the first branches of the first attention gating module and the second attention gating module is 3×3. The kernel size of the convolutional layers in the second branches of the first attention gating module and the second attention gating module is 3×3; The first branch of the feature fusion module has an average pooling layer size of 2×2 and a convolutional kernel size of 1×1. The second branch of the feature fusion module has an average pooling layer size of 4×4 and a convolutional kernel size of 1×1. The third branch of the feature fusion module has an average pooling layer size of 6×6 and a convolutional kernel size of 1×1. The convolutional layer size of the feature fusion module is 1×1.

8. The gastrointestinal polyp segmentation method based on the Mix Transformer model according to claim 1, characterized in that: In step f), through the first Zhang's gastrointestinal images The segmented image and the first Zhang masked image IoU loss was calculated Through formula The total loss was calculated. In the formula , All are weighting coefficients. , This represents the cross-entropy loss.

9. The gastrointestinal polyp segmentation method based on the Mix Transformer model according to claim 1, characterized in that: Step f) uses the SGD optimizer to utilize the total loss. Train the image segmentation network to obtain the optimized image segmentation network. During training, set weight_decay to 0.0001 and learning rate lr to the base learning rate of 0.

01.

10. The gastrointestinal polyp segmentation method based on the Mix Transformer model according to claim 1, characterized in that: In step g), obtain the CVC-ColonDB medical image dataset, ETIS-LaribPolypDB medical image dataset, and EndoScene medical image dataset. The test set consists of several gastrointestinal and intestinal endoscopy images.

Images