Polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement

Abstract

The application discloses a polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement, which designs pixel dynamic aggregation for extracting stronger features and local perception enhancement for detecting difficult-to-distinguish detailed information; the overall design idea of the method comprises the following steps: obtaining a polyp image, performing data processing on the polyp image to obtain a transformed image; performing feature extraction on the polyp image to obtain image features of the polyp image; performing feature processing on the polyp image features to obtain target features of the polyp image; and segmenting a polyp region by using the target features to obtain a binary segmentation result of the polyp image, wherein the binary segmentation result is used for representing the type of an object to which each pixel in the polyp image belongs. The application solves the technical problems of low segmentation accuracy and poor generalization ability of polyp images in the related art.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing, and in particular relates to the field of medical image segmentation in computer vision. Background Technology

[0002] Colorectal cancer is one of the most common cancers in the world, and colonoscopy is an important step in preventing polyps from becoming cancerous. Traditional examination methods rely on doctors with extensive clinical experience for diagnosis. However, thanks to the rapid development of deep learning and computer vision, computer-aided diagnosis has become the most efficient examination method.

[0003] Currently, many models have been proposed to improve the segmentation of polyp images. Traditional image segmentation methods rely on manually constructed features, which cannot extract the latent features of polyp images. The emergence of fully convolutional neural networks (FCNs) has overcome this limitation. Accurate polyp image segmentation has been achieved using FCNs on VGG pre-trained models. The U-Net architecture establishes skip connections between full-scale features to fully utilize feature information, and is a typical network structure in the field of medical segmentation. Many scholars at home and abroad have replaced the skip connections in U-Net with dense connections, and have further improved image segmentation results by using new methods such as residual computation and attention mechanisms. Recent methods focus on extracting information from difficult regions to learn multi-scale features with rich semantics.

[0004] However, most current methods do not adequately address the boundary uncertainty problem in polyp image segmentation. For example, invention CN114972155A discloses a polyp image segmentation method based on contextual information and reverse attention, but it still suffers from blurred and inaccurate boundaries. Research has found that in many predicted images, due to the low contrast between polyps and mucosa, models struggle to detect the accurate location of polyps, or may misdetect mucosal regions as polyps. While current models can detect the location of most polyps, they tend to focus on regions with significant differences between foreground and background, lacking the ability to enhance edge details and background contrast. This leads to uncertainty in the detection of polyp boundaries, making it difficult to accurately segment polyp targets. These limitations restrict the research and application of polyp image segmentation methods. Summary of the Invention

[0005] To address the problems existing in the background art, the purpose of this invention is to provide a polyp image segmentation method with strong representation ability and good segmentation performance. To this end, this invention adopts the following technical solution:

[0006] The polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement includes data processing, pre-trained weight loading, encoder feature extraction, key pixel extraction, key pixel propagation, local perception enhancement, weighted feature fusion, model training and inference; the pixel dynamic aggregation includes key pixel extraction and key pixel propagation; the polyp image segmentation method is performed according to the following steps:

[0007] Step (1), data processing: The original polyp images are uniformly adjusted to 352×352, and the normalization method is used to change the image data distribution range from [0,1] to [-1,1], which conforms to the standard normal distribution and makes the model easier to converge; data augmentation is used to expand the number of samples. Five data augmentation methods are used, namely random flipping, random cropping, random rotation, color enhancement, and adding random noise. Among them, color enhancement includes changes in contrast, brightness, intensity, and sharpness.

[0008] Step (2), pre-trained weight loading: The feature encoder Res2Net-50 loads the publicly available pre-trained weights trained on the ImageNet dataset to initialize the network weights;

[0009] Step (3), Encoder Feature Extraction: Read the image data processed in step (1). After the Res2Net-50 feature encoder completes the pre-training weight loading in step (2), it extracts multi-scale features E from the input image. i (i = 0, 1, 2, 3, 4), the feature size of the i-th layer is C × H × W, and the feature size of the (i+1)-th layer is 2C × H / 2 × W / 2, where C is the number of feature channels, and H and W are the height and width of the feature, respectively; the deep multi-scale feature E′ in multi-scale features. i (i=1,2,3,4) contains less noise information and is used for pixel dynamic aggregation and local perception enhancement;

[0010] Step (4), key pixel extraction: extract key pixels from the multi-scale features extracted in step (3), and extract the deep multi-scale features E′ extracted by the feature encoder. i (i = 1, 2, 3, 4) Perform Hadamard products step by step to obtain the information-enhanced deep multi-scale augmented features F. i (i = 1, 2, 3, 4), the i-th deep multi-scale augmentation feature F with a scale of C × H × W. i Downsampled to the (i-1)th deep multi-scale augmentation feature F with a scale of C / 2×2H×2W i-1 With consistent size, the deep multi-scale enhanced features F are then... i and deep multi-scale enhancement features F i-1The concatenated feature map is then fed into a 3×3 convolution, where the number of channels is reduced from C to 1. An activation function is used to obtain an associated feature map. This associated feature map is then subjected to average pooling, multiplied by the high-resolution feature map, and subtracted pixel by pixel to obtain a structural feature map, which scatters a set of high-resolution key pixels. Simultaneously, the associated feature map is multiplied by the low-resolution feature map using max pooling to obtain a salient region feature map, which scatters a set of low-resolution key pixels. This low-resolution key pixels are then fused with the high-resolution key pixels, and the set of pixels with the highest correlation is selected as the key pixels.

[0011] Step (5), key pixel propagation: The key pixels generated in step (4) are propagated to spread the information of the key pixels from the top layer to the bottom layer of the network structure. After selecting the channel index of the key pixel in the associated feature map, the salient region feature map and the structural feature map propagate the key pixel information. The channel index k of the key pixels is sorted to obtain the key pixel index and key pixel coordinates. Based on the key pixel coordinates, the irregular sampling method is used to enhance the deep multi-scale feature F. i F i-1 Perform sampling to obtain the sampling feature map S i S i-1 Sampling feature map S i S i-1 After multiplication, the sampled feature map is obtained by processing the result with a sigmoid activation function. The similarity feature map is then compared with the sampled feature map S. i After multiplication, it is compared with the sampled feature map S. i-1 Pixel-by-pixel summation yields a structural propagation feature map. The structural feature map is then sampled based on key pixel indices to obtain a pixel-aggregated feature map P. i-1 The key pixel propagation process between different feature layers adopts a propagation mechanism from deep to shallow layers.

[0012] Step (6), Local Perception Enhancement: The deep multi-scale enhancement features extracted in step (3) are subjected to local perception enhancement, and the deep multi-scale enhancement features F i The modulated feature map is obtained by dynamic convolution and then fed into a dynamic convolutional layer. The efficient fusion characteristic of dynamic convolution is utilized to enhance feature representation. The modulated feature map is then combined with a deep multi-scale enhanced feature map F whose width and length are transformed from H×W to 1×1. iPixel-by-pixel multiplication is performed to generate a channel modulation feature map. Spatial feature enhancement is then used to extract finer edge details. The channel modulation feature map is flattened, the number of channels is reduced from C to 1, and then reduced back to the original number of channels C to obtain a stretched feature map. This channel scaling transformation enhances spatial representation capabilities and amplifies the background difference before and after the polyp. The channel modulation feature map is then fed into dynamic convolution to obtain a channel-spatial modulation feature map, which is multiplied pixel-by-pixel with the stretched feature map to obtain the final local enhancement feature map. During local perceptual enhancement, a propagation method from shallow to deep layers is used to extract more refined local details.

[0013] Step (7), feature weighted fusion: The pixel aggregation feature map generated by pixel dynamic aggregation in steps (4) and (5) and the local enhancement feature map generated by local perception enhancement in step (6) are subjected to feature weighted fusion operation to obtain the fused feature map. The weights of the pixel aggregation feature map generated by pixel dynamic aggregation and the local enhancement feature map generated by local perception enhancement are 0.9 and 0.1, respectively. The fused feature map is fed into the sigmoid activation function to obtain the final prediction map.

[0014] Step (8), Model Training and Inference: During the experiment, five widely used public datasets in the polyps field were used for model training and inference: Kvasir-SEG, ClinicDB, ColonDB, Endoscene, and ETIS. When dividing the training, test, and validation sets, 80% of the images in the Kvasir and CVC-ClinicDB datasets were randomly used for training, the remaining 10% for validation, and 10% for testing. All images in the ColonDB, Endoscene, and ETIS datasets were used for testing. During training, the optimizer used was Adam, the learning rate was set to 0.0001, the learning rate adopted a gradient decay strategy with a decay rate of 0.1, a decay period of 50, a total training period of 100, and a batch size of 20. During inference, the size of each batch of input images was also uniformly adjusted to 352×352.

[0015] While adopting the above technical solutions, the present invention may also adopt or combine the following further technical solutions:

[0016] As a preferred technical solution of the present invention: In step (4), after downsampling, splicing, convolution and activation function operations are performed on the high-resolution features and low-resolution features, a related feature map is obtained. The related feature map is further combined with the high-resolution features and scattered to obtain a set of key pixels of high-resolution features. The related feature map is also combined with the low-resolution feature map and scattered to obtain a set of key pixels of low-resolution features. Among them, the number of key pixel indexes is the same as the number of scattered pixels and is a multiple of 16. It can be set to 16, 32, 64, 128 and 256. The method is set to 32. The set of key pixels of high-resolution features and the set of key pixels of low-resolution features are fused and the set of pixels with the highest correlation is selected as the key pixels.

[0017] As a preferred technical solution of the present invention: in step (5), sampling matrices α in different width W and length H directions are obtained based on the coordinates of the 32 key pixels obtained by sampling. i and α i-1 α i α i-1 After multiplication and fusion, a propagation feature map is obtained. At this point, pixel-by-pixel random sampling is performed using 32 key pixel indices to obtain the final sampled feature map S. i S i-1 .

[0018] As a preferred technical solution of the present invention: In step (6), the deep multi-scale enhancement features are multiplied after channel transformation and dynamic convolution modulation weights to obtain a channel modulation map, thereby increasing the channel feature representation capability. When the channel dimension is reduced, the channel dimension size is transformed from C to 1. When the channel dimension is enlarged, the channel dimension size is transformed from 1 to C. The channel modulation map after spatial dimension scaling transformation is multiplied and combined with the original channel modulation map to obtain a spatial modulation feature map. When the spatial dimension is reduced, the spatial dimension size is transformed from H×W to 1×1. When the spatial dimension is enlarged, the spatial dimension size is transformed from 1×1 to H×W.

[0019] As a preferred technical solution of the present invention: In step (7), different weights are assigned to the pixel aggregation feature map generated by pixel dynamic aggregation and the local enhancement feature map generated by local perception enhancement, and weighted fusion is performed. When allocating weights, the pixel aggregation feature map plays a decisive role, and the weight value set is greater than that of the local enhancement feature map, with a value at least greater than 0.5.

[0020] The polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement provided by this invention is suitable for medical image segmentation tasks. To address the problems of high similarity between foreground and background and boundary uncertainty in polyp image segmentation, it employs pixel dynamic aggregation to extract more representative features, while also using local perception enhancement to detect difficult-to-distinguish details and amplify the differences between foreground and background, achieving a refined segmentation effect. Compared with existing technologies, this invention has the following advantages:

[0021] (1) Good segmentation performance. The model proposed in this invention was tested on 5 public datasets, and the evaluation results of the mean similarity metric (mean Dice) and mean absolute error (mean absolute error) reached the best results to date.

[0022] (2) In the process of pixel dynamic aggregation, in order to capture the key features of polyp images, the key pixel extraction and key pixel propagation realize the propagation of important semantic information between features at different scales. More rich representation information can be dynamically extracted and propagated to shallow features. By calculating the correlation map, more accurate salient regions can be detected, which greatly improves the segmentation accuracy and efficiency of the model.

[0023] (3) In the process of local perception enhancement, the present invention amplifies the difference between the front and back background of edge details, making the edge of the polyp more prominent. By enhancing the attention to edge pixels, the model emphasizes edge features more during learning, so as to extract more effective local feature information and greatly improve the model's ability to detect difficult features.

[0024] (4) The feature weighted fusion method proposed in this invention achieves compensation and enhancement of different features, which significantly improves the model’s representation ability and segmentation accuracy.

[0025] (5) Strong generalization performance: The polyp image segmentation method proposed in this invention is applicable to other medical image segmentation tasks, such as brain region segmentation and tumor segmentation. Attached Figure Description

[0026] Figure 1 This is an overall flowchart of the polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement of the present invention;

[0027] Figure 2 This is a model structure diagram of the polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement of the present invention;

[0028] Figure 3 This is a data processing flowchart provided by the present invention;

[0029] Figure 4This is a flowchart of the pixel dynamic aggregation process provided by the present invention;

[0030] Figure 5 The flowchart for local perception enhancement provided by this invention;

[0031] Figure 6 The image shows the test sample results of the method described in this invention. Detailed Implementation

[0032] The specific embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0033] Figure 1 This is an overall flowchart of the polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement provided by the present invention. The method includes acquiring a polyp image, performing data processing on the polyp image to obtain a transformed image; extracting features from the polyp image to obtain image features of the polyp image; performing feature processing on the polyp image features to obtain target features of the polyp image; and performing binary segmentation using the target features of the polyp image to obtain the binary segmentation result of the polyp image.

[0034] Figure 2 The model structure diagram of the polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement provided for this example includes data processing, encoder feature extraction, pixel dynamic aggregation, local perception enhancement, and feature weighted fusion. Pixel dynamic aggregation includes key pixel extraction and key pixel propagation.

[0035] Step (1), the data processing flow is as follows Figure 3As shown, firstly, image data is directly read, and data augmentation is used to expand the sample size. Five data augmentation methods are employed: random flipping, random cropping, random rotation, color enhancement, and adding random noise. For random flipping, images are randomly selected and flipped from left to right. For random cropping, the cropping boundary length is set to 30 pixels. For random rotation, the rotation angle is 15 degrees. Color enhancement includes changes in brightness, contrast, intensity, and sharpness. The ranges for brightness and contrast are [0.5, 1.5], for color intensity [0, 2], and for sharpness [0, 3]. The added random noise is Gaussian noise. The Gaussian noise average was set to 0.1, and the sigma value was set to 0.35. Then, the original polyp images were uniformly resized to 352×352. Normalization was used to change the image data distribution range from [0,1] to [-1,1], conforming to a standard normal distribution, making the model easier to converge. The mean and standard deviation of the normalization function were taken from the ImageNet dataset, a common practice in image processing. The processed images retained their original indices; during subsequent feature extraction and processing, the corresponding images were directly retrieved by searching the index when loading data.

[0036] Step (2): The feature encoder used in this invention example is Res2Net-50. Before network training, the pre-trained weights obtained by training on the ImageNet dataset are loaded to initialize the network weights.

[0037] Step (3), the data processing flow is as follows Figure 3 As shown, the image data processed in step (1) is read. During the encoder feature extraction process, after the Res2Net-50 feature encoder completes the pre-trained weight loading in step (2), the feature encoder extracts multi-scale features E from the input image. i (i = 0, 1, 2, 3, 4), the feature size of the i-th layer is C × H × W, and the feature size of the (i+1)-th layer is 2C × H / 2 × W / 2. Specifically, the feature size of the 0th layer is 64 × 176 × 176, the feature size of the 1st layer is 256 × 88 × 88, the feature size of the 2nd layer is 512 × 44 × 44, the feature size of the 3rd layer is 1024 × 22 × 22, and the feature size of the 4th layer is 2048 × 11 × 11. The extracted shallow features contain rich detailed information, but the excessive noise information leads to information redundancy. Deep features have stronger representation capabilities. In the subsequent feature processing stage, only the deep multi-scale features E′ obtained from the multi-scale features, which contain a small amount of noise information, are used. i (i = 1, 2, 3, 4);

[0038] Step (4) involves extracting key pixels from the deep multi-scale features extracted in step (3), such as... Figure 4 The pixel dynamic aggregation flowchart shown illustrates the deep features extracted by the encoder, namely the deep multi-scale features E′. i (i = 1, 2, 3, 4), perform Hadamard products step by step to obtain the deep multi-scale enhancement features F. i (i = 1, 2, 3, 4), the next step is to input the i-th layer deep multi-scale augmentation feature F with a scale of C × H × W. i Downsampled to the (i-1)th deep multi-scale augmented feature F of scale size C / 2×2H×2W i-1 Consistent, the features of layer i-1 are downsampled to the size of the features of layer i, and the deep multi-scale enhanced features F are... i and deep multi-scale enhancement features F i-1 The concatenated feature maps are then fed into a 3×3 convolution, reducing the number of channels from C to 1. Finally, an activation function is applied to obtain the associated feature map M. i As shown in equation (1):

[0039] M i =sig(con(cat(F i ,F i-1 ))) (1)

[0040] Where sig(*) represents the sigmoid activation function, con(*) represents a 3×3 convolution, and cat(*) represents a concatenation operation;

[0041] The obtained associated feature map is then subjected to average pooling, multiplied by high-resolution features, and then subtracted pixel by pixel to obtain a structural feature map, which scatters the set of high-resolution feature key pixels S. h As shown in equation (2):

[0042] S h =F i -avgpool(M i ·F i (2)

[0043] Here, avgpool(*) represents the average pooling operation.

[0044] While obtaining the set of high-resolution key pixels, the features obtained by max pooling the associated feature map are multiplied with the low-resolution features to scatter the set of low-resolution key pixels S. l As shown in equation (3):

[0045] S l =F i +maxpool(Mi ·F i (3)

[0046] Finally, the high-resolution feature key pixel set and the low-resolution feature key pixel set are fused together, and the set of pixels with the highest correlation is selected as the key pixels.

[0047] Step (5) involves propagating the key pixels generated in step (4), such as... Figure 4 The pixel dynamic aggregation flowchart shown propagates key pixel information from the top layer to the bottom layer of the network structure. After selecting the channel index of the key pixel in the associated feature map, the key pixel information propagates from the salient region feature map and the structural feature map. The channel index k of the key pixel is sorted by channel to obtain the key pixel index In. i Based on the key pixel coordinates (pixel(x,y), an irregular sampling method is used to enhance the deep multi-scale feature F. i F i-1 Perform sampling to obtain the sampling feature map S i S i-1 Sampling feature map S i and S i-1 After multiplication, the result is fed into a sigmoid activation function to obtain a similarity feature map. The similarity feature map is then compared with F... i After multiplication, with F i-1 By adding pixels one by one, we obtain the structured propagation feature map Q. i-1 The structural feature map is sampled based on the key pixel index to obtain the pixel aggregate feature map P. i-1 As shown in equations (4), (5), (6), (7), (8), and (9):

[0048] In i =topk(maxpool(M) i (4)

[0049]

[0050]

[0051] S i S i -1=bmm(F i F i-1 S h (7)

[0052] Q i-1 =sig(S i *S i-1 )*F i +Fi-1 (8)

[0053] P i-1 =bmm(Q i-1 (9)

[0054] Where maxpool(*) represents the max pooling operation, topk(*) represents the operation of sorting and selecting the channel with the largest weight, and In i Represents the key point index of the i-th layer, W and H are the width and height of the feature respectively, sig(*) is the sigmoid activation function, bmm(*) is the sampling operation, and the key pixel propagation adopts the propagation mechanism from deep to shallow layers during the pixel dynamic aggregation process.

[0055] Step (6) involves performing local perceptual enhancement on the multi-scale features extracted in step (3). The local perceptual enhancement process is as follows: Figure 5 As shown, in the process of local perception enhancement, the deep multi-scale enhancement feature F is... i The modulated feature map is obtained by dynamic convolution and then fed into a dynamic convolutional layer. The efficient fusion characteristic of dynamic convolution is utilized to enhance feature representation. The modulated feature map is then combined with a deep multi-scale enhanced feature map F whose width and length are transformed from H×W to 1×1. i Pixel-wise multiplication is performed to generate a channel modulation feature map. Spatial feature enhancement is then used to extract finer edge details. The channel modulation feature map is first flattened, the number of channels is changed from C to 1, and then it is changed back to the original number of channels C. The spatial representation capability can be enhanced by scaling the number of channels, thus obtaining a scaled feature map. The difference between the background before and after the polyp is magnified. The channel modulation feature map is fed into dynamic convolution to obtain a channel-spatial modulation feature map. Pixel-wise multiplication with the scaled feature map yields the final local enhancement feature map. In the local perceptual enhancement process, a propagation method from shallow to deep layers is used to extract more local fine-grained information.

[0056] In step (7), during the feature weighted fusion process, the pixel aggregation feature map generated by pixel dynamic aggregation in steps (4) and (5) and the local enhancement feature map generated by local perception enhancement in step (6) are subjected to feature weighted fusion operation to obtain the fused feature map. The weight values ​​of the pixel aggregation feature map of pixel dynamic aggregation and the local enhancement feature map of local perception enhancement are 0.9 and 0.1, respectively. The fused feature map is fed into the sigmoid activation function for processing to obtain the prediction map.

[0057] Step (8), Model Training and Inference: During the experiment, five widely used public datasets in the polyps field were used for model training and inference: Kvasir-SEG, ClinicDB, ColonDB, Endoscene, and ETIS. When dividing the training set, test set, and validation set, 80% of the images in the Kvasir dataset and CVC-ClinicDB dataset were randomly used for training, the remaining 10% for validation, and 10% for testing. All of the ColonDB, Endoscene, and ETIS datasets were used for testing. During training, the optimizer used was Adam, the learning rate was set to 1e-4, the learning rate of the model adopted a gradient decay strategy, the decay rate was 0.1, and the decay period was 50. The total number of training periods was 100, and the batch size was 20. During inference, the size of each batch of input images was also uniformly adjusted to 352×352.

[0058] The loss function used in this invention is a structural loss function that is widely applicable to segmentation tasks, as shown in equation (10) below:

[0059] loss = ω1 * loss iou +ω2*loss bce (10)

[0060] Where, loss iou Represents the IOU (weighted intersection-over-union) loss function. bce ω1 represents the cross-entropy loss function, ω2 is the weight value of the IOU loss function, and ω3 is the weight value of the BCE loss function.

[0061] Result verification: Figure 6 The test sample results of a series of segmentation models under the above parameter settings are shown in the figure. The binary segmentation results are used to characterize the type of object to which each pixel in the polyp image belongs. The binary segmentation results fully demonstrate that this method can achieve accurate binary segmentation by acquiring polyp images, extracting features from polyp images, extracting image features from polyp images, and then processing the image features.

[0062] Table 1 presents the measurement and evaluation metrics results for different models on the public dataset ColonDB, namely the mean similarity metric (mean Dice) and the mean absolute error (mean absolute error). The mean similarity metric is calculated as follows:

[0063]

[0064] Among them, P iRepresents the prediction graph, G i The diagram represents the ground truth, where n is the total number of samples, and i = {0, 1, ..., n}. The Dice metric is used to calculate the similarity between two samples, with a threshold of [0, 1]. A higher Dice value indicates better segmentation performance. As shown in Table 1, compared to other models whose mean Dice values ​​range from 0.469 to 0.756, the mean Dice value of this invention is 0.764, achieving the best segmentation performance.

[0065] The Mean Absolute Error is calculated as follows:

[0066]

[0067] Among them, P i Represents the prediction graph, G i The MAE represents the true value graph, where i = {0, 1, 2, ..., n}, ranging from [0, +∞). It equals 0 when the predicted value perfectly matches the true value, representing a perfect model; the larger the error, the larger this value. A smaller MAE value indicates better accuracy of the prediction model. As shown in Table 1, compared to other models whose MAE scores range from 0.038 to 0.094, the MAE score of this invention is 0.030, indicating that the method of this invention has higher segmentation accuracy.

[0068] Table 1 Evaluation index results of different models

[0069] Model Method mean Dice MAE U-net 0.512 0.061 Unet++ 0.483 0.064 SFA 0.469 0.094 DCRNet 0.704 0.052 MSEG 0.735 0.038 ACSNet 0.716 0.039 PraNet 0.712 0.043 EU-Net 0.756 0.045 SANet 0.753 0.043 MSNet 0.755 0.041 COMMA 0.712 0.045 PlutoNet 0.718 N / A BLE-Net 0.731 0.044 Method of the present invention 0.764 0.030

[0070] The above specific embodiments are used to explain and illustrate the present invention, and are only preferred embodiments of the present invention, not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement, characterized in that: The polyp image segmentation method includes data processing, pre-trained weight loading, encoder feature extraction, key pixel extraction, key pixel propagation, local perception enhancement, feature weighted fusion, model training and inference; the pixel dynamic aggregation includes key pixel extraction and key pixel propagation; the polyp image segmentation method is performed according to the following steps: Step (1), data processing: The original polyp images are uniformly adjusted to 352×352, and the normalization method is used to change the image data distribution range from [0,1] to [-1,1], which conforms to the standard normal distribution and makes the model easier to converge; data augmentation is used to expand the number of samples. Five data augmentation methods are used, namely random flipping, random cropping, random rotation, color enhancement, and adding random noise. Among them, color enhancement includes changes in contrast, brightness, intensity, and sharpness. Step (2), pre-trained weight loading: The feature encoder Res2Net-50 loads the publicly available pre-trained weights trained on the ImageNet dataset to initialize the network weights; Step (3), Encoder Feature Extraction: Read the image data processed in step (1). After the Res2Net-50 feature encoder completes the pre-training weight loading in step (2), it extracts multi-scale features E from the input image. i (i = 0, 1, 2, 3, 4), the feature size of the i-th layer is C × H × W, and the feature size of the (i+1)-th layer is 2C × H / 2 × W / 2, where C is the number of feature channels, and H and W are the height and width of the feature, respectively; the deep multi-scale feature E′ in multi-scale features. i (i=1,2,3,4) contains less noise information and is used for pixel dynamic aggregation and local perception enhancement; Step (4), key pixel extraction: extract key pixels from the multi-scale features extracted in step (3), and extract the deep multi-scale features E′ extracted by the feature encoder. i (i = 1, 2, 3, 4) Perform Hadamard products step by step to obtain the information-enhanced deep multi-scale augmented features F. i (i = 1, 2, 3, 4), the i-th deep multi-scale augmentation feature F with a scale of C × H × W. i Downsampled to the (i-1)th deep multi-scale augmentation feature F with a scale of C / 2×2H×2W i-1 With consistent size, the deep multi-scale enhanced features F are then... i and deep multi-scale enhancement features F i-1 The concatenated feature map is then fed into a 3×3 convolution, where the number of channels is reduced from C to 1. An activation function is used to obtain an associated feature map. This associated feature map is then subjected to average pooling and multiplied by the high-resolution feature map. Finally, it is subtracted pixel by pixel from the associated feature map to obtain a structural feature map, which scatters a set of high-resolution key pixels. Simultaneously, the associated feature map is multiplied by the low-resolution feature map using max pooling to obtain a salient region feature map, which scatters a set of low-resolution key pixels. This low-resolution key pixel map is then fused with the high-resolution key pixel map, and the set of pixels with the highest correlation is selected as the key pixels. Step (5), key pixel propagation: The key pixels generated in step (4) are propagated to spread the information of the key pixels from the top layer to the bottom layer of the network structure. After selecting the channel index of the key pixel in the associated feature map, the salient region feature map and the structural feature map propagate the key pixel information. The channel index k of the key pixels is sorted to obtain the key pixel index and key pixel coordinates. Based on the key pixel coordinates, the irregular sampling method is used to enhance the deep multi-scale feature F. i F i-1 Perform sampling to obtain the sampling feature map S i S i-1 Sampling feature map S i S i-1 After multiplication, the sampled feature map is obtained by processing the result with a sigmoid activation function. The similarity feature map is then compared with the sampled feature map S. i After multiplication, it is compared with the sampled feature map S. i-1 Pixel-by-pixel summation yields a structural propagation feature map. The structural feature map is then sampled based on key pixel indices to obtain a pixel-aggregated feature map P. i-1 The key pixel propagation process between different feature layers adopts a propagation mechanism from deep to shallow layers. Step (6), Local Perception Enhancement: The deep multi-scale enhancement features extracted in step (3) are subjected to local perception enhancement, and the deep multi-scale enhancement features F i The modulated feature map is obtained by dynamic convolution and then fed into a dynamic convolutional layer. The efficient fusion characteristic of dynamic convolution is utilized to enhance feature representation. The modulated feature map is then combined with a deep multi-scale enhanced feature map F whose width and length are transformed from H×W to 1×1. i Pixel-wise multiplication is performed to generate a channel modulation feature map. Spatial feature enhancement is then used to extract finer edge details. The channel modulation feature map is flattened, the number of channels is changed from C to 1, and then back to the original number of channels C to obtain a stretched feature map. This channel scaling transformation enhances spatial representation capabilities and amplifies the background difference before and after the polyp. The channel modulation feature map is then fed into dynamic convolution to obtain a channel-spatial modulation feature map, which is multiplied pixel-wise with the stretched feature map to obtain the final local enhancement feature map. During local perceptual enhancement, a propagation method from shallow to deep layers is used to extract more refined local details. Step (7), feature weighted fusion: The pixel aggregation feature map generated by pixel dynamic aggregation in steps (4) and (5) and the local enhancement feature map generated by local perception enhancement in step (6) are subjected to feature weighted fusion operation to obtain the fused feature map. The weights of the pixel aggregation feature map generated by pixel dynamic aggregation and the local enhancement feature map generated by local perception enhancement are 0.9 and 0.1, respectively. The fused feature map is fed into the sigmoid activation function to obtain the final prediction map. Step (8), Model Training and Inference: During the experiment, five widely used public datasets in the polyps field were used for model training and inference: Kvasir-SEG, ClinicDB, ColonDB, Endoscene, and ETIS. When dividing the training, test, and validation sets, 80% of the images in the Kvasir and CVC-ClinicDB datasets were randomly used for training, the remaining 10% for validation, and 10% for testing. All images in the ColonDB, Endoscene, and ETIS datasets were used for testing. During training, the optimizer used was Adam, the learning rate was set to 0.0001, the learning rate adopted a gradient decay strategy with a decay rate of 0.1, a decay period of 50, a total training period of 100, and a batch size of 20. During inference, the size of each batch of input images was also uniformly adjusted to 352×352.

2. The polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement according to claim 1, characterized in that, In step (4), after downsampling, concatenation, and activation function operations, the high-resolution features and low-resolution features are combined with the high-resolution features and scattered to obtain a set of key pixels of high-resolution features. The associated feature map is also combined with the low-resolution feature map and scattered to obtain a set of key pixels of low-resolution features. The number of key pixel indexes is the same as the number of scattered pixels and is a multiple of 16. It can be set to 16, 32, 64, 128, or 256. The method sets it to 32. The set of key pixels of high-resolution features and the set of key pixels of low-resolution features are fused and the set of pixels with the highest correlation is selected as the key pixels.

3. The polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement according to claim 1, characterized in that, In step (5), sampling matrices α in different width W and length H directions are obtained based on the coordinates of the 32 key pixels obtained by sampling. i and α i-1 α i α i-1 After multiplication and fusion, a propagation feature map is obtained. At this point, pixel-by-pixel random sampling is performed using 32 key pixel indices to obtain the final sampled feature map S. i S i-1 .

4. The polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement according to claim 1, characterized in that, In step (6), the deep multi-scale enhancement features are multiplied after channel transformation and dynamic convolutional modulation weights to obtain a channel modulation map, which increases the channel feature representation capability. When the channel dimension is reduced, the channel dimension size changes from C to 1. When the channel dimension is enlarged, the channel dimension size changes from 1 to C. The channel modulation map after spatial dimension scaling transformation is multiplied and combined with the original channel modulation map to obtain a spatial modulation feature map. When the spatial dimension is reduced, the spatial dimension size changes from H×W to 1×1. When the spatial dimension is enlarged, the spatial dimension size changes from 1×1 to H×W.

5. The polyp image segmentation method based on pixel dynamic aggregation and local perception enhancement according to claim 1, characterized in that, In step (7), different weights are assigned to the pixel aggregation feature map generated by dynamic pixel aggregation and the local enhancement feature map generated by local perception enhancement, and weighted fusion is performed. When assigning weights, the pixel aggregation feature map plays a decisive role, and the weight value set is greater than that of the local enhancement feature map, with a value at least greater than 0.5.

Images