A method for detecting colonoscopy lesions based on texture perception and fine-grained feature compensation

Abstract

A method for detecting colonoscopy lesions based on texture perception and fine-grained feature compensation is proposed. The acquired colonoscopy images are input into a backbone network to extract multi-level features. Texture features are mined from feature C2 and compared with feature C... m Semantic feature integration in C m Background information is suppressed, fine-grained texture features are captured and enhanced; features C3 to C m Each feature is convolved separately, and then fused with the enhanced features to obtain m-2 fused features; feature C is then... m Features are obtained through attention mechanisms and spatial pyramid pooling; the obtained features are fused in a multi-scale manner from top to bottom and from bottom to top, and a pyramid-decomposable attention module is introduced in at least one of the fusion processes to balance the channel information in the skip links and capture the interaction between channels. Based on the multi-scale fusion results, high-precision target detection is achieved.

Description

Technical Field

[0001] This invention belongs to the field of computer technology and relates to medical image analysis, particularly to a method for detecting colonoscopy lesions based on texture perception and fine-grained feature compensation. Background Technology

[0002] In the actual practice of colorectal cancer screening using colonoscopy, lesions are often missed or misdiagnosed due to factors such as high similarity between the foreground and background, varying lesion morphology, randomized location, and significant individual patient differences. Computer-aided diagnosis (CAD) technology can assist physicians in performing endoscopic examinations, reducing human error and workload, and improving the current situation of missed or misdiagnosed polyps caused by subjective and objective factors.

[0003] With the development of deep learning (DL), the use of convolutional neural networks (CNNs) for automatic detection of intestinal polyps can achieve relatively good target classification in colonoscopy images. However, in practical applications, for polyps with complex backgrounds or small / medium-sized polyps, firstly, there is limited available information about the target, and existing technologies struggle to accurately extract discriminative information from similar intestinal wall backgrounds. Secondly, because small-scale targets have limited feature information, more fine-grained information is needed for discriminative differentiation. While multi-scale feature learning in existing models can help the model obtain more accurate localization and discriminative information, it is prone to causing the loss of fine-grained features at deeper layers when acquiring multi-scale features. This prevents the network from obtaining discriminative information, leading to incorrect judgments. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the present invention aims to provide a colonoscopy lesion detection method based on texture perception and fine-grained feature compensation. By introducing shallow texture information to preserve the fine-grained features of the target, the method solves the problem of losing subtle differences in deep layers due to CNN operation, thereby improving the detection accuracy of small target polyps.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for detecting colonoscopy lesions based on texture perception and fine-grained feature compensation includes the following steps:

[0007] Step 1: Input the acquired colonoscopy images into the backbone network to extract multi-level features C. j , 2≤j≤m;

[0008] Step 2: Mine texture features from feature C2 and compare them with feature C. m Semantic feature integration in, through feature C mSuppress background information to capture fine-grained texture features;

[0009] Step 3: Enhance the fine-grained texture features;

[0010] Step 4, add features C3 to C m-1 Each feature is convolved separately, and then fused with the enhanced features from step 3 to obtain m-2 fused features P3 to P4. m-1 ;

[0011] Step 5, transfer feature C m Feature P is obtained through attention mechanism and spatial pyramid pooling. m ;

[0012] Step 6: Perform multi-scale feature fusion from top to bottom and from bottom to top on the features obtained in Step 4 and Step 5, and introduce a pyramid-separable attention module in at least one of the fusion processes to balance the channel information in the skip links and capture the interaction between channels.

[0013] Step 7: Perform target detection based on the multi-scale fusion results.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0015] First, to address the issue of subtle differences disappearing at deeper layers, a Texture Awareness Module (TAM) is designed. This module mines the rich texture information at lower layers and utilizes higher-level semantic information for background suppression, thereby capturing cleaner texture information. Second, a Texture Feature Enhancement Module (TFEM) is designed to enhance the lower-level texture information from the TAM. Finally, a Residual Pyramid Splittable Attention Module (RPSA) is proposed to balance the channel information in skip connections, improving the overall detection performance of the network.

[0016] In summary, this invention extracts fine-grained texture features and introduces shallow texture information to preserve the fine-grained features of the target. After enhancement, these features are fused with multi-level features. At the same time, a pyramid-shaped split attention module is introduced to acquire more feature information and capture the interaction between channels, thereby significantly improving the fusion effect and ultimately improving the accuracy of target detection. It is especially suitable for the identification and detection of small polyps in colonoscopy images.

[0017] The effectiveness of the proposed method was validated on four public datasets. Experimental results show that, compared with mainstream detection algorithms, FGFCN improves the detection performance of the model by mining fine-grained features, and the detection accuracy is better than existing methods, thus having greater clinical application value. Attached Figure Description

[0018] Figure 1 A data distribution diagram for different datasets.

[0019] Figure 2 This is a general framework diagram of a colonoscopy lesion detection method based on texture perception and fine-grained feature compensation according to the present invention.

[0020] Figure 3 This is a structural diagram of the Texture Aware Module (TAM).

[0021] Figure 4 This is a structural diagram of the Texture Enhancement Module (TFEM).

[0022] Figure 5 This is a structural diagram of the Residual Pyramid Decomposable Module (RPSA).

[0023] Figure 6 This is a structural diagram of the SPC module in RPSA.

[0024] Figure 7 This is a structural diagram of the SEweight module in RPAS.

[0025] Figure 8 A visualization of the comparison results on the Polysets dataset.

[0026] Figure 9 A visualization of the comparison results on the CVC-ClinicDB dataset.

[0027] Figure 10 Visualize the results of features at different layers.

[0028] In the picture The summation of feature maps is represented by U, upsampling by C, channel summation by S, and the sigmoid activation function by S. This indicates the dot product operation. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0030] Referring to Table 1, the embodiments of the present invention use four datasets for experiments: Polysets, CVC-ClinicDB, LDPoly, and Kvasir-Sessile.

[0031] Polysets is a publicly available polyp dataset from Kaidong Li (Director of Otorhinolaryngology at OspedalePoliclinico San Martino, University of Genova). It consists of intestinal polyp images from multiple centers used for different purposes, including both hyperplastic and adenomatous polyps. This invention extracts 28,773 polyp images from this dataset, of which 25,623 are used as the training set and 3,150 as the test set. In this dataset, each image contains at least one polyp, and the polyps are relatively small with high foreground-background similarity.

[0032] CVC-ClinicDB consists of 612 images of various types of polyps extracted from colonoscopy videos.

[0033] LDPoly consists of 100 video sequences. The image frames in each video are too similar. In order to balance the data, after extraction and post-processing, 965 polyp images from different angles were finally selected.

[0034] Kvasir-Sessile contains 198 images of polyps from different angles within the intestine. Unlike Polysets, CVC-ClinicDB, LDPoly, and Kvasir-Sessile contain multiple targets within a single image, and their data distribution characteristics vary. To verify the model's stability, robustness, and generalization ability, this invention was validated on datasets from different sources with varying distribution characteristics. Specific information about the datasets is shown in Table 1, and the data distribution characteristics of the four datasets are as follows: Figure 1 As shown.

[0035] Table 1. Dataset partitioning methods and details

[0036]

[0037] Using the aforementioned dataset, the colonoscopy lesion detection method of this invention based on texture perception and fine-grained feature compensation was validated, with reference to... Figure 2 The steps include:

[0038] Step 1: Input colonoscopy images into the backbone network to extract multi-level features C j .

[0039] In embodiments of the present invention, the backbone network adopts a Darknet53 backbone network, such as CSPDarknet53, for a given input features. The backbone network can generate features C at different scales from the input image. j Where 2≤j≤m, in Figure 2 In the structural framework shown, m = 5, meaning j takes values ​​of 2, 3, 4, and 5. In the above formula, C is the number of channels, and H × W represents the image resolution.

[0040] Step 2: Mine texture features from feature C2 and compare them with feature C. m Semantic feature integration in, through feature C m Background information is suppressed to capture fine-grained texture features.

[0041] This invention employs a Texture Awareness Module (TAM) to implement this step. The structure of the Texture Awareness Module is described below. Figure 3 For example, when m=5, the Texture Aware Module (TAM) mines fine-grained features of the polyp target from the low-level features C2, which contain local texture details. Since the texture features mined from the low level contain local texture details of the target, but mixed with interference information, while C5 contains global information under object texture supervision and has high-level semantics related to the object, this invention integrates the texture information in C2 with the semantic information in C5, and suppresses background information through C5, thereby capturing high-quality low-level fine-grained texture features, enhancing the ability to represent object discriminable information, and improving the detection performance of the model.

[0042] Furthermore, while the low-level features contain rich fine-grained features, they inevitably introduce noise. Therefore, high-level semantic or positional information is needed to facilitate further filtering of the target texture. Based on this, two 1×1 convolutional layers are first used to change the channels of C2 and C5 to 256 (C2') and 1024 (C5') respectively. Then, the feature (C5') is upsampled, and the two are then concatenated.

[0043] To ensure the obtained texture information more accurately characterizes the target, the above concatenation results are further processed through two 3×3 convolutions and one 1×1 convolution, and finally subjected to the Sigmoid function. TAM is a simple and effective module for extracting texture features. Through rigorous feature selection from coarse to fine granular, it forms new image features, thereby achieving the goal of mining and learning more information about polyp images.

[0044] Step 3: Enhance the obtained fine-grained texture features.

[0045] This invention employs a Texture Feature Enhancement Module (TFEM) to implement this step; the structure of the Texture Feature Enhancement Module is described below. Figure 4 Since fine-grained features are inherently small, to enhance the ability to represent discriminable information, the fine-grained texture features obtained by TAM are further enhanced by TFEM.

[0046] TFEM mainly consists of parallel residual branches with different dilation rates d and a main branch, where the number of residual branches should generally be at least 3. First, the first convolutional layer in each branch utilizes a 1×1 convolution operation (C... onv 1×1) Reduce the channel size to 32; secondly, starting from the second residual branch, the i-th residual branch b i Perform (2i-1)×(2i-1) convolution operations and dilated convolution operations with an expansion rate of (2i-1); then, convert all channels of the first to third residual branches to C through 3×3 convolution operations; finally, concatenate the multi-scale features obtained from each residual branch and then fuse them with the main branch. The fused result is then input into the ReLU function to obtain the enhanced final feature f. k TFEM uses receptive fields of different sizes to capture texture information in images, and further enhances the extracted texture features through dilated convolution, thereby achieving the purpose of enhancing texture features.

[0047] In an embodiment of the present invention, the number of residual branches is taken as 4, and the (2i-1)×(2i-1) convolution operation is decomposed into a sequence of two steps, namely (2i-1)×1 and 1×(2i-1) kernel, wherein:

[0048] The first residual branch b1 performs a 1×1 convolution on the fine-grained texture features, reducing the input feature channel size to C, thus obtaining b1;

[0049] The second residual branch b2 performs 1×1 convolution, 1×3 convolution, and 3×1 convolution on the fine-grained texture features in sequence. Finally, it performs multi-scale processing on the features using dilated convolution with an expansion rate of 3, and then performs 3×3 convolution operation to reduce the channel size to C, thereby obtaining contextual information b2 at a small scale.

[0050] The third residual branch b3 performs 1×1 convolution, 1×5 convolution, and 5×1 convolution on the fine-grained texture features in sequence. Finally, it performs multi-scale processing on the features using dilated convolution with an expansion rate of 5, and then performs 3×3 convolution operation to reduce the channel size to C, thereby obtaining contextual information b3 at the medium scale.

[0051] The fourth residual branch b4 performs 1×1 convolution, 1×7 convolution, and 7×1 convolution on the fine-grained texture features in sequence. Finally, it performs multi-scale processing on the features using dilated convolution with an expansion rate of 7, and then performs 3×3 convolution operation to reduce the channel size to C, thereby obtaining large-scale contextual information b4.

[0052] b1, b2, b3, and b4 are the multi-scale features obtained from each residual branch.

[0053] Step 4: Add features C3 to C m-1 Each feature is convolved separately, and then fused with the enhanced features from step 3 to obtain m-2 fused features P3 to P4. m-1 .

[0054] refer to Figure 2 In this embodiment of the invention, C2 and C5 are processed by the TAM module, and then the output is processed by the TFEM module to be fused with the convolution results of C3 and C4 respectively to obtain fused features P3 and P4.

[0055] Step 5, transfer feature C m Feature P is obtained through attention mechanism and spatial pyramid pooling. m .

[0056] refer to Figure 2 In this embodiment of the invention, C5 is used to obtain feature P5 through attention mechanism and spatial pyramid pooling.

[0057] Step 6, for the features obtained in Steps 4 and 5, i.e., P3 to P m Multi-scale feature fusion is performed from top to bottom and from bottom to top, and a pyramid-separable attention module is introduced in at least one of the fusion processes to balance channel information in skip links and capture the interaction between channels.

[0058] Different feature channels often contain different semantics. During multi-scale feature fusion, channel information can be lost. To enhance the utilization of inter-channel dependencies and achieve robust integration and powerful feature representations, this invention proposes the Residual Pyramid Splittable Attention Module (RPSA) to explore cross-channel interactions and uncover key clues between different channels. This invention uses RPSA to balance channel information in skip links after fusion, capturing inter-channel interactions for inference, thereby improving overall detection performance.

[0059] This invention will feature P m ~P3 performs top-down multi-scale feature fusion to obtain feature F m ~F3, which will be the feature F3~F mPerform bottom-up multi-scale feature fusion to obtain features E3 to E m The pyramid-shaped, separable attention module is introduced into one or more bottom-up fusion processes, and can be further introduced into the feature E. m The output location. Figure 2 The cases F5 through F3 are shown, where RPSA is introduced in each fusion process.

[0060] refer to Figure 5 The pyramid-shaped detachable attention module of the present invention performs the following steps:

[0061] Step (1): Extract multi-scale features using the SPC module.

[0062] The structure of the SPC module is as follows: Figure 6 As shown, the principle is to extract spatial information from the input feature map using a multi-branch approach. Each branch has an input channel dimension of C, which yields richer location information. Subsequently, by compressing the channel dimension of the input tensor, spatial information at different scales on each channel feature can be effectively extracted. Then, for each branch, multi-scale spatial information is learned independently, and cross-channel interactions are established locally. The SPC module effectively extracts multi-scale features from the input image, supplementing feature diversity and thus improving model performance.

[0063] Step (2) uses each feature obtained through the SPC module as input to the SEWeight module to balance channel information. Global average pooling is used to obtain 1×1×C compressed features, which are then activated (excitation) through two fully connected layers. Finally, Softmax is used to recalibrate the channel attention vectors, obtaining recalibrated weights for multi-scale channels to generate new features. Specifically, let... The input features are defined by H, W, and C, which represent the height, width, and number of input channels, respectively. An SEWeight module consists of two parts: squeezing and excitation, used to encode global information and adaptively supplement or calibrate channel relationships, respectively. Channel statistics are generated using global average pooling to embed global spatial information into the channel descriptors. The global average pooling operator is calculated as shown in formula (1):

[0064]

[0065] The attention weight of the Cth channel in the SEWeight module can be written as shown in formula (2):

[0066] w c =σ(W1δ(W0(g) c (2)

[0067] The symbol δ represents the regular linear unit (ReLU) operation. and The term "full connected layer" (FC) represents a two-layer fully connected layer. This effectively combines linear information between channels, facilitating the interaction of high-dimensional and low-dimensional information. The symbol σ represents the activation function. By using the activation function, weights are assigned to channels after their interaction, thus extracting information more effectively. The SEWeight module structure is as follows: Figure 7 As shown.

[0068] In step (3), after passing through the SPC and SEWeight modules, the importance of each channel is adjusted according to its weight. To compensate for information loss while establishing remote channel dependencies, the new features from step (2) are multiplied with the input features to form residual connections. RPSA is used to better coordinate channel information and assist in the fusion of multi-scale features from top to bottom and bottom to top, respectively.

[0069] Step 7: Perform target detection based on the multi-scale fusion results.

[0070] The detection results of the embodiments are evaluated:

[0071] This invention uses the mean average precision (mAP), a commonly used metric in object detection, as a key evaluation indicator to measure the accuracy between the predicted image and the label. Here, mAP@0.5 means that the IOU is set to 0.5, and the AP of all images in each class is calculated and averaged. Similarly, mAP@0.75 and mAP@0.5-0.95 represent the average mAP at different IOU thresholds (from 0.5 to 0.95, with a step size of 0.05). The calculation method is as shown in formula (5), where Q is the number of samples and q is the lookup value for mean precision.

[0072]

[0073] To evaluate the effectiveness of the proposed module, this invention demonstrates the effectiveness of the proposed model from two aspects. (1) The model proposed in this invention is compared with other existing detectors on four datasets, and the results are shown in Tables 2, 3, 4, 5, and 6. (2) The generalization ability of the model is verified by testing with data of different distributions. (3) Ablation experiments are conducted on different datasets, and the results are shown in Tables 7 and 8.

[0074] (1) Comparative Experiment

[0075] Single dataset training

[0076] The effectiveness of the proposed FGFCN model is verified by comparing its detection performance with various object detection algorithms on the Polysets and CVC-ClinicDB datasets. The evaluation results are shown in Tables 2 and 3, and the visualization results are as follows: Figure 8 , Figure 9 As shown.

[0077] Table 2 Comparative experiments on the Polysets dataset

[0078]

[0079]

[0080] Table 3 Comparative experiments on the CVC-ClinicDB dataset.

[0081] Baseline Years mAP@0.5 mAP@0.75 mAP@0.5:0.95 <![CDATA[Faster RCNN

[26] ]]> 2015 86.1 45.6 47.5 <![CDATA[SSD

[27] ]]> 2016 91.9 57.5 53.1 <![CDATA[CenterNet

[28] ]]> 2019 52.3 12.8 21.1 <![CDATA[Yolov3(v9.6.0)

[29] ]]> 2021 96.5 77.8 68.0 <![CDATA[Yolov4

[30] ]]> 2020 76.9 44.0 44.4 <![CDATA[YoloX

[31] ]]> 2021 89.7 49.5 49.9 <![CDATA[Yolov5(v6.1)

[32] ]]> 2022.2 95.5 85.7 66.6 <![CDATA[Yolov6

[33] ]]> 2022.6 93.5 72.4 62.0 <![CDATA[Yolov7

[34] ]]> 2022.7 81.3 50.9 45.8 <![CDATA[Yolov8

[35] ]]> 2023.1 97.2 82.2 66.8 OURS --- 98.0 85.3 68.6

[0082] Training on multiple datasets

[0083] To validate the model's effectiveness, this part of the work was performed on the CVC-ClinicDB and LDPoly datasets. To more fully demonstrate the model's performance, the CVC-LDP dataset (which represents training the model by merging the training sets of the CVC-ClinicDB and LDPoly datasets, and then testing them separately) was used to verify the model's stability on data with different distributions. The comparison results are shown in Table 4.

[0084] Table 4 Comparative experiments on the CVC-LDP dataset.

[0085]

[0086]

[0087] (2) Model generalization verification

[0088] To verify the generalization performance of the model, CVC / LDP (meaning the model was trained using all data from CVC-ClinicDB and tested using all data from LDPoly) and CVC-LDP / Kvasir (meaning the model was trained using all data from ClinicDB and LDPoly and tested using all data from Kvasir-Sessile) were used to verify the model's generalization performance. The comparison results are shown in Tables 5 and 6.

[0089] Table 5 Comparative experiments on the CVC / LDP dataset.

[0090] Baseline Years mAP@0.5 mAP@0.75 mAP@0.5:0.95 Faster RCNN 2015 62.3 33.4 35.9 SSD 2016 33.1 27.7 23.4 CenterNet 2019 52.0 23.2 26.3 Yolov3 (v9.6.0) 2021 40.7 26.4 24.4 Yolov4 2020 42.0 17.6 20.6 YoloX 2021 52.4 30.5 30.0 Yolov5 (v6.1) 2022.2 55.8 27.7 29.7 Yolov6 2022.6 49.4 24.1 25.8 Yolov7 2022.7 24.2 11.1 11.9 Yolov8 2023.1 31.3 12.9 15.3 OURS --- 60.2 41.2 37.3

[0091] Table 6 Comparative experiments on the CVC-LDP / Kvasir dataset.

[0092]

[0093]

[0094] (3) Ablation test

[0095] To further verify the effectiveness of the proposed method, the proposed modules RPSA, EAM, and TEFM were validated on the single dataset Polysets and the multi-dataset CVC-LDP (meaning the training sets of the CVC and LDP datasets were combined for model training, and the remaining data from both datasets were used for testing). E.1 represents the baseline model. The ablation experiment results on the two datasets are shown in Tables 7 and 8, respectively. Figure 10 This is a visualization of the proposed module before and after applying features from different layers on the Polysets dataset.

[0096] Table 7 Ablation experiments on the Polysets dataset

[0097] Index Model Arch mAP@0.5 mAP@0.75 mAP@0.5:0.95 E.1 Yolov5 (V6.1m) 99.5 98.3 87.7 E.2 E.1+EIOU+Mish 99.5 98.5 87.8 E.3 E.1+PSA+EIOU+Mish 99.5 98.8 88.1 E.4 E.1+RPSA+EIOU+Mish 99.5 98.7 88.4 E.5 E.1+RPSA+TAM+EIOU+Mish 99.5 98.6 88.7 E.6 E.1+RPSA+TAM+TEFM+EIOU+Mish 99.5 98.7 88.9

[0098] Table 8 Ablation experiments on the CVC-LDP dataset

[0099] Index Model Arch mAP@0.5 mAP@0.75 mAP@0.5:0.95 E.1 Yolov5 (V6.1m) 94.3 63.0 58.8 E.2 E1+EIOU+Mish 94.0 66.4 59.0 E.3 E1+PSA+EIOU+Mish 94.2 65.4 58.9 E.4 E1+RPSA+EIOU+Mish 94.5 67.2 59.3 E.5 E1+RPSA+TAM+EIOU+Mish 93.9 67.4 59.8 E.6 E1+RPSA+TAM+TEFM+EIOU+Mish 95.1 70.7 62.8

[0100] Overall, FGFCN outperforms all other models on all evaluation metrics, including mAP@0.75 and mAP@0.5-0.95, achieving the highest values ​​on all three metrics. Compared to the baseline network (Yolov5), FGFCN improves performance by 0.8, 7.7, and 4 on each evaluation metric, validating the model's stability and robustness.

[0101] Finally, both subjectively and objectively, FGFCN demonstrates good performance on four publicly available datasets, especially showing significant advantages in mAP@0.75 and mAP@0.5:0.95, making it more suitable for lesion detection tasks compared to other models. All the results of the above experiments quantitatively and qualitatively demonstrate the effectiveness of the strategy of compensating for deep feature loss and utilizing shallow texture for intestinal polyp detection. The framework of this invention can detect and automatically identify the specific location and type of lesions in electronic colonoscopy images during colonoscopy.

[0102] In practical applications, the steps of this invention can be written into a computer program and applied to an identification or detection instrument. That is, it can be a standalone identification instrument that detects and identifies polyps from external colonoscopy images. Alternatively, it can be integrated with a colonoscope, serving as part of the colonoscope, and directly receiving images after image acquisition to detect polyps.

[0103] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made in accordance with the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting colonoscopy lesions based on texture perception and fine-grained feature compensation, characterized in that, Includes the following steps: Step 1: Input the acquired colonoscopy images into the backbone network to extract multi-level features C. j , 2≤j≤m; Step 2: Mine texture features from feature C2 and compare them with feature C. m Semantic feature integration in, through feature C m Suppress background information to capture fine-grained texture features; Step 3: Enhance the fine-grained texture features; Step 4, add features C3 to C m-1 Each feature is convolved separately, and then fused with the enhanced features from step 3 to obtain m-2 fused features P3 to P4. m-1 ; Step 5, add feature C m Feature P is obtained through attention mechanism and spatial pyramid pooling. m ; Step 6: Perform multi-scale feature fusion from top to bottom and from bottom to top on the features obtained in Step 4 and Step 5, and introduce a pyramid-separable attention module in at least one of the fusion processes to balance the channel information in the skip links and capture the interaction between channels. Step 7: Perform target detection based on the multi-scale fusion results; in: In step 2, two 1×1 convolutional layers are first used to convolve C2 and C... m The channels are changed to 256 and 1024, then the features are upsampled, followed by concatenation, then two 3×3 convolutions and one 1×1 convolution, and finally the Sigmoid function is applied. In step 3, at least three parallel residual branches with different expansion rates d and one main branch are used. First, the first convolutional layer in each branch reduces the channel size to 32 using a 1×1 convolution operation. Second, starting from the second residual branch, the i-th residual branch b... i Perform (2i-1)×(2i-1) convolution operations and dilated convolution operations with an expansion rate of 2i-1; then, convert all channels of the first to third residual branches to C through 3×3 convolution operations; finally, concatenate the multi-scale features obtained from each residual branch and then fuse them with the main branch. The fused result is then input into the ReLU function to obtain the enhanced final feature f. k '.

2. The colonoscopy lesion detection method based on texture perception and fine-grained feature compensation according to claim 1, characterized in that, The backbone network is the Darknet53 backbone network, with m=5.

3. The colonoscopy lesion detection method based on texture perception and fine-grained feature compensation according to claim 1, characterized in that, The number of residual branches is 4, and the (2i-1)×(2i-1) convolution operation is decomposed into a sequence of two steps, namely (2i-1)×1 and 1×(2i-1) kernels, where: The first residual branch b1 performs a 1×1 convolution on the fine-grained texture features, reducing the input feature channel size to C, thus obtaining b1; The second residual branch b2 performs 1×1 convolution, 1×3 convolution, and 3×1 convolution on the fine-grained texture features in sequence. Finally, it performs multi-scale processing on the features using dilated convolution with an expansion rate of 3, and then performs 3×3 convolution operation to reduce the channel size to C, thereby obtaining contextual information b2 at a small scale. The third residual branch b3 performs 1×1 convolution, 1×5 convolution, and 5×1 convolution on the fine-grained texture features in sequence. Finally, it performs multi-scale processing on the features using dilated convolution with an expansion rate of 5, and then performs 3×3 convolution operation to reduce the channel size to C, thereby obtaining contextual information b3 at the medium scale. The fourth residual branch b4 performs 1×1 convolution, 1×7 convolution, and 7×1 convolution on the fine-grained texture features in sequence. Finally, it performs multi-scale processing on the features using dilated convolution with an expansion rate of 7, and then performs 3×3 convolution operation to reduce the channel size to C, thereby obtaining large-scale contextual information b4. b1, b2, b3, and b4 are the multi-scale features obtained from each residual branch.

4. The colonoscopy lesion detection method based on texture perception and fine-grained feature compensation according to claim 1, characterized in that, In step 4, feature P m ~P3 performs top-down multi-scale feature fusion to obtain feature F m ~F3, which will be the feature F3~F m Perform bottom-up multi-scale feature fusion to obtain features E3 to E m The pyramid-shaped split attention module is introduced into one or more fusion processes from the bottom up.

5. The colonoscopy lesion detection method based on texture perception and fine-grained feature compensation according to claim 4, characterized in that, The pyramid-shaped split attention module is introduced in feature E m The output location.

6. The method for detecting colonoscopy lesions based on texture perception and fine-grained feature compensation according to claim 1, 3, 4, or 5, characterized in that, The pyramid-shaped, detachable attention module performs the following steps: Step (1): Extract multi-scale features using the SPC module; Step (2) uses each feature obtained through the SPC module as input to the SEWeight module to balance channel information. It obtains 1×1×C compressed features through global average pooling, and then performs activation (excitation) operation through two fully connected layers. Finally, it uses Softmax to recalibrate the channel attention vectors to obtain recalibrated weights for multi-scale channels and generate new features. Step (3) involves performing a dot product operation between the new features from step (2) and the input features to form a residual connection.

7. The colonoscopy lesion detection method based on texture perception and fine-grained feature compensation according to claim 6, characterized in that, The SPC module extracts spatial information of the input feature map in a multi-branch manner. The input channel dimension of each branch is C, and the channel dimension of the input tensor is compressed. For each branch, multi-scale spatial information is learned independently and cross-channel interaction is established in a local manner. The SEWeight module is set as follows: The input features are defined by H, W, and C, which represent the height, width, and number of input channels, respectively. An SEWeight module consists of two parts: squeezing and excitation, which are used to encode global information and adaptively supplement or calibrate channel relationships, respectively. Channel statistics are generated by using global average pooling to embed global spatial information into the channel descriptors.

8. The colonoscopy lesion detection method based on texture perception and fine-grained feature compensation according to claim 7, characterized in that, The global average pooling operator is calculated as follows: The attention weight of the Cth channel in the SEWeight module is shown in the following formula: w c =σ(W1δ(W0(g c ))) The symbol δ represents the regular linear unit operation. and This represents a fully connected layer, which combines linear information between two fully connected layers to achieve the interaction of high-dimensional and low-dimensional information in the channels. The symbol σ represents the activation function.

Images