Rectum polyp detection method, system and device based on DFIB and multilayer aggregation network, and storage medium
By introducing DFIB modules and multi-layer polymerization networks into the YOLOv8 network, the shortcomings of existing polyp detection technologies in terms of detection accuracy are solved, and higher polyp detection accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202411565413.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-05
- Publication Date
- 2025-02-21
AI Technical Summary
The existing polyp detection technology has shortcomings in detection accuracy, especially in capturing small-sized polyps and edge fuzzy polyps, taking into account global and local information, and processing multi-scale features, which has led to the small-sized system for clinical applications. The detection rates of polyps and occult polyps did not meet expectations.
The rectal polyp detection method based on DFIB and multi-layer polymerization network is adopted. By replacing the C2f module with the DFIB module in the YOLOv8 network, it is used to extract the characteristics of receptive field expansion, and a multi-layer polymerization network is applied to the neck of the network, with different fusion and weighting. stage features to enhance the model's ability to capture key features in polyp detection.
It improves the accuracy and robustness of polyp detection, can capture the key features of polyp more comprehensively, enhances the processing ability of multi-scale targets, and improves the detection effect of small targets.
Smart Images
Figure CN119494827A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method, system, device and storage medium for detecting rectal polyps, and in particular to a method, system, device and storage medium for detecting rectal polyps based on DFIB and a multi-layer aggregation network. Background Art
[0002] Colorectal cancer (CRC) is the third most common cancer in the world and has a high mortality rate. Colonoscopy is the gold standard for CRC screening. Population-based screening programs have been proven to effectively reduce the mortality and morbidity of CRC. The core of the program is to detect and remove adenomatous polyps that may induce CRC. Early detection can significantly improve the five-year survival rate from 18% to 88.5%. The adenoma detection rate (ADR) is an important indicator for evaluating the quality of colonoscopy. For every 1% increase in ADR, the risk of CRC can be reduced by 3%. However, the missed detection rate of polyps ranges from 6% to 27%, which may be affected by factors such as polyp characteristics, bowel preparation quality, physician experience or fatigue. Studies have shown that adding a second observer to colonoscopy can improve the polyp detection rate, although the effect of improving ADR is questionable.
[0003] With the development of technology, computer-aided diagnosis (CAD) systems based on artificial intelligence have been developed to assist endoscopists in identifying polyps more effectively. In recent years, several polyp detection systems have adopted architectures based on convolutional neural networks (CNNs). For example, PLPNet developed by Jia et al. uses improved learning techniques and feature pyramid structures; Wittenberg et al. use a Mask R-CNN-based method to perform transfer learning using pre-trained weights; and Shin et al. developed a system based on Faster R-CNN to reduce the false positive rate. The YOLO-based system proposed by Zheng et al. supports real-time detection, although its performance is slightly lower. The research of Jha et al. showed that YOLOv4 performed well in both speed and detection performance, meeting the clinical needs of a processing speed of at least 30 frames per second.
[0004] Although the current polyp detection technology has made some progress, there are still many shortcomings in detection accuracy. First, the existing convolutional neural network (CNN) architecture tends to ignore small-sized polyps and polyps with blurred edges when extracting features, making it difficult to accurately detect these targets with special morphology. Secondly, due to the limited feature fusion capability of the feature pyramid structure or transfer learning model, it is difficult to take into account both global and local information, resulting in a decrease in the model's ability to distinguish polyps in complex backgrounds. In addition, the current system has limitations in the extraction of multi-scale features and cannot fully express the details and semantic information of polyps, especially in the detection of low-contrast or irregularly shaped polyps. The above shortcomings have caused some CAD systems to fail to meet expectations in the detection rate of small polyps and occult polyps in clinical applications, affecting the overall accuracy and reliability of colonoscopy. Summary of the invention
[0005] Purpose of the invention: The purpose of the present invention is to provide a method, system, device and storage medium for detecting rectal polyps based on DFIB and multi-layer aggregation network, which can improve detection accuracy.
[0006] Technical solution: The present invention discloses a method for detecting rectal polyps based on DFIB and multi-layered aggregation network, comprising:
[0007] Create an image dataset containing rectal polyp targets, annotate the dataset, and divide the dataset into training set, test set, and validation set;
[0008] Improvements are made based on the YOLOv8 network to construct a rectal polyp detection model; the improvements include replacing the C2f module in the YOLOv8 network with a DFIB module to extract features for expanding the receptive field; applying a multi-layer aggregation network to the neck of the YOLOV8n network to fuse and weight features at different stages;
[0009] The rectal polyp detection model is trained, and the trained rectal polyp detection model is used to detect target rectal polyps.
[0010] Furthermore, the step of preparing a picture dataset containing rectal polyp targets, annotating the dataset, and dividing the dataset into a training set, a test set, and a validation set includes:
[0011] Collect images containing rectal polyp targets; convert all images into a preset format; use annotation tools to annotate all image datasets; divide the prepared datasets into training sets, test sets, and validation sets according to the proportions;
[0012] The rectal polyp detection model is trained using a training set and a test set.
[0013] Furthermore, the DFIB module includes average pooling AvgPool and maximum pooling MaxPool; through multi-level and multi-scale feature extraction and fusion, it reduces information loss caused by single scale or channel limitation, and is used to enhance the model's ability to capture key features.
[0014] Furthermore, the DFIB module implementation method is as follows: first, the input feature map is split to obtain the first and second parts after the split; the first part of the features enters the average pooling AvgPool and the maximum pooling MaxPool for processing, and then the processed features are integrated and reduced in dimension through a 1x1 convolution layer, and then finally processed through a Sigmoid activation function; the second part of the features enters a sequence composed of multiple Bottleneck layers, and the features after Bottleneck processing are integrated through another 1x1 convolution layer and fused with the features of the first part after final processing.
[0015] Furthermore, the multi-layer aggregation network reduces the spatial size of the feature map by downsampling, increases the depth and semantic complexity of the feature map, and effectively combines low-level features with high-level features by upsampling, thereby improving the detection capability of small targets.
[0016] Furthermore, the multi-layer aggregation network interweaves up and down sampling paths so that feature maps at different levels are alternately connected and fused, and establishes direct up and down sampling paths between the feature maps of the bottom, middle and high layers. The specific implementation method is: the low-level feature map is transferred layer by layer to the feature map of the deeper layer through downsampling, the spatial resolution of the feature map is reduced, the abstractness of the semantic information is increased, and the network extracts high-level semantic features; the high-level feature map is transferred back to the low-level feature map layer by layer through upsampling, and the high-level semantic features are transferred to the low layer, so that the spatial detail information of the low layer is combined with the high-level semantic information.
[0017] Furthermore, the training of the rectal polyp detection model includes:
[0018] Input the training set into the rectal polyp detection model for training, train a number of images in each batch, and define the completion of training for all images in the training set as one round of training;
[0019] During the training, the image size is adjusted to a preset ratio. The rectal polyp detection model continuously extracts features and performs two-fold upsampling on the input image until the feature map size reaches the preset requirement. The prediction head performs prediction on feature maps of three sizes: 80×80, 40×40, and 20×20. When the specified training round is reached, the result with the highest map@.5 value in all rounds is selected and saved to complete the training.
[0020] Based on the same inventive concept, the present invention also provides a rectal polyp detection system based on DFIB and multi-layered aggregation network, comprising:
[0021] A preprocessing module is used to create an image dataset containing rectal polyp targets, annotate the dataset, and divide the dataset into a training set, a test set, and a validation set;
[0022] A model building module, used for improving the YOLOv8 network to build a rectal polyp detection model; the improvement includes replacing the C2f module in the YOLOv8 network with a DFIB module to extract features of expanded receptive field; applying a multi-layer aggregation network to the neck of the YOLOV8n network to fuse and weight features at different stages;
[0023] The monitoring module is used to train the rectal polyp detection model and use the trained rectal polyp detection model to detect target rectal polyps.
[0024] Based on the same inventive concept, the present invention also provides a computing device, comprising: one or more processors, one or more memories and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the rectal polyp detection method based on DFIB and multi-layer aggregation network as described in any one of the above items are implemented.
[0025] Based on the same inventive concept, the present invention also provides a storage medium, which stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the steps of the rectal polyp detection method based on DFIB and multi-layer aggregation network according to any one of the above-mentioned items.
[0026] Beneficial effects: Compared with the prior art, the present invention aims at the problem that the current method may not capture sufficiently representative features to distinguish polyps and non-polyp tissues, and designs a Dual Feature Integration Block (DFIB) module. Through multi-level and multi-scale feature extraction and fusion, the information loss caused by single scale or channel limitation is reduced to enhance the model's ability to capture key features in polyp detection; the existing detection system focuses on the extraction of global features and ignores the challenge of local details. This paper introduces Multi-Layer Path Aggregation Network (MLPAN). Through the interweaving of upsampling and downsampling paths, MLPAN can better handle targets of various sizes to cope with the multi-scale changes of polyps; by integrating DFIB and MLPAN, this paper constructs DM-Ne. The DFIB module is responsible for multi-level and multi-scale feature extraction and fusion, while the MLPAN module further enhances the processing capability of multi-scale targets through the interweaving of upsampling and downsampling paths. DM-Net can more comprehensively capture the key features of polyps and improve the accuracy and robustness of polyp detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 is a flow chart of a method according to an embodiment of the present invention;
[0028] Figure 2 Schematic diagram of the DFIB module structure of an embodiment of the present invention;
[0029] Figure 3 A schematic diagram of a multi-layer aggregation network structure according to an embodiment of the present invention;
[0030] Figure 4 This is a schematic diagram of the structure of a rectal polyp detection model according to an embodiment of the present invention;
[0031] Figure 5 2 is a diagram showing the detection results of an embodiment of the present invention. DETAILED DESCRIPTION
[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0033] As attached Figure 1 As shown, the rectal polyp detection method based on DFIB and multi-layer aggregation network of this embodiment includes:
[0034] S1: Create an image dataset containing rectal polyp targets, annotate the dataset, and divide the dataset into training set, test set, and validation set;
[0035] S2: Improvements are made based on the YOLOv8 network to build a rectal polyp detection model; the improvements include replacing the C2f module in the YOLOv8 network with the DFIB module to extract features for expanding the receptive field; applying a multi-layer aggregation network to the neck of the YOLOV8n network to fuse and weight features at different stages;
[0036] S3: Train the rectal polyp detection model, and use the trained rectal polyp detection model to detect target rectal polyps.
[0037] Specifically, in step S1, a picture data set containing rectal polyp targets is prepared, and the data set is annotated and divided according to an appropriate ratio.
[0038] Create an image dataset containing rectal polyp targets, annotate the dataset, and divide it according to an appropriate ratio; convert all images into jpg format; annotate all image datasets, use labelimg as the annotation tool, use a rectangle as the annotation box, and annotate rectal polyps as the target; the requirement for the rectangular annotation box is the minimum rectangle that encloses a single target, and multiple targets are allowed to be annotated in one image; modify all image file names to match the corresponding label file names.
[0039] The created data set is divided into a training set and a validation set according to a ratio of 8.5:1.5. In this embodiment, 851 training set images and 150 validation set images are obtained.
[0040] In step S2, the DFIB module and multi-layer aggregation network are designed and integrated into YOLOv8n. The DFIB module reduces the information loss caused by single scale or channel limitation through multi-level and multi-scale feature extraction and fusion to enhance the model's ability to capture key features in polyp detection.
[0041] For the DFIB module, its structure is as follows Figure 2 As shown in the figure, it includes average pooling (AvgPool) and maximum pooling (MaxPool); the DFIB module starts with the splitting of the input feature map. After the splitting, part of the features enter the average pooling (AvgPool) and the maximum pooling (MaxPool). Next, the processed features are further integrated and reduced in dimension through a 1x1 convolution layer. Then, these features will pass through the Sigmoid activation function. The other part of the features enters a sequence composed of multiple Bottleneck layers. The features after the Bottleneck processing are integrated through another 1x1 convolution layer and fused with the previous features.
[0042] For multi-layer aggregation structure MLPAN, such as Figure 3 As shown, in Figure 3Figure (a) shows the FPN structure, and Figure (b) shows the PAN structure. The red line indicates the upsampling operation, and the blue line indicates the downsampling operation. Figure (c) shows the specific structure of the multi-layer aggregation network (MLPAN). Through the interlaced design of upsampling and downsampling paths, the effective fusion of multi-scale features is achieved. In this structure, feature maps at different levels are alternately connected and fused to enhance the diversity and richness of feature expression. The network establishes direct upsampling and downsampling paths between the feature maps of the bottom, middle and high layers, where the blue arrows indicate the downsampling operation and the red arrows indicate the upsampling operation. Specifically, the low-level feature maps are passed layer by layer to the deeper feature maps through downsampling. This operation reduces the spatial resolution of the feature maps, but increases the abstractness of the semantic information, enabling the network to extract higher-level semantic features. At the same time, the network passes the high-level feature maps back to the low-level feature maps layer by layer through upsampling, passing richer high-level semantic features to the low layers, so that the spatial detail information of the low layers can be combined with the high-level semantic information. This alternating fusion structure of upsampling and downsampling paths enables the features at different levels to be fully complementary in space and semantics, which not only retains fine-grained spatial details but also enhances global semantic information. In addition, the intersection of upsampling and downsampling paths between feature maps at each layer enables the network to better adapt to the detection needs of targets of different scales and improves the detection effect of small targets. Through this interwoven multi-layer fusion mechanism, MLPAN has significant advantages in multi-scale feature expression, thereby enhancing the adaptability and robustness of the network to complex detection tasks.
[0043] In the MLPAN structure, downsampling and upsampling paths are crucial to achieve effective fusion of features at different scales. Downsampling increases the depth and semantic complexity of feature maps by reducing the spatial size of feature maps, which helps capture higher-level abstract features, while upsampling helps effectively combine low-level features with high-level features, improving the detection capability of small targets. Through the interweaving of these paths, MLPAN is able to better handle targets of various sizes, especially in multi-scale polyp detection scenarios. The design of this structure not only improves the expressive power of features, but also enhances the adaptability and accuracy of the network to polyp targets of different sizes.
[0044] This embodiment integrates multiple convolutional neural network operations, optimizes feature extraction and fusion, and enhances the model's ability to identify and detect targets. The rectal polyp detection model structure of this embodiment is as follows: Figure 4 shown.
[0045] In step S3, the built model is trained using the training set and the test set to obtain the training results. The specific implementation process is as follows:
[0046] The data set was input into the built model for training. Each batch of 16 images was trained. One round of training was completed after all the images in the training set were trained. A total of 200 rounds were trained. The initial learning rate was set to 0.01, and the final learning rate was 0.001. The training used a variety of data enhancement strategies such as random splicing, random mixing, and random left-right flipping. The usage rate of random splicing was 100%, the usage rate of random left-right flipping was 50%, the usage rate of random mixing was 15%, and the usage rate of random copy and paste was 50%. The total video memory used for model training was 32G, consisting of two GPUs with 16G video memory. The GPU model was Teala V100. The details of the training environment are shown in Table 1.
[0047] Table 1 Training environment information table
[0048]
[0049] During training, the image size is uniformly adjusted to 640×640. The model will continuously extract features and downsample the input image by two times until the feature map size reaches 20×20. The prediction head will make predictions on feature maps of 80×80, 40×40, and 20×20 sizes. The prediction will generate a rectangular box to mark the target position. After reaching the specified training round, the best result of all rounds is selected and saved to obtain the training weight file Train_Result.pt. The training results are shown in Table 2.
[0050] Table 2 Training results
[0051]
[0052] In the table, map@.5 represents the mean average precision when the IoU threshold is 0.5.
[0053] Compared with YOLOv8n, the rectal polyp detection method based on DFIB and multi-layer aggregation shows good precision, recall rate and map@.5. The specific data are shown in Table 3.
[0054] Table 3 Effect comparison table
[0055]
[0056] Using the training results to detect rectal polyp targets, the specific steps are as follows:
[0057] Use the above Train_Result.pt to detect the pictures in the test set; the detection results are marked with a rectangular box, and the category information and confidence value are marked, such as Figure 5 shown.
[0058] Based on the same inventive concept, this embodiment also provides a rectal polyp detection system based on DFIB and multi-layer aggregation network, including:
[0059] A preprocessing module is used to create an image dataset containing rectal polyp targets, annotate the dataset, and divide the dataset into a training set, a test set, and a validation set;
[0060] A model building module, used for improving the YOLOv8 network to build a rectal polyp detection model; the improvement includes replacing the C2f module in the YOLOv8 network with a DFIB module to extract features of expanded receptive field; applying a multi-layer aggregation network to the neck of the YOLOV8n network to fuse and weight features at different stages;
[0061] The monitoring module is used to train the rectal polyp detection model and use the trained rectal polyp detection model to detect target rectal polyps.
[0062] Based on the same inventive concept, this embodiment also provides a computing device, including: one or more processors, one or more memories, and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the rectal polyp detection method based on DFIB and multi-layer aggregation network as described in any one of the above items are implemented.
[0063] Based on the same inventive concept, this embodiment also provides a storage medium, which stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the steps of the rectal polyp detection method based on DFIB and multi-layer aggregation network according to any one of the above-mentioned items.
[0064] The storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer.
[0065] The processor is used to execute the computer program stored in the memory to implement each step of the method involved in the above embodiment.
Claims
1. A method for detecting rectal polyps based on DFIB and multi-layer aggregation network, characterized in that: include: Create an image dataset containing rectal polyp targets, annotate the dataset, and divide the dataset into training set, test set, and validation set; Improvements are made based on the YOLOv8 network to construct a rectal polyp detection model; the improvements include replacing the C2f module in the YOLOv8 network with a DFIB module to extract features for expanding the receptive field; applying a multi-layer aggregation network to the neck of the YOLOV8n network to fuse and weight features at different stages; The rectal polyp detection model is trained, and the trained rectal polyp detection model is used to detect target rectal polyps.
2. The method for detecting rectal polyps based on DFIB and multi-layer aggregation network according to claim 1, characterized in that: The method of preparing a picture dataset containing rectal polyp targets, annotating the dataset, and dividing the dataset into a training set, a test set, and a validation set includes: Collect images containing rectal polyp targets; convert all images into a preset format; use annotation tools to annotate all image datasets; divide the prepared datasets into training sets, test sets, and validation sets according to the proportions; The rectal polyp detection model is trained using a training set and a test set.
3. The method for detecting rectal polyps based on DFIB and multi-layer aggregation network according to claim 1, characterized in that: The DFIB module includes average pooling AvgPool and maximum pooling MaxPool; through multi-level and multi-scale feature extraction and fusion, it reduces the information loss caused by single scale or channel limitation, and is used to enhance the model's ability to capture key features.
4. The method for detecting rectal polyps based on DFIB and multi-layer aggregation network according to claim 3, characterized in that: The DFIB module implementation method is as follows: first, the input feature map is split to obtain the first and second parts after the split; the first part of the features enters the average pooling AvgPool and the maximum pooling MaxPool for processing, and then the processed features are integrated and reduced in dimension through a 1x1 convolution layer, and then finally processed through a Sigmoid activation function; the second part of the features enters a sequence composed of multiple Bottleneck layers, and the features after Bottleneck processing are integrated through another 1x1 convolution layer and fused with the features of the first part after final processing.
5. The method for detecting rectal polyps based on DFIB and multi-layer aggregation network according to claim 1, characterized in that: The multi-layer aggregation network reduces the spatial size of the feature map by downsampling, increases the depth and semantic complexity of the feature map, and effectively combines low-level features with high-level features by upsampling, thereby improving the detection capability of small targets.
6. The method for detecting rectal polyps based on DFIB and multi-layer aggregation network according to claim 5, characterized in that: The multi-layer aggregation network interweaves up and down sampling paths so that feature maps at different levels are alternately connected and fused, and establishes direct up and down sampling paths between the feature maps of the bottom, middle and high layers. The specific implementation method is: the low-level feature map is transferred layer by layer to the feature map of the deeper layer through downsampling, the spatial resolution of the feature map is reduced, the abstractness of the semantic information is increased, and the network extracts high-level semantic features; the high-level feature map is transferred back to the low-level feature map layer by layer through upsampling, and the high-level semantic features are transferred to the low layer, so that the spatial detail information of the low layer is combined with the high-level semantic information.
7. The method for detecting rectal polyps based on DFIB and multi-layer aggregation network according to claim 1, characterized in that: The training of the rectal polyp detection model includes: Input the training set into the rectal polyp detection model for training, train a number of images in each batch, and define the completion of training for all images in the training set as one round of training; During the training, the image size is adjusted to a preset ratio. The rectal polyp detection model continuously extracts features and performs two-fold upsampling on the input image until the feature map size reaches the preset requirement. The prediction head performs prediction on feature maps of three sizes: 80×80, 40×40, and 20×20. When the specified training round is reached, the result with the highest map@.5 value in all rounds is selected and saved to complete the training.
8. A rectal polyp detection system based on DFIB and multi-layer aggregation network, characterized in that: include: A preprocessing module is used to create an image dataset containing rectal polyp targets, annotate the dataset, and divide the dataset into a training set, a test set, and a validation set; A model building module, used for improving the YOLOv8 network to build a rectal polyp detection model; the improvement includes replacing the C2f module in the YOLOv8 network with a DFIB module to extract features of expanded receptive field; applying a multi-layer aggregation network to the neck of the YOLOV8n network to fuse and weight features at different stages; The monitoring module is used to train the rectal polyp detection model and use the trained rectal polyp detection model to detect target rectal polyps.
9. A computing device, characterized in that include: One or more processors, one or more memories, and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the rectal polyp detection method based on DFIB and multi-layer aggregation network according to any one of claims 1 to 7 are implemented.
10. A storage medium storing a computer program, characterized in that: The computer program comprises program instructions, which, when executed by a processor, cause the processor to perform the steps of the rectal polyp detection method based on DFIB and multi-layer aggregation network according to any one of claims 1 to 7.