An artificial intelligence-based colonoscopy qualified withdrawal time evaluation method and system
Patent Information
- Application Number
- CN202411153222.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2026-11-03
- Estimated Expiration
- 2044-08-21
AI Technical Summary
Current technology makes it difficult to accurately quantify the effective mucosal examination time during colonoscopy in real time, resulting in a high rate of missed adenomas and affecting the early detection and treatment of colorectal cancer.
An image quality assessment model and a cecum recognition model based on the ViT algorithm were constructed. Video streams were acquired and images were parsed using an endoscopic examination device. Qualified images and cecum frames were identified in real time, and the qualified time during colonoscopy withdrawal was calculated.
This enabled precise quality assessment of colonoscopy, improved the detection rate of adenomas, reduced the rate of missed adenomas, standardized colonoscopy procedures, and improved examination efficiency.
Smart Images
Figure CN119091206B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image classification, specifically an artificial intelligence-based method and system for evaluating the qualified withdrawal time of colonoscopy. Background Technology
[0002] Colorectal cancer (CRC) remains the third most common malignancy and the second leading cause of cancer-related death worldwide. Early detection and removal of precancerous adenomas via colonoscopy is a key measure to reduce the incidence and mortality of colorectal cancer. Although colonoscopy has been proven effective in preventing colorectal cancer, significant variations in the quality of colonoscopy among different endoscopists result in the missed detection of 26% of adenomas and 9% of advanced adenomas. Missed lesions may contribute to the development of post-colonoscopy CRC (PCCRC). Therefore, several quality control indicators for colonoscopy have been established, including withdrawal time, bowel preparation quality, and cecal cannulation rate, to improve the adenoma detection rate (ADR) and thus reduce the risk of PCCRC.
[0003] Withdrawal time is defined as the time spent withdrawing the colonic mucosa from the cecum to the anus without intervention. Current guidelines recommend a minimum withdrawal time of 6 minutes to ensure adequate mucosal examination and increase adverse drug response (ADR). However, recent studies have shown that longer withdrawal times (8–13 minutes) may further increase ADR and reduce the rate of missed adenomas (AMR), suggesting a need for more detailed examination of the colonic mucosa. Despite these findings, withdrawal time is an indirect measure of quality. Its practical effect lies in the thoroughness of mucosal visualization, particularly the time spent examining mucosal folds where adenomas are often hidden. Studies emphasize that techniques that increase mucosal exposure are more critical for increasing ADR than simply extending withdrawal time. Effective mucosal exposure can significantly reduce mucosal examination time without compromising lesion detection. These results indicate that effective examination time is far more important than total withdrawal time. However, accurately quantifying effective mucosal examination time in real time remains a significant challenge in routine practice.
[0004] In recent years, with the development of artificial intelligence (AI), integrating AI into medical imaging has shown great promise in improving diagnostic accuracy and operational quality. In colonoscopy, AI is mainly focused on the real-time detection and characterization of polyps or adenomas, demonstrating the potential for standardized procedures and reduced operator variability. However, studies using AI to calculate effective mucosal observation time and validate it as a quality control indicator for colonoscopy are limited. Summary of the Invention
[0005] This invention proposes an artificial intelligence-based method for assessing the qualified withdrawal time of colonoscopy.
[0006] The technical solution to achieve the objective of this invention is: a method for assessing the qualified withdrawal time of colonoscopy based on artificial intelligence, comprising:
[0007] An image quality assessment model and an image cecum recognition model based on the ViT algorithm were constructed, and the image quality assessment model and the cecum recognition model were pre-trained respectively.
[0008] The video of a colonoscopy is acquired using an endoscopic examination device, the video stream is parsed into images, and the images are cropped at a set rate.
[0009] The cecal frame was identified in the cropped image using a cecal recognition model to determine the start time of endoscope withdrawal.
[0010] Based on time-series changes, the image quality assessment model is used to classify the captured images as qualified images, calculate the total number of qualified images during colonoscopy withdrawal, determine the real-time qualified withdrawal time, and the withdrawal end time.
[0011] Preferably, the image training set used for pre-training the image quality assessment model and the appendix recognition model is specifically as follows:
[0012] Images and videos collected during colonoscopy were acquired, preprocessed, and labeled. Training sets for the image quality assessment model and the cecum recognition model were constructed. Images in the image quality assessment model training set were labeled as in vitro images, non-information images, foreign body images, intervention images, unqualified images, and qualified images, respectively. Images in the cecum recognition model training set were labeled as cecum images and images of other intestinal segments, respectively.
[0013] Preferably, the specific method for acquiring and preprocessing the images and videos collected during colonoscopy is as follows:
[0014] The entire video frame of the colonoscopy examination was extracted, and the effective regions of the extracted images and the acquired images were cropped. Information borders irrelevant to the recognition task were removed. The OpenCV algorithm was used to deduplicate images with high similarity and the images were adjusted to a uniform size.
[0015] Preferably, the image quality assessment model includes a Transformer encoder and an MLP classification head. The Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process global features of the image, while the MLP classification head classifies the input colonoscopy image into external images, non-information images, foreign body images, intervention images, unqualified images, and qualified images based on the feature vector output by the encoder and through a multilayer perceptron.
[0016] Preferably, the input to the image quality assessment model is a preprocessed non-overlapping image patch, each image patch incorporates positional information through position embedding, and all weight parameters of the image quality assessment model are pre-trained on ImageNet and then fine-tuned on the collected dataset.
[0017] Preferably, the cecal recognition model includes a Transformer encoder and an MLP classification head. The Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process global features of the image, while the MLP classification head classifies the input colonoscopy image into cecal images and other intestinal segment images based on the feature vector output by the encoder and through a multilayer perceptron.
[0018] Preferably, the input to the appendix recognition model is a preprocessed non-overlapping image patch, each image patch having location information introduced through location embedding, and all weight parameters of the image quality assessment model are pre-trained on ImageNet and then transferred to the collected dataset for fine-tuning.
[0019] Preferably, the specific method for determining the start time of endoscope withdrawal by using the cecal recognition model to perform cecal frame recognition on the cropped image is as follows:
[0020] The cecum recognition model is used to identify cecum frames in the cropped images. If M images are identified as cecum images out of N consecutive images, the withdrawal time is determined, and the withdrawal time is the time after that. The in vitro image recognition model is used to identify in vitro images in the cropped images. If M images are identified as in vitro images out of N consecutive images, the withdrawal time is determined, and the calculation of the qualified withdrawal time for colonoscopy is stopped. N and M are both natural numbers, and N>M.
[0021] Preferably, the real-time qualified lens removal time is as follows:
[0022]
[0023] This invention also proposes an artificial intelligence-based system for assessing the qualified withdrawal time of colonoscopy, comprising:
[0024] The image input module is used to acquire colonoscopy withdrawal images and perform image preprocessing.
[0025] Cecal recognition module, used to recognize cecal images;
[0026] The colonoscopy image quality assessment module is used to identify qualified images and in vitro images;
[0027] The comprehensive quality control module is used to determine the start and end times of endoscopy withdrawal, as well as the qualified endoscopy withdrawal time, based on the recognition results of cecal images, qualified images, and in vitro images.
[0028] Compared with the prior art, the significant advantages of this invention are: it can automatically classify colonoscopy images, identify qualified withdrawal images in real time, determine the start and end times of colonoscopy withdrawal, and calculate the qualified withdrawal time. This helps endoscopists to conduct more accurate quality assessments of the colonoscopy process, further standardize colonoscopy withdrawal operations, and thus improve the detection of adenomas by endoscopists.
[0029] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description
[0030] Figure 1 This is a flowchart of the present invention.
[0031] Figure 2 A schematic diagram of image processing by the artificial intelligence module in this invention.
[0032] Figure 3 This invention provides examples and heatmaps for image classification and cecal identification.
[0033] Figure 4 This is a display of the basic logic and related results of the present invention in practical applications. Detailed Implementation
[0034] An AI-based auxiliary quality control method for determining the qualified withdrawal time in colonoscopy mainly includes the following steps:
[0035] Step 1: Identify the cases to be examined by colonoscopy and obtain the images and videos collected during the colonoscopy.
[0036] The specific procedures for identifying colonoscopy cases are as follows: exclude cases with inflammatory bowel disease, a history of colorectal surgery, etc., and obtain the images and video data collected during the colonoscopy.
[0037] Step 2: Preprocess the collected image data. Specifically, extract all video frames from the colonoscopy video, crop the effective regions of the extracted images and the acquired images, remove information borders that are irrelevant to the recognition task, and use the OpenCV algorithm to deduplicate images with high similarity.
[0038] Specifically, the process of establishing the colonoscopy image dataset is as follows: image data from the colonoscopy process was collected and preprocessed to obtain usable image data (a total of 3888 cases, 120 videos, and 130993 images). The images were then divided into training set, test set, and internal and external validation set according to a predetermined ratio.
[0039] Step 3: Senior endoscopists, based on the final classification criteria, annotate all images using the examination image quality assessment model and the cecal identification model to obtain training and test samples. Specifically, the examination image quality assessment model annotations are categorized as in vitro images, non-information images, foreign body images, intervention images, unqualified images, and qualified images. This annotation is used to classify qualified image frames and in vitro frames during the examination. The cecal identification model annotations are categorized as cecal images and images of other intestinal segments. This annotation is used to identify the cecal location during the examination.
[0040] Step 4: Establish an image quality assessment model based on the ViT algorithm. This model includes a Transformer encoder and an MLP classification head. The Transformer encoder utilizes a self-attention mechanism and a feedforward neural network to capture and process global image features. The MLP classification head, based on the feature vector output by the encoder, uses a multilayer perceptron to classify the input colonoscopy image into external images, non-information images, foreign body images, intervention images, unacceptable images, and acceptable images. For the input samples to the image quality assessment model, the preprocessed images are segmented into 16x16 non-overlapping image blocks before being input into the model. Each image block incorporates positional information through position embedding. All weight parameters of the model are pre-trained on ImageNet and then fine-tuned on a collected dataset.
[0041] Step 5: Calculate the target loss function based on the training classification results and the true labels, and update the relevant parameters of the artificial intelligence model; the model training process uses the cross-entropy loss function as the optimization objective. To enhance the robustness of the model, Mixup is used during training, and an early stopping strategy is employed to prevent overfitting.
[0042] Step 6: Determine whether the preset number of training rounds has been reached, until the training ends.
[0043] Step 7: Use the test samples to verify the accuracy of the image classification of the quality assessment model.
[0044] Specifically, the steps to verify the effectiveness of the image quality assessment model are as follows: First, the classification effectiveness of the artificial intelligence module for colonoscopy image quality assessment is verified on internal, external, and prospective image datasets respectively; then, a human-machine comparative experiment is conducted based on the internal video test set to compare the differences in the proportion distribution of each category of the model and the endoscopist's colonoscopy video images.
[0045] Step 8: Establish a cecal recognition model based on the ViT algorithm. The model's specific structure includes a Transformer encoder and an MLP classification head. The Transformer encoder utilizes a self-attention mechanism and a feedforward neural network to capture and process global features of the image. The MLP classification head, based on the feature vector output by the encoder, uses a multilayer perceptron to classify the input colonoscopy image into cecal images and images of other intestinal segments. For the input image samples, before inputting them into the model, the preprocessed image is segmented into 16x16 non-overlapping image patches, and each patch incorporates positional information through position embedding. All weight parameters of the model are pre-trained on ImageNet and then fine-tuned on our collected dataset.
[0046] Step 9: Calculate the target loss function based on the training classification results and the true labels, and update the relevant parameters of the artificial intelligence model; the model training process uses the cross-entropy loss function as the optimization objective. To enhance the robustness of the model, Mixup is used during training, and an early stopping strategy is used to prevent overfitting.
[0047] Step 10: Determine whether the preset number of training rounds has been reached, until the training ends.
[0048] Step 11: Verify and test the accuracy of the trained cecal recognition model using the test samples;
[0049] Specifically, the steps to verify the effectiveness of the cecum recognition model are as follows: First, verify the effectiveness of the artificial intelligence module for cecum recognition on internal, external, and prospective image datasets respectively; then, conduct a human-machine comparative experiment based on the internal video test set to compare the average absolute difference between the model and the endoscopist's judgment of the colonoscopy withdrawal time.
[0050] Step 12: Acquire colonoscopy video using endoscopic equipment, parse the video stream into images, and capture images at a rate of 25 frames per second.
[0051] Step 13: Use the cecum recognition model to identify cecum frames in the cropped images. If 5 out of 10 consecutive images are identified as cecum images, the time point for withdrawal of the endoscope is determined, and the time point for withdrawal of the endoscope after that is the time for colonoscopy.
[0052] Step 14: After recognizing the start of colonoscopy withdrawal, based on temporal changes, the image quality assessment model is used to classify the captured images as qualified images, and the total number of qualified images during colonoscopy withdrawal is calculated; while Perform real-time statistics on qualified lens removal time.
[0053] Step 15: Check if the image quality assessment model identifies 5 out of 10 consecutive images as in vitro images, which determines the end point of colonoscopy withdrawal, and stop calculating the qualified withdrawal time for colonoscopy.
[0054] An artificial intelligence-based colonoscopy withdrawal time assessment system includes an image input module for acquiring and preprocessing colonoscopy withdrawal images, an image quality assessment module for colonoscopy examination based on an artificial intelligence model, a cecum identification module based on an artificial intelligence model, and a comprehensive quality control module for calculating the qualified withdrawal time.
[0055] The image input module is used to acquire and preprocess image data from the patient's colonoscopy (cropping the effective area of the image and removing information borders that are irrelevant to the recognition task).
[0056] An image quality assessment module based on the ViT (Vision Transformer) model is used to classify and determine the quality of input colonoscopy images, and to assist in determining the start time and end time of colonoscopy withdrawal.
[0057] The cecal recognition module based on the ViT model is used to identify cecal images during colonoscopy and determine the start time of endoscopy withdrawal.
[0058] By combining the image quality assessment module and the cecum recognition module, the algorithm determines the start and end times of colonoscopy withdrawal, calculates and displays the qualified withdrawal time of colonoscopy in real time.
[0059] This invention provides a method for calculating the qualified withdrawal time of colonoscopy. Combining the image quality assessment module and cecal recognition module proposed in this invention, it can effectively locate the start and end times of the colonoscopy and evaluate the frame quality during the scanning process. The qualified withdrawal time proposed in this invention, as a monitoring indicator of colonoscopy quality, not only provides real-time feedback to physicians, allowing them to continuously monitor the qualified withdrawal time, but also provides summary feedback to physicians, evaluating the quality of their colonoscopy examinations. This method can be effectively used for quality monitoring of colonoscopy examinations in hospitals.
Claims
1. A method for assessing the qualified withdrawal time of colonoscopy based on artificial intelligence, characterized in that, include: An image quality assessment model and a cecal recognition model based on the ViT algorithm were constructed and pre-trained respectively. The image quality assessment model includes a Transformer encoder and an MLP classification head. The Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process global features of the image. The MLP classification head classifies the input colonoscopy image into external images, non-information images, foreign body images, intervention images, unqualified images, and qualified images based on the feature vector output by the encoder and through a multilayer perceptron. The video of a colonoscopy is acquired using an endoscopic examination device, the video stream is parsed into images, and the images are cropped at a set rate. The appendix recognition model is used to identify appendix frames in the cropped images to determine the start time of scope withdrawal. The specific method is as follows: The cecum recognition model is used to identify cecum frames in the cropped images. If M images are identified as cecum images out of N consecutive images, the withdrawal start time is determined, and the withdrawal time is the time after that. The in vitro image recognition model is used to identify in vitro images in the cropped images. If M images are identified as in vitro images out of N consecutive images, the withdrawal end time is determined, and the calculation of the qualified withdrawal time for colonoscopy is stopped. N and M are both natural numbers, and N>M. Based on time-series changes, the image quality assessment model is used to classify the captured images as qualified images, calculate the total number of qualified images during colonoscopy withdrawal, determine the real-time qualified withdrawal time, and the withdrawal end time. The specific real-time qualified withdrawal time is as follows: Qualified withdrawal time = ×Time to exit the camera.
2. The method for assessing the qualified withdrawal time of colonoscopy based on artificial intelligence according to claim 1, characterized in that, The image training set used for pre-training the image quality assessment model and the cecal recognition model is as follows: Images and videos collected during colonoscopy were acquired, preprocessed, and labeled. Training sets for the image quality assessment model and the cecum recognition model were constructed. Images in the image quality assessment model training set were labeled as in vitro images, non-information images, foreign body images, intervention images, unqualified images, and qualified images, respectively. Images in the cecum recognition model training set were labeled as cecum images and images of other intestinal segments, respectively.
3. The method for assessing the qualified withdrawal time of colonoscopy based on artificial intelligence according to claim 2, characterized in that, The specific methods for acquiring and preprocessing images and videos collected during colonoscopy are as follows: The entire video frame of the colonoscopy examination was extracted, and the effective regions of the extracted images and the acquired images were cropped. Information borders irrelevant to the recognition task were removed. The OpenCV algorithm was used to deduplicate images with high similarity and the images were adjusted to a uniform size.
4. The method for assessing the qualified withdrawal time of colonoscopy based on artificial intelligence according to claim 1, characterized in that, The input to the image quality assessment model is a preprocessed non-overlapping image patch. Each image patch incorporates positional information through position embedding. All weight parameters of the image quality assessment model are pre-trained on ImageNet and then fine-tuned on the collected dataset.
5. The method for assessing the qualified withdrawal time of colonoscopy based on artificial intelligence according to claim 1, characterized in that, The cecal recognition model includes a Transformer encoder and an MLP classification head. The Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process global features of the image, while the MLP classification head classifies the input colonoscopy image into cecal images and other intestinal segment images based on the feature vector output by the encoder and through a multilayer perceptron.
6. The method for assessing the qualified withdrawal time of colonoscopy based on artificial intelligence according to claim 1, characterized in that, The input to the appendix recognition model is a preprocessed non-overlapping image patch. Each image patch incorporates positional information through position embedding. All weight parameters of the image quality assessment model are pre-trained on ImageNet and then fine-tuned on the collected dataset.
7. An artificial intelligence-based colonoscopy withdrawal time assessment system based on the method of any one of claims 1 to 6, characterized in that, include: The image input module is used to acquire colonoscopy withdrawal images and perform image preprocessing. Cecal recognition module, used to recognize cecal images; The colonoscopy image quality assessment module is used to identify qualified images and in vitro images; The comprehensive quality control module is used to determine the start and end times of endoscope withdrawal, as well as the qualified endoscope withdrawal time, based on the recognition results of cecal images, qualified images, and in vitro images.