Enteroscope qualified retreating time evaluation method and system based on artificial intelligence

Through the colonoscopy image quality evaluation model and cecum recognition model based on ViT algorithm, the problem of difficult to quantify the effective examination time in colonoscopy is solved, and the effect of real-time recognition of the resink time and improving the detection rate of adenoma is achieved.

CN119091206AActive Publication Date: 2024-12-06NANJING DRUM TOWER HOSPITAL +1

Patent Information

Application Number
CN202411153222.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-12-06
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

The prior art is difficult to quantify the effective mucosal examination time in colonoscopy in real time, resulting in challenges in colonoscopy quality control, affecting the adenoma detection rate and missed diagnosis rate.

Method used

Using the ViT algorithm-based image quality evaluation model and cecum recognition model, the video stream is examined through colonoscopy, the start and end time of recoding is identified, the real-time qualified recoding time is calculated, and image classification is used using Transformer encoder and MLP classification head. and feature processing.

Benefits of technology

Automatic colonoscopy image classification and real-time identification of qualified responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive responsive res

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091206A_ABST
    Figure CN119091206A_ABST
Patent Text Reader

Abstract

The invention provides an artificial intelligence-based colonoscope qualified refection time evaluation method. The method comprises the following steps: constructing a ViT algorithm-based examination image quality evaluation model and a ViT algorithm-based examination image cecum identification model, and pre-training the examination image quality evaluation model and the cecum identification model respectively; the method comprises the following steps: acquiring an enteroscopy video through endoscopy equipment, analyzing a video stream into a picture, and intercepting the picture according to a set rate; performing cecum frame identification on the intercepted picture by using a cecum identification model, and determining the mirror retreating starting time; and on the basis of the time sequence change, checking the image quality evaluation model to carry out qualified image classification on the intercepted image, calculating the total number of qualified images in the enteroscope retreating process, and determining the real-time qualified retreating time and the retreating ending time. The method can automatically classify enteroscopy images, identify qualified retreating images in real time, and judge the starting time and the ending time of enteroscopy retreating, so as to calculate qualified retreating time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of medical image classification, and specifically relates to an artificial intelligence-based colonoscopy qualified endoscope withdrawal time assessment method and system. Background Art

[0002] Colorectal cancer (CRC) remains the third most common malignancy and the second leading cause of cancer-related death worldwide. Early detection and removal of precancerous adenomas through colonoscopy are key measures to reduce the incidence and mortality of colorectal cancer. Although colonoscopy has been proven to be effective in preventing colorectal cancer, 26% of adenomas and 9% of advanced adenomas are missed during colonoscopy due to large differences in the quality of colonoscopy among different endoscopists. Missed lesions may promote the development of post-colonoscopy CRC (PCCRC). Therefore, several quality control indicators of colonoscopy have been established, including withdrawal time, bowel preparation quality, and cecal intubation rate, to improve the adenoma detection rate (ADR) and thereby reduce the risk of PCCRC.

[0003] Withdrawal time is defined as the time spent withdrawing the scope from the cecum to the anus without intervention. Current guidelines recommend a minimum withdrawal time of 6 minutes to ensure adequate mucosal examination and increase the ADR. However, recent studies have shown that longer withdrawal times (8–13 minutes) may further increase the ADR and reduce the adenoma miss rate (AMR), suggesting the need for a more meticulous examination of the colonic mucosa. Despite these findings, withdrawal time is an indirect measure of quality. Its actual effect lies in the thoroughness of mucosal visualization, especially the time spent examining mucosal folds where adenomas are often hidden. Studies have highlighted that techniques that increase mucosal exposure are more critical to increasing ADR than simply prolonging withdrawal time. Effective mucosal exposure can significantly reduce mucosal examination time without compromising lesion detection. These results suggest that effective examination time is much more important than total withdrawal time. However, in routine practice, it remains a great challenge to accurately quantify effective mucosal examination time in real time.

[0004] In recent years, with the development of artificial intelligence (AI), the integration of AI into medical imaging tests has shown great promise in improving diagnostic accuracy and operation quality. In colonoscopy, AI mainly focuses on the real-time detection and characterization of polyps or adenomas, showing the potential for operation standardization and reducing operator variability. However, there are few studies that use AI to calculate effective mucosal observation time and validate it as a quality control indicator for colonoscopy. Summary of the invention

[0005] The present invention proposes an artificial intelligence-based method for evaluating the qualified colonoscopy withdrawal time.

[0006] The technical solution to achieve the purpose of the present invention is: a method for evaluating the qualified withdrawal time of colonoscopy based on artificial intelligence, comprising:

[0007] Construct an inspection image quality assessment model based on the ViT algorithm and an inspection image cecum recognition model based on the ViT algorithm, and pre-train the inspection image quality assessment model and the cecum recognition model respectively;

[0008] Obtain colonoscopy video through endoscopy equipment, parse the video stream into images, and capture the images at a set rate;

[0009] Use the cecum recognition model to perform cecum frame recognition on the captured image and determine the start time of the mirror withdrawal;

[0010] Based on the time series changes, the inspection image quality assessment model classifies the captured images into qualified images, calculates the total number of qualified images during the colonoscopy withdrawal process, determines the real-time qualified withdrawal time, and the withdrawal completion time.

[0011] Preferably, the image training set for pre-training the inspection image quality assessment model and the cecum recognition model is specifically:

[0012] The images and video data collected during colonoscopy were obtained, preprocessed and annotated, and the inspection image quality assessment model training set and the cecum recognition model training set were constructed respectively. The images in the inspection image quality assessment model training set were annotated as in vitro images, non-information images, foreign body images, intervention images, unqualified images, and qualified images; the images in the cecum recognition model training set were annotated as cecum images and other intestinal segment images.

[0013] Preferably, the specific method for obtaining and preprocessing the images and video data collected during colonoscopy is:

[0014] The full video frame of the colonoscopy video is extracted, and the effective area of ​​the image extracted from the full video frame and the acquired image is cropped to remove the information borders irrelevant to the recognition task. The OpenCV algorithm is used to deduplicate images with high similarity and adjust the images to a uniform size.

[0015] Preferably, the inspection image quality assessment model includes a Transformer encoder and an MLP classification head, wherein the Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process the global features of the image, and the MLP classification head classifies the input colonoscopy image into in vitro images, non-information images, foreign body images, intervention images, unqualified images, and qualified images through a multi-layer perceptron based on the feature vector output by the encoder.

[0016] Preferably, the input of the inspection image quality assessment model is preprocessed non-overlapping image blocks, each image block introduces position information through position embedding, and all weight parameters of the inspection image quality assessment model are pre-trained based on ImageNet and then migrated to the collected data set for fine-tuning.

[0017] Preferably, the cecum recognition model includes a Transformer encoder and an MLP classification head, wherein the Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process the global features of the image, and the MLP classification head classifies the input colonoscopy image into a cecum image or other intestinal segment images through a multi-layer perceptron based on the feature vector output by the encoder.

[0018] Preferably, the input of the cecum recognition model is preprocessed non-overlapping image blocks, each image block introduces position information through position embedding, and all weight parameters of the inspection image quality assessment model are pre-trained based on ImageNet and then migrated to the collected dataset for fine-tuning.

[0019] Preferably, the specific method of using the cecum recognition model to perform cecum frame recognition on the captured image and determining the start time of the mirror withdrawal is:

[0020] The cecum recognition model is used to perform cecum frame recognition on the captured images. When M images are identified as cecum images among N consecutive images, it is determined as the start time of scope withdrawal, and the subsequent time is the withdrawal time of the colonoscopy. The inspection image quality assessment model is used to perform in vitro image recognition on the captured images. When M images are identified as in vitro images among N consecutive images, it is determined as the end time of scope withdrawal, and the calculation of the qualified withdrawal time of the colonoscopy is stopped. N and M are both natural numbers, and N>M.

[0021] Preferably, the real-time qualified mirror removal time is specifically:

[0022]

[0023] The present invention also proposes an artificial intelligence-based colonoscopy qualified endoscope withdrawal time assessment system, comprising:

[0024] An image input module is used to obtain the colonoscopy withdrawal image and pre-process the image;

[0025] A cecum recognition module, used to recognize cecum images;

[0026] Colonoscopy image quality assessment module to identify qualified images as well as in vitro images;

[0027] The comprehensive quality control module is used to determine the start and end time of the scope withdrawal, as well as the qualified scope withdrawal time based on the cecal image, qualified image and in vitro image recognition results.

[0028] Compared with the prior art, the present invention has the following significant advantages: the present invention can automatically classify colonoscopy images, identify qualified scope withdrawal images in real time, determine the start and end time of colonoscopy withdrawal, and thus calculate the qualified scope withdrawal time, which helps endoscopists to conduct more accurate quality assessment of the colonoscopy examination process, further standardize the colonoscopy withdrawal operation, and thus improve the detection of adenoma by endoscopists.

[0029] The present invention is described in further detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the present invention.

[0031] Figure 2 Schematic diagram of image processing by the artificial intelligence module in the present invention.

[0032] Figure 3 This is an example and heat map display of inspection image classification and cecum recognition according to the present invention.

[0033] Figure 4 It is a display of the basic logic and related results of the present invention in practical applications. DETAILED DESCRIPTION

[0034] An artificial intelligence-based auxiliary quality control method for qualified colonoscopy withdrawal time mainly includes the following steps:

[0035] Step 1: determine the cases that need colonoscopy and obtain the images and video data collected during the colonoscopy;

[0036] The specific operations to determine the cases for colonoscopy are as follows: exclude cases with inflammatory bowel disease and a history of colorectal surgery from the cases for colonoscopy, and obtain the images and video data collected during the colonoscopy.

[0037] Step 2, preprocessing the collected image data, specifically: extracting the full video frame of the colonoscopy video, and cropping the effective area of ​​the image extracted from the full video frame and the collected image, removing the information border irrelevant to the recognition task, and using the OpenCV algorithm to deduplicate images with high similarity.

[0038] Specifically, the process of establishing the colonoscopy image dataset is as follows: image data from the colonoscopy process is collected, and after preprocessing, a total of usable image data (3,888 cases, 120 videos, and 130,993 images) is obtained. The images are divided into training sets, test sets, and internal and external validation sets according to the established ratio.

[0039] Step 3: Senior endoscopists examine all images according to the final classification criteria, using the image quality assessment model and the cecum recognition model annotations to obtain training samples and test samples. Specifically, the image quality assessment model annotations are in vitro images, non-information images, foreign body images, intervention images, unqualified images, and qualified images. This annotation is used to classify qualified image frames and in vitro frames during the inspection process; the cecum recognition model annotations are cecum images and other intestinal segment images. This annotation is used to identify the cecum location during the inspection process.

[0040] Step 4, establish an inspection image quality assessment model based on the ViT algorithm; the inspection image quality assessment model includes a Transformer encoder and an MLP classification head. The Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process the global features of the image, and the MLP classification head classifies the input colonoscopy image into in vitro images, non-information images, foreign body images, intervention images, unqualified images, and qualified images based on the feature vector output by the encoder through a multi-layer perceptron. For the input samples of the inspection image quality assessment model, before entering the model, the preprocessed images are divided into 16x16 non-overlapping image blocks, and each image block introduces position information through position embedding. All weight parameters of the model are pre-trained on ImageNet and then migrated to the collected dataset for fine-tuning.

[0041] Step 5, calculate the target loss function based on the training classification results and the true labels, and update the relevant parameters of the artificial intelligence model; the model training process uses the cross entropy loss function as the optimization target. In order to enhance the robustness of the model, Mixup is used in the training process, and an early stopping strategy is used to prevent overfitting.

[0042] Step 6: Determine whether the preset number of training rounds has been reached until the training is completed.

[0043] Step 7: Use the test sample to verify the accuracy of image classification of the inspection quality assessment model.

[0044] Specifically, the specific steps to verify the effectiveness of the inspection image quality assessment model are as follows: first, the classification effectiveness of the artificial intelligence module for colonoscopy image quality assessment is verified on internal, external, and prospective image data sets; a human-machine control experiment is conducted based on the internal video test set to compare the differences in the distribution of each classification proportion of the full colonoscopy video inspection images between the model and the endoscopist.

[0045] Step 8, establish a cecum recognition model based on the ViT algorithm; the specific structure of the model includes a Transformer encoder and an MLP classification head. The Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process the global features of the image, and the MLP classification head classifies the input colonoscopy image into cecum images and other intestinal segment images based on the feature vector output by the encoder through a multi-layer perceptron. For the input samples of the image, before entering the model, the preprocessed image is divided into 16x16 non-overlapping image blocks, and each image block introduces position information through position embedding. All weight parameters of the model are pre-trained based on ImageNet, and then migrated to the dataset we collected for fine-tuning.

[0046] Step 9, calculate the target loss function based on the training classification results and the true labels, and update the relevant parameters of the artificial intelligence model; the model training process uses the cross entropy loss function as the optimization target. In order to enhance the robustness of the model, Mixup is used in the training process, and an early stopping strategy is used to prevent overfitting.

[0047] Step 10, determine whether the preset number of training rounds has been reached, until the training is completed.

[0048] Step 11, using the test sample to verify and test the accuracy of the trained cecum recognition model;

[0049] Specifically, the steps to verify the effectiveness of the cecum recognition model are as follows: first, verify the effectiveness of the artificial intelligence module for cecum recognition on internal, external, and prospective image datasets; conduct a human-machine control experiment based on the internal video test set to compare the average absolute difference between the model and the endoscopist's judgment of the colonoscopy withdrawal time.

[0050] Step 12, obtain the colonoscopy video through the endoscopy device, parse the video stream into pictures, and capture the pictures at a rate of 25 frames per second.

[0051] Step 13, using the cecum recognition model to perform cecum frame recognition on the captured image, and if 5 out of 10 consecutive images are recognized as cecum images, it is determined as the start time point of withdrawing the scope, and then it is the time for withdrawing the scope for colonoscopy.

[0052] Step 14, after recognizing the start of the colonoscope withdrawal, based on the time sequence change, the image quality assessment model is checked to classify the captured images as qualified images, and the total number of qualified images in the colonoscope withdrawal process is calculated; and Conduct real-time statistics on qualified mirror removal time.

[0053] Step 15, if the image quality assessment model identifies 5 out of 10 consecutive images as in vitro images, it is determined as the end time point for the end of the colonoscopy withdrawal, and the calculation of the qualified colonoscopy withdrawal time is stopped.

[0054] A colonoscopy qualified withdrawal time assessment system based on artificial intelligence includes an image input module for acquiring colonoscopy withdrawal images and performing preprocessing, a colonoscopy image quality assessment module constructed based on an artificial intelligence model, a cecum recognition module constructed based on an artificial intelligence model, and a comprehensive quality control module for calculating the qualified withdrawal time.

[0055] Image input module, used to obtain patient colonoscopy image data and preprocess (cut off the effective area of ​​the image and remove the information border irrelevant to the recognition task);

[0056] The inspection image quality assessment module based on the ViT (Vision Transformer) model is used to classify and judge the input colonoscopy images that are qualified, and to assist in determining the start time and end time of colonoscopy;

[0057] The cecum recognition module based on the ViT model is used to identify the cecum image during colonoscopy and determine the start time of the scope withdrawal;

[0058] Combined with the examination image quality assessment module and the cecum recognition module, the algorithm is used to determine the start and end time of the scope withdrawal, and the scope withdrawal time for qualified colonoscopy is calculated and displayed in real time.

[0059] The present invention provides a method for calculating the qualified withdrawal time of a colonoscope. Combined with the withdrawal inspection image quality assessment module and the cecum recognition module proposed in the present invention, it can effectively locate the time nodes of the start and end of the colonoscopy and perform a corresponding assessment of the frame quality during the scanning process. The qualified withdrawal of the colonoscope proposed in the present invention is used as a monitoring indicator of the quality of colonoscopy. It can not only provide real-time feedback to the physician, always pay attention to the qualified withdrawal time, but also provide a summary feedback to the physician to evaluate the quality of the physician's colonoscopy, which can be well used for the quality monitoring of the hospital's colonoscopy.

Claims

1. A method for evaluating the qualified colonoscopy withdrawal time based on artificial intelligence, characterized in that: include: Construct an inspection image quality assessment model based on the ViT algorithm and an inspection image cecum recognition model based on the ViT algorithm, and pre-train the inspection image quality assessment model and the cecum recognition model respectively; Obtain colonoscopy video through endoscopy equipment, parse the video stream into images, and capture the images at a set rate; Use the cecum recognition model to perform cecum frame recognition on the captured image and determine the start time of the mirror withdrawal; Based on the time series changes, the inspection image quality assessment model classifies the captured images into qualified images, calculates the total number of qualified images during the colonoscopy withdrawal process, determines the real-time qualified withdrawal time, and the withdrawal completion time.

2. The method for evaluating the qualified colonoscopy withdrawal time based on artificial intelligence according to claim 1 is characterized in that: The image training set for pre-training the inspection image quality assessment model and the cecum recognition model is as follows: The images and video data collected during colonoscopy were obtained, preprocessed and annotated, and the inspection image quality assessment model training set and the cecum recognition model training set were constructed respectively. The images in the inspection image quality assessment model training set were annotated as in vitro images, non-information images, foreign body images, intervention images, unqualified images, and qualified images; the images in the cecum recognition model training set were annotated as cecum images and other intestinal segment images.

3. The method for evaluating the qualified colonoscopy withdrawal time based on artificial intelligence according to claim 2 is characterized in that: The specific method for obtaining and preprocessing the images and video data collected during colonoscopy is as follows: The full video frame of the colonoscopy video is extracted, and the effective area of ​​the image extracted from the full video frame and the acquired image is cropped to remove the information borders irrelevant to the recognition task. The OpenCV algorithm is used to deduplicate images with high similarity and adjust the images to a uniform size.

4. The method for evaluating the qualified colonoscopy withdrawal time based on artificial intelligence according to claim 1 is characterized in that: The inspection image quality assessment model includes a Transformer encoder and an MLP classification head. The Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process the global features of the image. The MLP classification head classifies the input colonoscopy image into in vitro images, non-information images, foreign body images, intervention images, unqualified images, and qualified images through a multi-layer perceptron based on the feature vector output by the encoder.

5. The method for evaluating the qualified colonoscopy withdrawal time based on artificial intelligence according to claim 4 is characterized in that: The input of the inspection image quality assessment model is preprocessed non-overlapping image blocks, and each image block introduces position information through position embedding. All weight parameters of the inspection image quality assessment model are pre-trained based on ImageNet and then migrated to the collected dataset for fine-tuning.

6. The method for evaluating the qualified colonoscopy withdrawal time based on artificial intelligence according to claim 1 is characterized in that: The cecum recognition model includes a Transformer encoder and an MLP classification head, wherein the Transformer encoder uses a self-attention mechanism and a feedforward neural network to capture and process the global features of the image, and the MLP classification head classifies the input colonoscopy image into a cecum image or other intestinal segment images through a multi-layer perceptron based on the feature vector output by the encoder.

7. The method for evaluating the qualified colonoscopy withdrawal time based on artificial intelligence according to claim 1 is characterized in that: The input of the cecum recognition model is preprocessed non-overlapping image blocks, and each image block introduces position information through position embedding. All weight parameters of the inspection image quality assessment model are pre-trained based on ImageNet and then migrated to the collected dataset for fine-tuning.

8. The method for evaluating the qualified colonoscopy withdrawal time based on artificial intelligence according to claim 1 is characterized in that: The specific method of using the cecum recognition model to perform cecum frame recognition on the captured image and determine the start time of the mirror withdrawal is as follows: The cecum recognition model is used to perform cecum frame recognition on the captured images. When M images are identified as cecum images among N consecutive images, it is determined as the start time of scope withdrawal, and the subsequent time is the withdrawal time of the colonoscopy. The inspection image quality assessment model is used to perform in vitro image recognition on the captured images. When M images are identified as in vitro images among N consecutive images, it is determined as the end time of scope withdrawal, and the calculation of the qualified withdrawal time of the colonoscopy is stopped. N and M are both natural numbers, and N>M.

9. The method for evaluating the qualified colonoscopy withdrawal time based on artificial intelligence according to claim 1 is characterized in that: The specific real-time qualified de-mirror time is:

10. An artificial intelligence-based colonoscopy qualified withdrawal time assessment system based on any of the methods of claims 1 to 9, characterized in that: include: An image input module is used to obtain the colonoscopy withdrawal image and pre-process the image; A cecum recognition module, used to recognize cecum images; Colonoscopy image quality assessment module to identify qualified images as well as in vitro images; The comprehensive quality control module is used to determine the start and end time of the scope withdrawal, as well as the qualified scope withdrawal time based on the cecal image, qualified image and in vitro image recognition results.

Citation Information

Patent Citations

  • Detection method for stationary state of enteroscope lens based on image recognition matching

    CN111839444A

  • Effective mirror retreating time evaluation method and device for enteroscopy and storage medium

    CN113962998A

  • Endoscope image detection auxiliary system and method, medium and electronic equipment

    CN114332019A

  • Endoscope image detection auxiliary system and method, medium and electronic device

    WO2023124876A1

Cited By

  • Endoscope operation evaluation system

    CN121032747A

  • Enteroscope observation integrity evaluation method and system based on artificial intelligence

    CN121686167A

  • An artificial intelligence-based colonoscopy observation integrity evaluation method and system

    CN121686167B