Digestive endoscopy video scene classification method based on convolutional neural network

Abstract

The invention relates to a digestive endoscopy video scene classification method based on a convolutional neural network. The method comprises the following steps: (1) acquiring a real-time scene image through digestive endoscopy equipment; (2) carrying out preliminary scene classification on the obtained scene image by using a CNN scene classifier; (3) bringing the preliminary classification result of the CNN scene classifier into a K sliding window time sequence signal queue, wherein K is the length of the time sequence signal queue; (4) counting the proportions of various states in the K sliding window queue; (5) determining a scene state according to the proportion of the current scene to the K sliding window queue state; the CNN scene classifier can efficiently perform primary scene classification on the image acquired by the endoscope equipment in real time, and in order to ensure the stability of the image scene classification acquired by the endoscope equipment on a time sequence signal, the scene state of the primary scene classification result is determined by using a sliding window statistical scene state conversion algorithm; and the reliability of scene classificationand scene state conversion is enhanced.

Description

technical field [0001] The invention belongs to the technical field of image processing of digestive endoscopy video, and in particular relates to a method for classifying digestive endoscopy video scenes based on a convolutional neural network. Background technique [0002] In recent years, with the emergence of deep learning algorithms, the field of artificial intelligence (AI) has shown revolutionary development and progress in research. With the support of massive data, AI systems based on deep learning algorithms have begun to be used in more and more application fields. It has the recognition and judgment ability similar to that of human experts, and even exceeds that of humans in some special fields; especially in the medical field, deep learning algorithms also show great potential, for example, in the diagnosis of skin lesions and diabetic retinopathy images, based on The deep learning AI system already has the same or even better recognition ability than medical ex...

AI Technical Summary

Problems solved by technology

Gastrointestinal surgery videos generally include the in vitro preparation process, entry process, internal process, and final return to the external process. Automatic scene recognition can help the CAD system automatically start the corresponding functional modules according to the current scene. For example, it needs to be started in upper gastrointestinal endoscopic surgery. For early screening of gastric cancer, the identification of intestinal polyps needs to be activated during colonoscopic surgery, and no lesion identification function needs to be activated when the lens is outside the body; on the contrary, if there is no automatic scene recognition, the CAD system can only manually input the current scene information by the user, so that Limits the functionality and use of the system

[0005] Usually image classification is to fully describe the entire single image through manual feature annotation or feature learning methods, and then use classifiers to learn features and identify object categories. With the development of deep learning, CNN-based image classification algorithms have improved in terms of accuracy. Rapid development, CNN-based image classifiers usually have excellent classification capabilities on a single image, but in the process of video judgment based on image streams, even the most accurate classifiers will inevitably encounter misjudgments; Digestive endoscopy video is image data collected by multi-frame digestive endoscopy lenses in a time-series direction. CNN-based scene image classifiers can more accurately identify the scenes represented by each single frame, but due to the image data generated by video collection The number is large, and it is inevitable that single frame misclassification will occur during the sequential image sampling process, resulting in the wrong judgment of the scene represented by the single frame image

Images