Capsule Endoscope Image Recognition Using 3D CNN Video Sequences
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing capsule endoscope image recognition methods struggle with low accuracy due to the inability to utilize spatial and temporal information from multiple images simultaneously, leading to confusion between gastrointestinal folds and lesions like polyps and bulges, especially when captured at specific angles.
Innovation Solution
A capsule endoscope image recognition method using a 3D convolutional neural network that processes multiple frames of images, segmenting them into groups using a sliding window technique, and combines RGB and optical flow data to improve recognition accuracy through a collaborative spatio-temporal feature structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single image recognition is used, then the recognition process is simple, but the recognition accuracy is low due to inability to utilize spatial and temporal information from multiple images
Solution Approach 1:
The patent transitions from 2D single-image recognition to 3D video sequence recognition by adding the temporal dimension. The 3D CNN model processes video clips consisting of multiple frames, enabling simultaneous extraction of spatial features (from individual frames) and temporal features (from frame sequences), thereby improving lesion recognition accuracy through multi-dimensional feature analysis
Solution Approach 2:
The patent divides the video sequence into multiple frames and processes them through segmented computational pathways. The sliding window technique segments video sequences into overlapping clips, and the dual-branch CNN architecture segments feature extraction into spatial and temporal components, allowing comprehensive analysis while managing computational complexity
2Measurement precision
If multiple images are processed to obtain spatial and temporal information, then the recognition accuracy improves, but the computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary processing by pre-extracting spatial features from individual frames using 2D convolutional layers before temporal feature extraction. This preliminary spatial feature extraction reduces the dimensionality of subsequent temporal processing, optimizing the balance between comprehensive analysis and processing efficiency
Solution Approach 2:
The patent employs dynamic feature extraction where the importance of spatial versus temporal features is adaptively determined. The model dynamically adjusts feature weighting based on the specific lesion type and video sequence characteristics, optimizing processing efficiency for different diagnostic scenarios
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention discloses a capsule endoscope image recognition method based on deep learning, and a device and a medium. After a plurality of frames of continuously captured images are processed to form an image sequence of a specific format, multi-channel recognition is performed on the plurality of frames of images by means of a 3D convolutional neural network model, and a recognition result is then output in combination with a recognition probability of each channel, such that the image recognition precision is improved.