Capsule Endoscope Image Recognition Using 3D CNN and Optical Flow

The patent transitions from 2D single-image analysis to 3D volumetric analysis by stacking multiple 2D images along the time dimension. This creates a 3D data structure (height, width, time) that enables the neural network to capture both spatial features within each image and temporal features across the image sequence, thereby improving lesion recognition accuracy while maintaining manageable computational complexity through efficient 3D convolution operations.

The patent performs preliminary preprocessing on the image sequence before feeding it to the neural network. This includes converting the sequence of 2D images into a 3D tensor format, applying data augmentation techniques, and preparing the input data in advance. These preliminary actions reduce the computational burden during the recognition phase and improve the overall efficiency of the system.

The patent replaces complex manual image processing and analysis mechanisms with an automated 3D convolutional neural network. Instead of using traditional image processing algorithms that require multiple separate steps for feature extraction and analysis, the system uses end-to-end deep learning that automatically learns spatial and temporal features from the 3D input, significantly reducing processing complexity while maintaining high accuracy.

The 3D convolutional neural network serves multiple functions simultaneously: it performs spatial feature extraction within each 2D image, temporal feature extraction across the image sequence, and lesion classification. This multi-functional approach consolidates what would otherwise require separate processing modules into a single unified system, reducing overall complexity while improving accuracy.