Endoscopic Image Recognition With Normal-Light Pretraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning methods for image recognition in endoscopy struggle with low accuracy due to insufficient training data, particularly for special light and pigment-sprayed images, which are not readily abundant, leading to decreased recognition performance when applied to images captured under different observation conditions.
Innovation Solution
A pre-training and fine-tuning method using a first image group of normal light images, followed by fine-tuning with a second image group of special light or pigment-sprayed images, incorporating ground truth data to enhance recognition accuracy for regions of interest.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning training is performed using only a limited number of endoscope images (full training), then the training process is simple and quick, but recognition accuracy cannot be obtained due to insufficient training data
Solution Approach 1:
The patent applies pre-training using ImageNet (a large-scale general object image dataset) before fine-tuning with endoscope images. This preliminary action of training on abundant general images initializes the neural network with useful feature extraction capabilities, enabling subsequent fine-tuning with limited endoscope images to achieve high recognition accuracy that would be impossible with endoscope images alone.
Solution Approach 2:
The patent uses ImageNet images as an intermediary dataset to bridge the gap between having no training data and having sufficient endoscope training data. The general object images in ImageNet serve as a mediator that transfers useful visual features to the endoscope image recognition task, allowing the model to achieve high accuracy with limited domain-specific training data.
2Measurement precision
If pre-training with ImageNet is performed followed by fine-tuning with endoscope images, then recognition accuracy is improved, but the training process becomes more complex and time-consuming
Solution Approach 1:
The pre-training phase using ImageNet is performed as a preliminary action before fine-tuning. Although this adds initial training time, it significantly reduces the amount of domain-specific fine-tuning needed, as the model already possesses general image recognition capabilities. The two-stage approach optimizes the trade-off between training time and final accuracy.
Solution Approach 2:
The patent performs partial fine-tuning rather than complete re-training on endoscope images. By leveraging the pre-trained model's existing capabilities and only adjusting parameters with limited endoscope data, the training process achieves high accuracy without requiring excessive domain-specific training time that would be needed for full training from scratch.
3Measurement precision
If pre-training with ImageNet is performed followed by fine-tuning with endoscope images, then recognition accuracy is improved, but the training process complexity increases
Solution Approach 1:
The pre-training step is performed as a preliminary action that establishes a solid foundation of general image recognition skills. This preliminary preparation simplifies the subsequent fine-tuning process, as the model only needs to adapt to domain-specific features rather than learn everything from scratch, making the overall complex process more manageable and systematic.
Solution Approach 2:
The training process is segmented into two distinct stages: pre-training on ImageNet and fine-tuning on endoscope images. This segmentation allows each stage to be optimized independently, with pre-training focusing on general feature extraction and fine-tuning focusing on domain-specific adaptation, making the overall complex training process more structured and controllable.
4Measurement precision
If fine-tuning is performed using only endoscope images, then the training data is easy to obtain, but recognition accuracy with high precision cannot be achieved due to insufficient data quantity
Solution Approach 1:
ImageNet serves as an intermediary data source that bridges the gap between easy-to-obtain but insufficient endoscope images and the need for large-scale training data. By using this external intermediary dataset for pre-training, the system achieves high accuracy without requiring large quantities of difficult-to-obtain annotated endoscope images.
Solution Approach 2:
Pre-training on ImageNet is performed as a preliminary action that equips the model with general image understanding before it encounters the limited endoscope training data. This preliminary data preparation enables the model to achieve high recognition accuracy despite the ease-of-acquisition constraint on endoscope images, as the model doesn't need to learn everything from the limited domain-specific data alone.
Data Source
AI summary
An image processing system includes a processor configured to acquire, as a processing target image, an in-vivo image, operate in accordance with a trained model, and output a recognition result representing a result of recognition of a region of interest in the processing target image. The trained model is trained by having undergone pre-training using a first image group including images captured in a first observation method, and having undergone, after the pre-training, fine-tuning that uses a second image group including images captured in a second observation method, as well as that uses ground truth regarding the region of interest included in the second image group. The first observation method is an observation method using normal light as illumination light, and the second observation method is an observation method using special light as the illumination light or an observation method in which a pigment has been dispersed onto the subject.