Endoscope Image Super-Resolution Using Degradation-Aware Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning methods struggle to generate high-resolution images when high-quality source images are unavailable, leading to inadequate reflection of image features and difficulty in achieving super resolution beyond the source image's resolution.
Innovation Solution
An image processing apparatus and method that utilizes a learning model with specific layers and processing techniques to generate high-resolution images by reducing and then increasing the number of pixels through filter and noise addition, followed by a trained model to output super resolution images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If deep learning is used to perform super resolution on images with unavailable high-quality source images, then super resolution processing can be performed, but the generated images cannot appropriately reflect the feature of the deteriorated image and achieving super resolution beyond the source image's resolution is difficult
Solution Approach 1:
The patent applies preliminary action by preparing multiple versions of training images in advance with different deterioration levels (mild, moderate, severe). The learning model is trained on these pre-prepared datasets before actual super resolution processing, enabling it to handle various degradation scenarios effectively without requiring high-quality source images during operation.
Solution Approach 2:
The patent changes parameters by varying the deterioration degree of training images across multiple datasets. By training the learning model on images with different degradation levels (controlled by parameter alpha), the model learns to reflect features appropriately under various conditions, resolving the contradiction between resolution enhancement and feature accuracy.
2Manufacturing precision
If high resolution source images are prepared for training, then super resolution can be performed with high accuracy, but it may not be possible to prepare the source image having a high image quality depending on the specifications of the apparatus
Solution Approach 1:
The patent applies inversion by reversing the conventional training approach. Instead of using high-quality source images to train the model, it uses artificially generated training pairs where low-resolution images are degraded to create training datasets. This inverted approach eliminates the need for difficult-to-obtain high-quality source images while maintaining training effectiveness.
Solution Approach 2:
The patent uses copying by creating synthetic training image pairs through artificial generation and deterioration processes. Rather than relying on physical high-resolution source images, the system creates copies and controlled degradations of images to build training datasets, making the preparation process easier while maintaining training quality.
3Quantity of substance
If the number of pixels in the feature map is increased to maintain information amount, then processing capacity increases, but the information amount of the feature map becomes excessively large
Solution Approach 1:
The patent applies dimensionality change by transitioning from spatial dimension to channel dimension. Instead of increasing the number of pixels (spatial dimension) to maintain information amount, the system increases the number of channels in the feature map. This allows the model to preserve information while keeping the spatial resolution manageable, reducing overall feature map size and complexity.
Data Source
AI summary
A processor generates a trained model by updating a learning model that uses an input image for training obtained by performing deterioration processing on a source image as input to output a first output image for training and a second output image for training, which have the number of pixels larger than the input image for training. A second intermediate layer of the learning model outputs a feature map to be input to a first output layer that outputs the first output image for training based on the feature map from a folded layer and a first intermediate layer. A third intermediate layer outputs the feature map to be input to a second output layer that outputs a second output image for training based on the feature map from the folded layer.