Endoscope Image Super-Resolution With Optical-System Simulation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing super resolution techniques using deep learning struggle to accurately recover high resolution images from low resolution images captured by small imaging systems like transnasal endoscopes, as they fail to consider the specific imaging characteristics of these systems, leading to inaccurate resolution recovery.

Innovation Solution

A trained model is developed to simulate the imaging characteristics of a low resolution imaging system, including optical system simulation and image sensor type, to accurately recover high resolution images by generating appropriate low resolution training images from high resolution images, using neural networks like CNNs, and incorporating blur and demosaicing processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a trained model is used to super resolve low resolution images, then the resolution of the output image is improved, but the accuracy of resolution recovery deteriorates when the imaging system characteristics are not considered

Engineering Contradiction:

Improveresolution recovery accuracyVSAvoidimage quality accuracy

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent changes the parameters of the training process by incorporating imaging system characteristics (optical system properties, image sensor type, imaging method) as additional training parameters. The trained model learns to map low resolution images to high resolution images while accounting for these specific parameters, thereby improving resolution recovery accuracy without sacrificing image quality accuracy.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent introduces an intermediary processing stage that simulates the imaging system characteristics during training. This intermediary component (imaging system characteristic simulation unit) generates training data that reflects the actual imaging process, allowing the trained model to accurately recover images while maintaining reliability in terms of image quality.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If low resolution processing is performed on high resolution training images, then the training data for simulating second imaging system is generated, but the resolution of the training data deteriorates

Engineering Contradiction:

Improveimaging system simulation capabilityVSAvoidtraining data resolution

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent applies preliminary action by performing low resolution processing on high resolution training images to generate training data that simulates the second imaging system before the actual inference process. This preliminary generation of simulated low resolution training data allows the model to learn the mapping relationship while preserving the necessary resolution information for accurate recovery.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent creates a copy of the imaging process by simulating the second imaging system characteristics on high resolution training images. This copying approach generates training data that replicates the actual imaging conditions without permanently reducing the resolution of the original training images, thereby maintaining adaptability while preserving measurement precision.

Inventive Principle:

Principle #26Copying

3Measurement precision

If optical system simulation processing is performed, then the resolution characteristic of the optical system is simulated, but the processing complexity increases

Engineering Contradiction:

Improveresolution characteristic accuracyVSAvoidprocessing complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by pre-calculating and storing optical system characteristics during the training phase. This allows the resolution characteristic simulation to be performed efficiently during inference without requiring complex real-time calculations, thereby reducing processing complexity while maintaining measurement precision.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent replaces complex mechanical/optical simulations with computational models trained during the training phase. By substituting the actual optical system simulation with a learned computational model, the processing complexity is reduced while maintaining the accuracy of resolution characteristic simulation.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20250299297A1Information processing system, endoscope system, and information storage medium

Publication Date: 2025.09.25 OLYMPUS CORPORATION(JP)

AI summary

An information processing system includes a processor. The trained model is trained to resolution recover a low resolution training image generated by low resolution processing performed on a high resolution training image to a high resolution training image that represents a high resolution image captured with a predetermined object through the first imaging system. The low resolution processing represents processing that generates a low resolution image as if captured with the predetermined object through the second imaging system and processing that simulates the second imaging method, and includes processing that simulates a resolution characteristic of an optical system of the second imaging system. The processor uses the trained model to resolution recover the processing target image captured through a second imaging system to an image having a resolution at which the first imaging system performs imaging.