Endoscopic Image Processing with Synthetic Defocus Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing techniques for extending the depth of field in endoscopic observation require a large number of training images due to varied optical degradation information, leading to increased network scale and processing costs.
Innovation Solution
A system and method using machine learning to generate training images through defocus and best focus simulation, associating object distance with filter characteristics for blur correction, enabling blur adjustment in captured images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large number of training images are used to capture various optical degradation information, then the depth of field extension capability is improved, but the network scale and processing costs increase
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing optical degradation information (point spread functions) for multiple object distances before actual image processing. This allows the system to handle various depth scenarios without requiring a proportionally large neural network, as the degradation characteristics are prepared in advance and stored in lookup tables rather than being learned in real-time through extensive training data
Solution Approach 2:
The patent uses copying by creating synthetic training images through defocus simulation processing that replicates optical degradation effects. Instead of capturing actual images at various depths requiring complex optical degradation information, the system generates virtual blurred images by convolving in-focus images with pre-computed point spread functions, thereby reducing the need for diverse real-world training data
2Reliability
If a large number of training images are used to capture various optical degradation information, then the depth of field extension capability is improved, but the processing ability and cost increase
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing optical degradation information (point spread functions) for multiple object distances before actual image processing. This allows the system to handle various depth scenarios without requiring a proportionally large neural network, as the degradation characteristics are prepared in advance and stored in lookup tables rather than being learned in real-time through extensive training data
Solution Approach 2:
The patent introduces an intermediary approach by using a two-stage processing system: first, a neural network performs initial blur correction based on object distance classification; second, a correction table provides additional refinement based on precise object distance measurement. This intermediary correction table mechanism allows the system to achieve high processing accuracy without requiring the neural network to handle all complexity alone, thereby improving overall processing ability
Data Source
AI summary
Defocus simulation processing is performed for a region on an optical axis of a first imaging system and a region other than on the optical axis in a training image, based on a transfer function or a point spread function on the optical axis. One or more processors use a trained model to generate an output image in which a blur of a processing target image which is an image captured by the first imaging system is corrected, and estimates an object distance of the processing target image. The one or more processors acquire a filter characteristic associated with the estimated object distance from a correction table, and performs blur adjustment processing for the output image using the acquired filter characteristic.