Endoscope Image Synthesis Using Multi-Frame Focus Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The combination of image synthesis methods for generating all-in-focus images from differently-focused images using frame sequential imaging techniques results in decreased focus position estimation accuracy due to non-overlapping wavelength bands of object contrast and illumination light, leading to impaired image synthesis, particularly in applications like endoscope imaging where tissue structures like blood vessels are affected.
Innovation Solution
An image processing device and method that acquires and synthesizes captured images using a synthesis map, calculating a first synthesis map from images differing in focus and utilizing a second synthesis map based on information from other frames to improve focus accuracy, especially for frames with low contrast signals like R-channel images, by weighted averaging of synthesis maps from different frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If frame sequential imaging with multiple wavelength bands is used to capture differently-focused images, then the coverage of focus positions is improved, but the focus position estimation accuracy deteriorates when object contrast and illumination light wavelength bands do not overlap
Solution Approach 1:
The patent introduces a synthesis map as an intermediary that combines focus position information from multiple wavelength bands. The synthesis map calculation section integrates first synthesis maps from different frames (R, G, B channels) to create a second synthesis map that compensates for low contrast in individual channels, thereby maintaining accurate focus position estimation across all wavelength bands
Solution Approach 2:
The patent merges focus position information from multiple wavelength bands by combining first synthesis maps from different frames into a second synthesis map. This combining process allows the system to leverage strength from each wavelength band, ensuring that even channels with low object contrast (like R-channel for blood vessels) contribute to accurate overall focus estimation
2Speed
If synthesis is performed using only current frame information, then processing speed is maintained, but synthesis accuracy deteriorates for frames with low contrast signals
Solution Approach 1:
The patent performs preliminary calculation of first synthesis maps for all frames (R, G, B) before final image synthesis. These first synthesis maps are pre-computed and stored, then rapidly combined into a second synthesis map during the actual synthesis process, enabling both speed and accuracy
3Measurement precision
If multiple first synthesis maps from different frames are combined to create a second synthesis map, then focus position estimation accuracy is improved, but calculation complexity increases
Solution Approach 1:
The patent changes the parameter combination strategy by weighting and averaging focus position information from different wavelength bands according to their respective contrast characteristics. This parameter-based fusion approach systematically manages calculation complexity while improving accuracy through controlled integration of multiple data sources
4Loss of information
If frame sequential imaging is used to capture R, G, and B frames, then color information is improved, but the wavelength band mismatch problem worsens for specific tissue structures
Solution Approach 1:
The patent applies local quality by treating each wavelength band (R, G, B) differently based on its contrast characteristics for specific tissue structures. The synthesis map calculation selectively weights information from each channel according to its local contrast performance, allowing optimal use of each wavelength band's strengths while compensating for its weaknesses
Data Source
AI summary
An image processing device includes a processor including hardware, the processor being configured to implement an image acquisition process that acquires captured images from an imaging section that performs a frame sequential imaging process in which one cycle includes first to N-th frames, and a synthesis process, wherein the processor implements the image acquisition process that acquires a plurality of captured images that have been captured in an i-th frame and differ from each other as to an in-focus object plane position, and the processor implements the synthesis process that calculates a second synthesis map based on a first synthesis map calculated with respect to the i-th frame, and the first synthesis map calculated with respect to a k-th frame, and synthesizes the plurality of images that have been captured in the i-th frame based on the second synthesis map.