Mixed Reality Focus Adjustment via Phase Difference Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing apparatuses for mixed reality spaces struggle to appropriately adjust focus, especially when capturing images that include both real and virtual objects, leading to potential image defocus issues.
Innovation Solution
An image processing apparatus equipped with a processor and memory that acquires image signals from an image capturing element with pixels arrayed to receive light from different pupil areas, combines virtual objects with these signals, and adjusts the lens position based on the image defocus amount between paired mixed reality images, using a phase difference system for focus detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual objects are combined with real space images, then mixed reality imaging capability is improved, but focus adjustment accuracy deteriorates
Solution Approach 1:
The image capturing element is segmented into multiple pixel groups, where each group receives light from different pupil areas. This segmentation enables the system to capture multiple images with different optical paths, allowing focus detection through phase difference comparison while maintaining mixed reality imaging capability
Solution Approach 2:
The patent introduces an intermediary focus detection mechanism that operates independently from the mixed reality image generation process. By using phase difference information from segmented pixels as an intermediary measurement, the system can determine focus accuracy without being interfered by virtual object composition
2Loss of information
If pixels receive light from different pupil areas, then depth information acquisition is improved, but image processing complexity increases
Solution Approach 1:
The patent extracts only the necessary phase difference information from the light rays received by different pixel groups, rather than processing all image data. By taking out specifically the depth-related phase information and using it for focus detection, the system reduces processing complexity while maintaining depth information acquisition capability
Solution Approach 2:
The system performs preliminary phase difference calculation on the captured images before final image rendering. By calculating focus-related phase information in advance and using it for focus adjustment, the patent reduces the computational burden during real-time mixed reality image generation
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables precise focus adjustment in mixed reality image capturing, ensuring that both real and virtual objects are in focus, improving the quality of mixed reality images by calculating and correcting defocus amounts using depth information from phase differences between image signals.
Implementation Method 1
a plurality of pixels that receive rays of light having passed through different pupil areas of an image forming optical system
Implementation Method 2
adjust a lens position of the image forming optical system based on an image defocus amount between the pair of mixed reality images
Data Source
AI summary
An image processing apparatus includes one or more processors, and a memory storing instructions which, when the instructions are executed by the one or more processors, cause the image processing apparatus to function as an acquisition unit configured to acquire image signals from an image capturing element in which a plurality of pixels that receive rays of light having passed through different pupil areas of an image forming optical system is arrayed, a combining unit configured to, based on the image signals acquired by the acquisition unit, combine virtual objects with the respective image signals corresponding to the different pupil areas to generate a pair of mixed reality images, and a focus adjustment unit configured to adjust a lens position of the image forming optical system based on an image defocus amount between the pair of mixed reality images.