Image Blending for Immersive XR via Multi-Camera Depth Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing imaging systems face challenges in generating images with high visual quality throughout the field of view due to varying optical depths and lens characteristics, leading to sub-optimal and non-immersive viewing experiences, especially in immersive XR environments.
Innovation Solution
An imaging system comprising a first camera with adjustable focus and a second camera with a wider field of view, where both cameras capture images simultaneously, and a processor determines blurred regions based on modulation transfer function variations to generate an output image by combining clear regions from the first camera with non-blurred regions from the second camera.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a camera focuses at a particular optical depth, then objects at that depth and its depth of field are captured clearly, but objects at other optical depths appear blurry
Solution Approach 1:
The patent divides the field of view into multiple depth zones (near field, intermediate field, far field) and captures images from multiple cameras positioned at different optical depths. Each camera is optimized for specific depth ranges, and the images are later blended to create a composite image with acceptable clarity across all depth zones, resolving the contradiction between focused clarity and overall depth coverage.
Solution Approach 2:
The patent combines images from multiple cameras capturing the same scene from slightly different positions and focal points. By merging these images through image blending algorithms, the system produces a final image that integrates clear regions from different source images, achieving comprehensive depth coverage while maintaining overall image quality.
2Area of stationary object
If the camera lens has high distortion to cover a wide field of view, then the field of view is expanded, but image projection becomes difficult and quality deteriorates
Solution Approach 1:
The patent segments the wide field of view into multiple overlapping sub-fields captured by different cameras. Each camera operates within a narrower angular range with lower distortion, and the resulting images are stitched together. This approach achieves wide overall coverage while maintaining acceptable projection quality in each segment.
Solution Approach 2:
The patent introduces image processing algorithms as an intermediary between the distorted camera captures and the final output. These algorithms include distortion correction, perspective transformation, and seamless blending operations that mediate the transition from high-distortion wide-angle captures to a final image with acceptable projection quality.
3Manufacturing precision
If optical focus is adjusted according to gaze direction, then visual quality is improved in the gaze-contingent region, but peripheral regions exhibit varying blur and noise
Solution Approach 1:
The patent divides the field of view into a central gaze-contingent region and peripheral regions. Multiple cameras are positioned to cover different regions, with some optimized for central viewing and others for peripheral viewing. This segmentation allows each camera to maintain appropriate focus for its designated region while contributing to an overall image with more uniform quality across the entire field of view.
4Device complexity
If a single camera is used, then the device complexity is reduced, but the ability to capture high resolution images across all depth zones is compromised
Solution Approach 1:
The patent employs multiple cameras, each dedicated to capturing specific depth zones. This segmentation of the imaging function across multiple devices enables high-resolution capture across all depth ranges, with each camera optimized for its designated depth field, thereby achieving multi-depth image quality that a single camera cannot provide.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution enables the generation of high-quality images throughout the field of view, eliminating unnatural blur and providing a realistic, immersive experience suitable for XR applications, including stereographic environment mapping and object recognition, while being robust and efficient even with gaze-tracking inaccuracies and high distortion.
Implementation Method 1
a first camera having an adjustable focus; a second camera... wherein the first camera and the second camera are arranged in a manner that the first field of view overlaps with a portion of the second field of view
Implementation Method 2
different objects in the field of view are located at different optical depths with respect to a given camera; therefore, focussing the given camera at a particular optical depth allows the objects at that particular optical depth and its depth of field to be captured clearly
Implementation Method 3
determine at least one blurred region of a given first image, based on lens characteristics of the first camera at a given focal length employed for capturing the given first image, wherein the lens characteristics are indicative of how a value of a modulation transfer function of the first camera varies across the first field of view
Data Source
Figure 1~2B
Figure 3A~4B
Figure 5A~7
AI summary
Disclosed is imaging system (100) comprising: first camera (102, 202a, 202b); second camera (104, 204a, 204b), second field of view of second camera being wider than first field of view of first camera, wherein first field of view overlaps with portion of second field of view; and processor(s) (106) configured to: capture first images and second images, wherein overlapping image segment (304) and non-overlapping image segment (306) of second image (302) correspond to said portion and remaining portion of second field of view; determine blurred region(s) (B1, B2) of first image (300); and generate output image (400) in manner that: inner image segment (402) of output image is generated from: region(s) of overlapping image segment that corresponds to blurred region(s) of first image, and remaining region of first image that is not blurred, and peripheral image segment (404) of output image is generated from non-overlapping image segment.