Display Image Generation Using Intermediate Images for AR Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of accurately aligning two-dimensional captured images with three-dimensional computer graphics (CG) in augmented and mixed reality systems, especially when the display field of view changes with user movement, leading to positional deviations and user discomfort during interactions with virtual objects.
Innovation Solution
A display image generation apparatus and method that uses intermediate images to align CG with captured images by projecting virtual objects onto a screen coordinate system corresponding to the imaging apparatuses, allowing for precise alignment and synthesis, even when the display field of view changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If captured image and CG are synthesized using traditional alignment methods, then alignment accuracy may be maintained in static conditions, but alignment precision deteriorates when display field of view changes with user movement
Solution Approach 1:
The system pre-generates intermediate images of virtual objects from the camera's viewpoint before actual display. These intermediate images are created in advance and stored, allowing rapid retrieval and synthesis without real-time rendering delays. This preliminary preparation enables accurate alignment even when the field of view changes, as the intermediate images already account for the camera's perspective.
Solution Approach 2:
The invention introduces intermediate images as a mediator between the virtual object data and the final synthesized display image. These intermediate images serve as a bridge that combines the advantages of pre-computed alignment accuracy with the flexibility to adapt to changing field of view. The intermediate images contain pre-calculated perspective information that facilitates accurate synthesis without requiring complex real-time computations.
2Measurement precision
If intermediate images are used for all virtual objects, then alignment accuracy improves, but computational load increases
Solution Approach 1:
The system applies different processing strategies to different virtual objects based on their individual characteristics and requirements. For objects that benefit from intermediate images, the system generates and uses them. For objects where this would be unnecessary overhead, the system uses direct rendering. This localized application of the intermediate image technique optimizes the balance between accuracy and computational load.
Solution Approach 2:
Instead of applying intermediate image generation to all virtual objects universally, the system selectively applies it only where necessary to achieve the desired alignment accuracy. This partial application reduces the overall computational burden while maintaining synthesis accuracy for the most critical objects that require precise alignment with the captured image.
3Measurement precision
If three-dimensional positional relations are precisely expressed, then user interaction accuracy improves, but system complexity increases
Solution Approach 1:
The system creates two-dimensional intermediate images that copy and represent the three-dimensional virtual objects from the camera's viewpoint. These intermediate images preserve the essential positional and spatial relationships needed for accurate user interaction without requiring the full complexity of three-dimensional processing. The copying approach simplifies the data structure while maintaining the accuracy needed for alignment and interaction.
Data Source
AI summary
Provided is a display image generation apparatus including a captured image acquisition section that acquires data of an image captured by a camera, an object arrangement section that arranges a virtual object to be operated by a user in a virtual three-dimensional space, a display image generation section that generates a display image by drawing an image of the virtual object and synthesizing the image of the virtual object with the captured image, and an output section that outputs data of the display image. The display image generation section switches whether or not to use an intermediate image representing the image of the virtual object from a viewpoint of the camera when drawing the image of the virtual object, according to a state of a display world including the virtual object.