Stereoscopic AR Texture Mapping Using Real Camera Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for fusing real and virtual spaces in stereoscopic displays are limited by the need for pre-generated CG images, lacking diversity and expression, as they do not effectively utilize images shot by real cameras.
Innovation Solution
An information processing system that uses real cameras to generate stereoscopic images by setting virtual cameras in accordance with real camera positions and orientations, applying real images as textures to virtual models, and displaying these images in a manner that varies with the real images shot by the cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pre-generated CG images are used for stereoscopic augmented reality, then the system can display virtual objects, but the expression diversity is limited and the range of display manners is restricted
Solution Approach 1:
The patent applies the copying principle by capturing real-world images with a camera and using these images as textures for virtual models. Instead of generating all visual content through CG rendering, the system copies actual visual information from the real environment and applies it to virtual objects, thereby achieving diverse expressions while maintaining system manageability
Solution Approach 2:
The patent merges real captured images with virtual model structures by applying real images as textures to virtual models. This combination allows the virtual objects to inherit the visual diversity of real-world scenes while maintaining the structural and positional control of virtual models, resolving the contradiction between expression diversity and system complexity
2Manufacturing precision
If real images are applied as textures to virtual models, then high-resolution stereoscopic display is achieved, but image quality may be reduced during processing
Solution Approach 1:
The patent applies preliminary action by capturing high-resolution real images before they are needed for texture application. The images are stored and processed only when required for rendering virtual models, ensuring that the original high-quality visual information is preserved and not degraded through repeated processing or regeneration
Solution Approach 2:
The system creates a copy of the real image data and applies it as texture to the virtual model without modifying the original captured image. This copying approach allows the high-resolution source image to be preserved while the texture application process can proceed with the copied data, minimizing information loss
3Adaptability or versatility
If virtual cameras are set according to real camera positions and orientations, then diverse display manners are enabled, but the processing complexity increases
Solution Approach 1:
The patent copies the position and orientation parameters from real cameras to define virtual cameras in the virtual space. By replicating the real camera's spatial configuration rather than creating entirely new virtual camera setups, the system achieves diverse display perspectives while simplifying the configuration process through direct parameter transfer
Solution Approach 2:
The virtual camera system is designed to be universal by accepting position and orientation data from any real camera configuration. This multi-functional approach allows the same virtual camera mechanism to handle diverse real-world camera setups, enabling display manner diversity without proportionally increasing processing complexity
Data Source
Figure 1
Figure 2
Figure 3(a)~3(d)
AI summary
A marker present in a real space is detected from each of a real image for a left eye and a real image for a right eye. The position and the orientation of each of imaging sections relative to the marker are calculated. A left virtual camera and a right virtual camera are set in accordance with the positions and the orientations of the imaging sections. A virtual model having a planar shape is placed in a marker coordinate system, and areas in the real image for a left eye and the real image for a right eye that correspond to the virtual model are set as a left texture area and a right texture area, respectively. Each of images included in a left texture area and a right texture area is applied, as a texture, to a deformation object into which the virtual model is deformed.