Augmented Reality Layer Embedding in 3D Virtual Environments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality systems, such as those using the Sony PS3 console and EyeToy, limit interaction with virtual entities to conventional inputs like joysticks and buttons, and struggle to seamlessly integrate real-world objects and environments into virtual spaces, restricting immersive user interaction.
Innovation Solution
The method involves capturing video images of real-world objects, such as books with fiduciary markers, and superimposing virtual graphics to create an augmented reality layer, which is then embedded within a 3D virtual environment, allowing users to interact with virtual objects and environments using motion sensors and controllers, enabling more immersive and interactive experiences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional input devices (joysticks, buttons) are used for interaction, then device complexity is reduced, but user interaction immersion and naturalness deteriorate
Solution Approach 1:
The patent replaces mechanical input devices (joysticks, buttons) with motion capture technology that detects natural human movements and gestures. The system uses cameras and sensors to track body movements, hand gestures, and facial expressions, converting these natural motions into control signals for virtual entities, thereby eliminating the need for physical controllers and enhancing immersion.
Solution Approach 2:
The system enables users to interact with virtual entities using their own natural body movements and gestures without requiring external control devices. The motion capture system automatically detects and interprets user actions, allowing the user's body to serve as the control interface, thus eliminating dependency on separate input devices.
2Ease of operation
If real-world objects are integrated into virtual environments, then user interaction immersion is improved, but system complexity increases
Solution Approach 1:
The patent merges the real-world physical environment with the virtual environment by integrating motion capture technology into the game system. Real-world objects and spaces are detected and incorporated into the virtual game world, allowing users to interact with virtual entities using physical gestures and movements in their actual environment, thereby creating a unified immersive experience.
Solution Approach 2:
The system introduces motion capture cameras and sensors as intermediaries that bridge the real world and virtual environment. These devices detect physical movements and gestures in the real world and translate them into corresponding actions within the virtual environment, facilitating seamless interaction between physical and digital domains.
3Ease of operation
If motion sensors and controllers are used for interaction, then user interaction naturalness is improved, but device complexity increases
Solution Approach 1:
The patent replaces mechanical controllers with optical motion capture systems that use cameras and computer vision algorithms to detect and track body movements. This substitution eliminates the need for physical controllers while enabling natural gesture-based interaction, as the system directly captures and interprets users' natural movements through visual sensing.
Data Source
Figure 1
Figure 2A~2B
Figure 3A
AI summary
A method of incorporating an augmented reality video image into a virtual environment comprises detecting a state of a real-world control object captured in a video image, augmenting the video image responsive to the state of the control object, calculating trajectory parameters for an action with respect to a local co-ordinate system of the video image, responsive to the state of the control object, embedding the augmented video image within a 3D virtual environment, calculating the trajectory parameters for the action with respect to a local co-ordinate system of the 3D virtual environment, and rendering a graphical effect corresponding to the action within the 3D virtual environment.