AR Service Apparatus Using 2D Video and Depth Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AR technologies face limitations in providing realistic 3D interactions with famous persons, such as K-pop stars, due to the time-consuming process of generating 3D avatars and the restricted ability to offer dynamic 3D experiences using 2D approaches, which lack natural interaction and are limited by pre-defined positions and scenarios.
Innovation Solution
A method and apparatus that combine 3D and 2D video image information to adjust the 3D space of AR content based on real-time 3D information of the object, allowing for dynamic 3D interaction by integrating live RGBD processing, AR content processing, and rendering units to synchronize the cybernaut's position with the AR content, enabling a realistic 2D/3D AR experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a 3D avatar of a real person is generated using 3D scanning or 3D modeling, then realistic AR interaction is enabled, but a lot of work time is required and technical restrictions limit precise shape expression and facial expression
Solution Approach 1:
The patent uses 2D video footage of the real person as a copy source instead of creating a 3D model. The 2D video is directly synthesized into AR content, avoiding the time-consuming 3D scanning and modeling process while preserving the person's appearance and expressions as they appear in the video.
Solution Approach 2:
The patent transforms the representation parameters of the person from 3D spatial coordinates to 2D video frame sequences. By changing the dimensional parameters from three-dimensional modeling to two-dimensional video synthesis, the system achieves realistic AR interaction without requiring complex 3D avatar generation processes.
2Productivity
If a 2D video of a real person is synthesized as AR content, then production time is reduced, but 3D interaction with the cybernaut is limited and the experience is restricted to pre-defined positions
Solution Approach 1:
The patent introduces depth information as a third dimension to the 2D video content. By synthesizing depth maps from multiple 2D video angles or using depth sensing technology, the system creates 2.5D or 3D-enhanced AR content that maintains production efficiency while enabling spatial interaction and dynamic positioning.
Solution Approach 2:
The patent enables dynamic positioning and interaction by making the AR content responsive to the cybernaut's movements and gestures in real-time. Instead of fixed pre-defined positions, the system dynamically adjusts the 2D video synthesis based on the user's spatial location and interaction inputs, allowing versatile 3D-like interaction experiences.
3Manufacturing precision
If 3D modeling and character rigging are used for AR service, then high quality digital content is achieved, but technical restrictions prevent realistic synthesizing for various factors such as precise shape expression and facial expression
Solution Approach 1:
The patent bypasses 3D modeling entirely by directly copying and synthesizing 2D video footage of the real person into AR content. This approach preserves the authenticity of facial expressions and shape details as they naturally appear in video, avoiding the loss of precision that occurs during 3D digitization and rendering processes.
Data Source
AI summary
The present invention relates to a method and apparatus for providing a realistic 2D/3D AR experience service based on a video. Particularly, the present invention provides a method of providing a realistic AR service with high quality based on AR by using a 2D video image and 3D information in association thereto at the same time, filming a cybernaut and AR content by using two sensors having viewpoints different from each other, and arranging 2D/3D content being an object to be matched within the AR content and which has a 3D position different to a position of the cybernaut in association with a dynamic 3D position of the cybernaut and with a dynamic viewpoint of a service sensor.