Mixed-Reality Avatar Superimposition for Seamless Physical-Virtual Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional VR and AR applications fail to provide seamless integration of ultra-high resolution, lifelike holographic three-dimensional objects with real physical objects for interactive and immersive mixed-reality experiences, lacking advanced interactivity and visualization capabilities such as dynamic real-time rendering and virtualized switchover roles between holograms and physical objects.
Innovation Solution
A method for creating physical target movement-mirroring 3D holographic avatar superimposition and visualization in a mixed-reality environment, involving the selection of a 3D holographic avatar, correlation with a physical target, tracking of motions and poses, and real-time retargeting of movements and expressions using deep-learning pose estimation, allowing the holographic avatar to mimic the physical target's actions and appear as a fully encapsulated representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional VR applications are used to create a purely computer-generated synthetic environment, then user interaction in virtual space is enabled, but seamless integration with real physical objects and advanced interactivity are lost
Solution Approach 1:
The patent merges virtual holographic objects with real physical objects into a unified mixed-reality environment. The system combines computer-generated 3D holograms with physical object recognition and tracking, allowing both virtual and real elements to coexist and interact in the same spatial context, thereby achieving seamless integration across virtual and physical domains
Solution Approach 2:
The mixed-reality system performs multiple functions simultaneously: it recognizes physical objects, tracks their movements, generates corresponding holographic representations, and enables interactions between virtual and real elements. This multi-functionality allows a single system to handle both VR and AR capabilities while adding advanced interactivity features
2Loss of information
If conventional AR applications are used to add informational overlays to physical objects, then information augmentation is achieved, but lifelike holographic three-dimensional objects and advanced interactivity are lost
Solution Approach 1:
The patent transitions from 2D informational overlays to 3D lifelike holographic objects. By adding the third dimension and using volumetric rendering techniques, the system creates immersive holographic representations that envelop physical objects, providing rich visual information and advanced interactivity beyond traditional AR information displays
Solution Approach 2:
The system creates accurate holographic copies of physical objects by tracking their movements and reproducing them in 3D space. These holographic copies mirror the physical objects' positions, orientations, and motions, enabling realistic virtual representations that maintain fidelity to the original physical entities
3Measurement precision
If reference coordinates are used to correlate hologram positions to physical objects, then basic positioning is achieved, but dynamic real-time motion feedback and advanced interactivity are lost
Solution Approach 1:
The patent implements continuous motion tracking and feedback mechanisms that monitor physical object movements in real-time and dynamically update corresponding holographic representations. The system uses camera-based tracking and pose estimation to capture motion data, processes it through deep-learning algorithms, and immediately reflects the movements in the holographic domain, enabling dynamic real-time interaction
Solution Approach 2:
The system performs preliminary setup by establishing coordinate systems and mapping relationships between physical objects and their holographic representations. This initial configuration enables subsequent real-time tracking and motion feedback without requiring complex calculations during runtime, improving processing speed while maintaining precision
4Ease of manufacture
If holograms are created in isolation from physical objects, then hologram generation is simplified, but advanced visualization and immersive interactivity are lost
Solution Approach 1:
The patent introduces physical objects as intermediaries that bridge the gap between simple hologram generation and immersive visualization. By using physical objects as reference points and tracking targets, the system automatically establishes spatial relationships and enables advanced interactivity without requiring complex manual configuration, thus maintaining ease of creation while achieving immersive capabilities
Data Source
AI summary
A novel electronic system provides real-time movement-mirroring and three-dimensional (3D) holographic avatar superimposition on a human subject or another physical target located in a holographic mixed-reality environment. In the visual perspectives of a holographic mixed-reality environment viewer wearing a mixed-reality headset or another mixed-reality visualization device, the physical target becomes invisible and is replaced by a 3D holographic avatar in the same coordinates of the physical space where the holographic mixed-reality environment is active. Typically, the 3D holographic avatar is chosen by a system user to represent or “body double” a particular physical target prior to activating the holographic mixed-reality environment. Once activated, the novel electronic system provides subject feature extraction, subject-to-avatar recognition, subject pose and expression matching, motion retargeting, and movement mirroring to reflect detailed movements and facial or bodily expressions of the physical target in the 3D holographic avatar in real time.