Sensor-Free 3D Digital Avatar Telepresence With Cloud Pose Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing telepresence systems face challenges in accurately inferring real-time human motion and rendering it onto a digital model, requiring specialized body sensor suits, which increases cost and limits democratized use, and consume significant network bandwidth.
Innovation Solution
A processor-implemented method and system for real-time live telepresence using a 3-D human model to generate a digital avatar, transmitting temporally consistent 3-D human pose and shape motion information via a public cloud infrastructure, encoding data into name-value pairs, and rendering it in real-time without specialized sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If specialized body sensor suits are used to transmit human body pose information explicitly, then the accuracy of motion capture is improved, but the cost increases and flexibility is reduced
Solution Approach 1:
The patent creates a digital copy (avatar) of the remote person that replicates their appearance, gestures, and environment. Instead of using complex sensor suits on the remote person, the system captures images from their location and generates a digital twin that can be rendered in real-time, achieving accurate motion representation through computational methods rather than physical sensors.
Solution Approach 2:
The patent replaces the mechanical sensor suit system with a computational image processing system. Instead of using physical sensors attached to the body to capture motion data, the system uses cameras and algorithms to extract pose and appearance information from images, substituting mechanical measurement with optical and computational methods.
2Reliability
If real-time human motion data transmission and rendering is implemented, then the telepresence quality is improved, but the network bandwidth consumption increases
Solution Approach 1:
The patent extracts only the essential visual features needed for telepresence from the full image data. Instead of transmitting complete high-resolution video streams, the system extracts key pose information, appearance characteristics, and environmental elements, transmitting only these extracted features to reduce bandwidth while maintaining telepresence quality.
Solution Approach 2:
The patent implements partial action by transmitting and rendering only the critical components needed for effective telepresence rather than complete visual fidelity. The system focuses on rendering the avatar's pose, appearance, and key environmental elements at appropriate quality levels, avoiding unnecessary transmission of excessive data while maintaining sufficient telepresence effectiveness.
3Ease of manufacture
If a digital avatar system is implemented without specialized sensors, then the cost and accessibility are improved, but the motion capture accuracy may deteriorate
Solution Approach 1:
The patent creates a universal system that works with standard acquisition devices like cameras that are already widely available. The same system can be deployed across multiple platforms and locations without requiring specialized sensor equipment, making the technology accessible and easy to manufacture while maintaining functionality through multi-functional image processing algorithms.
Solution Approach 2:
The patent changes the parameters of the input data by transforming standard 2D images into 3D pose estimates and avatar representations through computational algorithms. By changing how the data is processed and represented rather than requiring specialized sensors, the system achieves accurate pose estimation using conventional, easily accessible devices.
Data Source
AI summary
Real-time human motion capture, real-time human motion data transmission and the data rendering are the main challenges of the typical telepresence application. The present disclosure presents a maker-less 3-D digital human-based bandwidth-efficient Telepresence solution called Tele-avatar. The methods and systems of the present disclosure divide into an initialization phase and a live rendering phase. In the initialization phase, the digital avatar model is initialized and the same is conveyed to the rendering system. The initialization is done through parametric human model creation. This digital avatar model is then transmitted to the visual rendering device of the human observer for subsequent rendering. In the live rendering phase, the changes in body postures and facial expressions over time of the remote human presenter are transmitted to the visual rendering device of the human observer in real-time for final augmentation with the live view captured in the visual rendering device.