XR Pose Data Compression Using Relative Joints and Autoencoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The significant increase in data transmission volume in multi-user XR environments, particularly due to the need to synchronize head, hand, and body joint poses, facial expressions, and eye movements across multiple participants, poses a challenge in maintaining efficient data transfer and low latency.
Innovation Solution
A method and system that involves tracking pose data, converting it into relative pose data using a reference joint, reducing interphalangeal joint data, generating data points through an autoencoder, and transmitting these points to reduce data volume.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full pose data of all joints is transmitted for each user, then movement accuracy and interaction precision are improved, but data transmission volume increases significantly
Solution Approach 1:
The patent extracts and transmits only the essential movement information needed for XR interaction. By identifying and removing redundant joint data that does not contribute to meaningful interaction, the system maintains pose accuracy while significantly reducing data volume. The encoder selectively processes joint data to extract only the most relevant movement characteristics.
Solution Approach 2:
The patent transforms the representation of pose data by changing parameters from absolute joint positions to relative movement vectors. This parameter transformation reduces data redundancy while preserving the essential movement information needed for accurate XR interaction, effectively compressing the data without losing critical precision.
2Quantity of substance
If data compression techniques are applied to reduce transmission volume, then data transmission efficiency is improved, but pose data precision may deteriorate
Solution Approach 1:
The patent introduces an encoder as an intermediary component that bridges the gap between full-precision pose data and compressed transmission data. The encoder processes the original pose data through learned transformations, producing a compressed representation that retains sufficient precision for XR interactions while significantly reducing data volume. This intermediary layer enables lossy compression with controlled precision loss.
Solution Approach 2:
The patent performs preliminary processing of pose data before transmission by pre-computing relative movement vectors and filtering out redundant information. This preliminary action prepares the data in a format that is inherently more compressible while preserving essential movement characteristics, reducing the burden on subsequent compression stages and maintaining precision.
3Quantity of substance
If relative pose data conversion is performed for all joints, then data redundancy is reduced, but computational complexity increases
Solution Approach 1:
The patent segments the body into hierarchical groups of joints (e.g., upper body, lower body, extremities) and processes relative pose conversions separately for each segment. This segmentation allows the system to reduce redundancy in each local group without having to compute relative positions for all joints globally, thereby reducing computational complexity while still achieving significant redundancy reduction.
Data Source
AI summary
The embodiments of the disclosure provide a method and system for reducing data amount. The method includes: tracking a first pose data of each of a plurality of first joints on a first body part, wherein the first body part comprises a first extension portion having a plurality of second joints among the first joints; converting the first pose data of each of the first joints into a second pose data of each of the first joints with respect to a reference joint among the first joints; reducing the second pose data of the second joints based on a first base joint of the second joints; generating a plurality of first data points by feeding the second pose data of each of the first joints into an encoder of an autoencoder; and transmitting the first data points.