Virtual Object Control via User Skeleton Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual environments lack the ability to fully control digitized real-world objects that are not pre-determined, with limited interaction capabilities beyond head tracking and voice control, and do not allow users to control the movements of entire virtual objects using skeletal tracking information.
Innovation Solution
A system and method for controlling virtual objects by digitizing real-world objects and mapping user and object skeletons, allowing users to manipulate these objects in virtual environments through skeletal tracking, enabling real-time control of non-human objects such as pets, chairs, and stuffed animals by mirroring user movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If head tracking and voice control are used to control virtual avatars, then real-time interaction is achieved, but control is limited to pre-determined models and cannot fully control digitized real-world objects
Solution Approach 1:
The patent creates a digital copy of the real-world object by capturing its geometry and texture through imaging devices. This digital twin is then animated by mapping the user's skeletal tracking data to the object's skeletal structure, enabling full control of the digitized object without requiring pre-determined models
Solution Approach 2:
The patent replaces traditional mechanical control interfaces (controllers, keyboards) with motion capture technology. By using skeletal tracking to directly map user body movements to virtual object movements, the system substitutes mechanical input devices with a more natural and versatile motion-based control system
2Adaptability or versatility
If skeletal tracking is used to control entire virtual objects, then full body control is achieved, but this capability is not available for non-pre-determined digitized objects
Solution Approach 1:
The patent creates a universal control system that can animate any digitized object regardless of its type or original purpose. By establishing a general framework that maps human skeletal structures to object skeletal structures, the system enables full body control for chairs, pets, stuffed animals, and other non-pre-determined objects using the same methodology
3Ease of operation
If a simple one-to-one mapping between user head orientation and avatar head orientation is used, then control simplicity is maintained, but full object control using body movements is not achieved
Solution Approach 1:
The patent segments the control system into multiple independent skeletal joints and body parts. Instead of treating the user and object as single rigid entities, the system divides both into articulated segments (head, torso, limbs) that can be independently mapped and controlled, enabling nuanced full-body animation while maintaining intuitive control
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and computer-storage media having computer-usable instructions embodied thereon, for controlling objects in a virtual environment are provided. Real-world objects may be received into a virtual environment. The real-world objects may be any non-human object. An object skeleton may be identified and mapped to the object. A user skeleton of the real-world user may also be identified and mapped to the object skeleton. By mapping the user skeleton to the object skeleton, movements of the user control the movements of the object in the virtual environment.