3D Hand Coordinate Mapping for Precise AR Object Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face inaccuracies and inefficiencies in converting two-dimensional hand coordinates to three-dimensional coordinates for precise interactions due to estimation errors, leading to poor interaction experiences.
Innovation Solution
A method and apparatus that utilize a monocular camera and depth camera to directly obtain a two-dimensional screen space coordinate and real distance, determining a three-dimensional coordinate in a virtual world through a projection matrix, thereby avoiding estimation errors and improving accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If two-dimensional hand coordinates are converted to three-dimensional coordinates through estimation, then interaction with virtual objects can be performed, but measurement precision deteriorates due to estimation errors
Solution Approach 1:
The patent introduces a depth camera as an intermediary device to directly obtain real distance information between the hand and photographic device. This mediator provides accurate depth data that bridges the gap between 2D screen coordinates and 3D virtual world coordinates, eliminating the need for error-prone estimation methods while maintaining system feasibility
Solution Approach 2:
The patent replaces the traditional estimation-based coordinate conversion mechanism with a direct measurement mechanism using a depth camera. Instead of using complex mathematical estimation algorithms that introduce errors, the system substitutes this with direct optical measurement of real distance, achieving higher precision without proportionally increasing system complexity
2Productivity
If estimation process is used to convert two-dimensional coordinate to three-dimensional coordinate, then interaction can be performed, but productivity deteriorates due to low operation efficiency
Solution Approach 1:
The patent substitutes the computationally intensive estimation process with a direct measurement approach using depth camera data. The depth camera provides real distance information that can be directly integrated with 2D screen coordinates through a simplified projection matrix calculation, significantly reducing computation time while improving coordinate accuracy
Solution Approach 2:
The system performs preliminary depth measurement using the depth camera to obtain real distance information before the coordinate conversion is needed. This pre-acquired depth data is then readily available for rapid 3D coordinate calculation, eliminating the need for time-consuming estimation processes during interaction operations
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Provided are an interactive control method and apparatus, an electronic device, and a computer-readable storage medium, relating to the field of computer technologies. The interactive control method includes: obtaining a screen space coordinate of a key point of a predetermined part, and obtaining a real distance between the key point of the predetermined part and a photographic device (S 110); determining a three-dimensional coordinate of the key point of the predetermined part in a virtual world according to the real distance and the screen space coordinate (S120); and determining a spatial relationship between the key point of the predetermined part and a virtual object in the virtual world based on the three-dimensional coordinate, and controlling, based on the spatial relationship, the key point of the predetermined part to interact with the virtual object (S130). The method can obtain the accurate three-dimensional coordinate of the key point of the predetermined part in the virtual world, thereby precisely controlling the key point of the predetermined part to interact with the virtual object.