AR Object Pose Interaction Using 3D Spatial Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality technologies face limitations in providing natural and intuitive interactions between virtual and real objects due to reliance on two-dimensional screen gestures and neglecting the relative attitude between objects, leading to incongruent interaction experiences.
Innovation Solution
An object interaction method utilizing a camera, inertial sensor, and depth sensor to determine the pose of terminal devices and objects in a real environment coordinate system, considering both position and attitude, enabling more natural interactions by aligning virtual and real objects in a three-dimensional space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If touch operation on terminal screen is used for interaction, then interaction can be implemented, but it does not conform to people's interaction habits in daily activities resulting in inconvenience
Solution Approach 1:
The patent transitions from 2D screen touch interaction to 3D spatial interaction by detecting the pose (position and attitude) of real objects in three-dimensional space. Users can interact with virtual objects by naturally manipulating real objects in 3D space, making the interaction conform to daily habits while maintaining simplicity.
2Ease of operation
If interaction is realized only based on positional relationship between objects, then interaction can be implemented, but it leads to scenes that do not conform to the common sense of interaction resulting in poor user interaction experience
Solution Approach 1:
The patent enhances interaction accuracy by incorporating attitude (orientation) information in addition to position, creating a 6-degree-of-freedom pose detection system. This allows the system to understand not just where objects are, but how they are oriented, enabling interactions that conform to common sense such as grasping objects from appropriate directions.
Solution Approach 2:
The patent replaces simple positional detection with a comprehensive pose detection system using camera and depth sensor data to calculate both position and attitude of objects. This substitution enables the system to detect rotation angles and orientation, allowing for more accurate and intuitive interaction scenarios.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Disclosed are an object interaction method, apparatus and system, a computer-readable medium, and an electronic device, which relate to the technical field of augmented reality. The object interaction method comprises: determining a pose of a terminal device under a real environment coordinate system (S30); according to pixel information and depth information of a first object, determining a pose of the first object relative to the terminal device, wherein the first object is a physical object in a real environment (S32); according to the pose of the terminal device under the real environment coordinate system and the pose of the first object relative to the terminal device, determining a pose of the first object under the real environment coordinate system (S34); determining a pose of a second object under the real environment coordinate system, wherein the second object is a virtual object (S36); and according to the pose of the first object under the real environment coordinate system and the pose of the second object under the real environment coordinate system, controlling the first object and the second object to interact with each other (S38). Interaction between virtual and physical objects can be realized in a three-dimensional scale, such that the interaction effect is more realistic.