AR Object Interaction Using Pose Tracking and 3D Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) technologies lack focus on improving the authenticity of virtual objects, leading to poor user experience.
Innovation Solution
An AR interaction method that determines the pose information of real objects and sound sources, generates AR images with virtual objects, and plays 3D audio data to enhance the authenticity of virtual objects by simulating their interaction with real objects through visual and auditory perceptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional AR technology is used to integrate virtual and real worlds, then virtual objects can be displayed, but the authenticity of virtual objects is poor resulting in poor user experience
Solution Approach 1:
The patent creates virtual copies of real objects by capturing their pose information and reproducing it in the virtual environment. The virtual object is generated based on the real object's spatial coordinates, orientation, and physical characteristics, allowing the virtual representation to accurately mirror the real object's appearance and behavior, thereby enhancing authenticity.
Solution Approach 2:
The system continuously tracks the real object's pose and updates the virtual object's position and orientation in real-time. This feedback mechanism ensures that the virtual object remains synchronized with its real-world counterpart, maintaining high authenticity as the real object moves or changes姿态.
2Reliability
If virtual objects are made more authentic through detailed tracking and rendering, then user experience improves, but computational requirements and processing time increase
Solution Approach 1:
The system pre-processes and stores key characteristics of real objects such as color, shape, and texture information before AR rendering. This preliminary preparation allows the virtual object to be quickly generated with high fidelity during runtime, reducing processing time while maintaining authenticity.
Solution Approach 2:
The patent implements selective tracking by focusing computational resources on the most visually significant aspects of the real object (such as dominant colors, key geometric features, and primary motion patterns) rather than attempting to capture every detail. This partial action approach achieves sufficient authenticity for user experience while significantly reducing computational overhead.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This application provides an augmented reality interaction method and an electronic device. In this solution, an electronic device may determine pose information of a virtual object based on pose information of a real object or a location of a real sound source, to generate and display an AR image, and may further generate and play 3D sound data of the virtual object based on the pose information of the real object and the pose information of the virtual object. It is clear that, according to this solution, the virtual object may perceive a pose of the real object and the location of the real sound source, and make a corresponding action response or sound response based on the foregoing information, so that the virtual object may be the same as the real object. Therefore, in this method, virtual-real interaction that combines visual and auditory perceptions of the virtual object and the real object is implemented. This can improve an intelligent level of the virtual object, and further improves experience of virtual interaction.