AR Display Control Device Using Intermittent SLAM Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality technologies require significant computational resources to estimate camera position and orientation using SLAM for each frame, leading to high processing loads and potential delays in virtual image updates, especially on low-performance computers.
Innovation Solution
A display control device that acquires images at a predetermined frame rate, estimates camera position and orientation, and combines virtual images with taken images, using a two-stage processing approach where complex processing is performed only intermittently to reduce computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If SLAM processing is performed for each frame to estimate camera position and orientation, then the accuracy and responsiveness of augmented reality display is improved, but the processing load on the computer increases significantly
Solution Approach 1:
The patent segments the processing into two distinct modes: a first processing mode that performs full SLAM processing at low frequency (e.g., 30fps) to maintain accuracy, and a second processing mode that performs simplified processing at high frequency (e.g., 60fps or higher) to reduce load. This segmentation allows the system to balance between measurement precision and processing complexity by applying different processing intensities at different time intervals.
Solution Approach 2:
The patent implements periodic switching between the first processing mode (full SLAM) and the second processing mode (simplified processing). By periodically performing comprehensive SLAM processing only when necessary (at lower frame rates) and using simplified processing for intermediate frames, the system maintains accurate camera estimation while reducing overall computational burden through rhythmic, periodic action patterns.
2Device complexity
If SLAM processing is performed at low frame rates to reduce processing load, then the processing load is reduced, but the smoothness and naturalness of virtual object updates deteriorates
Solution Approach 1:
The patent dynamically adjusts the processing frequency and mode based on real-time conditions. The system can switch between first processing mode (lower frequency, full SLAM) and second processing mode (higher frequency, simplified processing) depending on computational resources available and movement characteristics. This dynamic adaptation allows the system to maintain smooth virtual object updates by increasing processing frequency when needed while reducing load when possible.
Solution Approach 2:
The patent performs preliminary comprehensive SLAM processing at the first frame rate to establish accurate baseline camera position and orientation estimates. These preliminary results are then used to inform and accelerate subsequent processing at the second frame rate, allowing the system to maintain high update speeds for smooth display while relying on pre-computed accurate data from the lower-frequency full processing mode.
Data Source
AI summary
Acquisition means of a display control device acquires taken images taken at a predetermined frame rate by image taking means, which is movable in a real space. First display control means estimates a current position and orientation of the image taking means based on the taken images, and combines a virtual image with one of the taken images to be displayed, the virtual image showing a view of a virtual three-dimensional object as seen from a virtual viewpoint based on a result of the estimation. Second display control means processes, in a frame subsequent to a frame on which the first display control means has performed display control, the virtual image based on movement information on a movement of the image taking means, and combines the processed virtual image with another one of the taken images to be displayed.