Markerless AR Scale Estimation via Feature Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality systems face challenges in accurately sizing and positioning digital representations within real-world environments without the use of markers or additional sensors, leading to unrealistic and unconvincing augmented reality views.
Innovation Solution
A markerless approach that utilizes inertial measurement units and feature detection algorithms to estimate the scale and orientation of real-world objects, allowing for accurate rendering of digital representations without the need for fiducials or multiple cameras, by capturing image data from different angles and generating a multi-view stereo model to determine 3D geometry.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If markerless approach is used without additional sensors, then device complexity is reduced, but measurement precision of scale and orientation deteriorates
Solution Approach 1:
The patent introduces feature detection algorithms as an intermediary between the single camera and the 3D geometry estimation process. These algorithms automatically identify and track distinctive features in the environment, serving as virtual markers that enable scale and orientation estimation without requiring physical fiducials or additional sensors, thus resolving the contradiction between device simplicity and measurement precision
Solution Approach 2:
The patent replaces the mechanical sensor-based measurement system with a computational vision system. Instead of using multiple cameras or depth sensors to directly measure 3D geometry, the system substitutes these mechanical/optical measurement devices with image processing algorithms that infer 3D information from 2D images, reducing hardware complexity while maintaining measurement capability
2Device complexity
If single camera is used instead of multiple cameras, then device complexity is reduced, but measurement precision of 3D geometry deteriorates
Solution Approach 1:
The patent transitions from 2D image capture to 3D geometry estimation by introducing temporal dimension through video sequences and spatial reasoning through feature tracking. The system captures 2D images from different angles and times, then uses multi-view stereo algorithms to reconstruct 3D geometry, effectively adding dimensional information through computational methods rather than additional spatial sensors
Solution Approach 2:
The patent performs preliminary feature detection and tracking before 3D reconstruction. By pre-identifying and tracking distinctive features across multiple frames, the system prepares the necessary correspondence information needed for accurate 3D geometry estimation from a single camera, enabling precise measurement without requiring multiple simultaneous cameras
3Ease of operation
If markerless approach is used without fiducials, then ease of operation is improved, but measurement precision of scale deteriorates
Solution Approach 1:
The patent implements self-service by enabling the system to automatically detect and utilize environmental features for scale estimation without requiring user-placed markers or fiducials. The feature detection algorithms autonomously identify suitable reference objects and features in the scene, extract their geometric properties, and use them for scaling the augmented reality content, making the system both easier to operate and self-sufficient
Solution Approach 2:
The patent changes the parameter space by transitioning from fixed physical markers to variable environmental features. The system adapts to different scenes by detecting and utilizing whatever distinctive features are present, changing the scale reference from predetermined fiducial dimensions to dynamically detected feature dimensions, thereby improving ease of operation while maintaining measurement precision through adaptive parameter selection
Data Source
AI summary
Systems and methods for a markerless approach to displaying an image of a virtual object in an environment are described. A computing device is used to capture an image of a real-world environment; for example including a feature-rich planar surface. One or more virtual objects which do not exist in the real-world environment are displayed in the image, such as by being positioned in a manner that they appear to be resting on the planar surface, based at least on a sensor bias value and scale information obtained by capturing multiple image views of the real-world environment.