SLAM Landmark Selection for Marker-Free AR Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision technologies face challenges in localizing and mapping environments without the use of AR markers, especially in scenarios where large or inaccessible objects are being augmented, as they require a priori knowledge of camera position and object positions, which can be impractical or inconvenient.
Innovation Solution
A system and method for simultaneous localization and mapping (SLAM) using a head-mountable display (HMD) with image sensors that detect feature points and generate a map by selecting stable landmark points based on object size and mobility classification, allowing for accurate tracking and mapping without the need for AR markers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If AR markers are used to assist in localization and mapping, then the gaming device can recognize size and orientation of objects, but it requires users to carry and position markers which is inconvenient
Solution Approach 1:
The patent extracts the localization function from external AR markers and relocates it to the environment itself by using natural features (corners, edges, intersections) as landmarks. This eliminates the need for separate marker objects while maintaining the ability to determine camera position and orientation.
Solution Approach 2:
The environment serves itself for localization purposes by providing natural geometric features that can be detected and used as landmarks. The building structures, furniture, and other environmental elements automatically provide the reference points needed for SLAM without requiring external assistance from markers.
2Measurement precision
If AR markers are used for localization and mapping, then object position can be determined, but it is impractical for large or inaccessible objects
Solution Approach 1:
The patent transitions from using 2-D AR markers to utilizing 3-D environmental structures as landmarks. By detecting corners, edges, and intersections in the 3-D environment, the system can determine camera position and orientation without being constrained by the 2-D limitations of markers, enabling application to large or inaccessible objects.
3Loss of information
If all detected feature points are used as landmark points, then comprehensive mapping is achieved, but processing efficiency decreases and tracking failures increase due to moving landmarks
Solution Approach 1:
The patent applies different selection criteria to different feature points based on their local properties. Landmarks are selected based on their stability characteristics - choosing feature points that belong to stationary objects and excluding those from moving objects. This localized quality assessment improves both mapping completeness and processing efficiency.
Solution Approach 2:
Instead of using all detected feature points, the patent selectively uses only the necessary subset that meets stability criteria. This partial action approach reduces processing load while maintaining sufficient mapping information by focusing on the most reliable landmarks.
4Reliability
If traditional SLAM techniques are used without landmark selection, then localization can be performed, but tracking failures occur due to moving landmarks
Solution Approach 1:
The system incorporates feedback about landmark stability by continuously monitoring whether detected landmarks remain stationary across multiple frames. This feedback mechanism allows the system to identify and exclude moving landmarks, improving both localization reliability and tracking accuracy through iterative refinement.
Data Source
Figure 1~2
Figure 3a~4
Figure 5~6
AI summary
A data processing apparatus comprises receiving circuitry to receive a plurality of images of an environment captured from respective different viewpoints, detection circuitry to detect a plurality of feature points in the plurality of captured images and to associate image information with each detected feature point indicative of an image property for a detected feature point, wherein each detected feature point represents a candidate landmark point for mapping the environment, selection circuitry to select one or more of the plurality of candidate landmark points, the one or more selected landmark points corresponding to a subset of the plurality of candidate landmark points, and mapping circuitry to generate, for the environment, a map comprising one or more of the selected landmark points, wherein each landmark point included in the map is defined by a three dimensional position and the associated image information for that landmark point.