Mobile Device 6DOF Localization Using Orientation-Binned Feature Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current localization technologies for mobile devices in wide-area environments, such as outdoors, face challenges in achieving accurate 6-degree-of-freedom self-localization quickly and efficiently, which is crucial for augmented reality applications, due to the limitations of existing computer vision methods that are often computationally demanding and only provide coarse-scale or 2-3 degree-of-freedom pose estimation.
Innovation Solution
A mobile device uses a combination of vision, orientation, and position sensor data to perform 6-degree-of-freedom localization by capturing images and receiving geographic orientation data, accessing a database of pre-generated 3D model features binned by orientation, and matching features to determine the device's position and orientation, enabling fast and accurate localization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional computer vision algorithms are used for localization, then measurement precision is improved, but productivity deteriorates due to computationally demanding processing
Solution Approach 1:
The patent segments the feature database into multiple bins organized by geographic orientation (e.g., north-facing, south-facing, east-facing, west-facing). This segmentation allows the system to quickly narrow down the search space by first determining the general orientation category and then searching only within the relevant bin, rather than searching through all features in the database. This resolves the contradiction by maintaining high localization accuracy through systematic search while dramatically improving processing speed through reduced search space.
Solution Approach 2:
The patent performs preliminary organization of the 3D model features into orientation-based bins during the offline data preparation phase. This preliminary action stores features in an optimized structure that enables rapid querying during runtime localization. By pre-organizing the data according to geographic orientation, the system eliminates the need for exhaustive searches during actual localization operations, thus achieving both high accuracy and fast processing speeds.
2Measurement precision
If full 6-degree-of-freedom pose estimation is performed, then measurement precision is improved, but device complexity increases due to limited processing power on mobile devices
Solution Approach 1:
The patent segments the 6DOF pose estimation problem into two independent parts: (1) determining the 2DOF geographic orientation (north/south/east/west facing) using the orientation sensor and binned feature database, and (2) determining the 3DOF position and remaining orientation using standard computer vision techniques on the filtered feature set. This segmentation reduces processing complexity by solving the orientation component first through a simplified lookup process, then using the result to constrain the subsequent position estimation, rather than performing full 6DOF estimation simultaneously which would be computationally intensive.
Solution Approach 2:
The patent introduces geographic orientation data from the orientation sensor as an intermediary that mediates between the image data and the 3D model features. This intermediary provides a coarse constraint on the possible locations, which then guides the more precise computer vision-based feature matching. The orientation data acts as a filter that reduces the search space before applying computationally intensive vision algorithms, thus achieving 6DOF accuracy with reduced processing complexity.
3Adaptability or versatility
If conventional localization approaches are used, then ease of manufacture is maintained, but adaptability deteriorates for wide-area environments
Solution Approach 1:
The patent creates a universal localization system that works across diverse wide-area environments by combining multiple data sources (image data, orientation sensor data, position data) and multiple processing strategies (geographic orientation-based binning, feature matching, pose estimation). The system is designed to handle various outdoor scenarios by using the orientation-based binning structure that can accommodate different geographic contexts. This multi-functional approach enables the system to adapt to different environments without requiring environment-specific customization, thus achieving high adaptability while maintaining manageable system complexity through a unified architecture.
Data Source
AI summary
A mobile device uses vision and orientation sensor data jointly for six degree of freedom localization, e.g., in wide-area environments. An image or video stream is captured while receiving geographic orientation data and may be used to generate a panoramic cylindrical map of an environment. A bin of model features stored in a database is accessed based on the geographic orientation data. The model features are from a pre-generated reconstruction of the environment produced from extracted features from a plurality of images of the environment. The reconstruction is registered to a global orientation and the model features are stored in bins based on similar geographic orientations. Features from the panoramic cylindrical map are matched to model features in the bin to produce a set of corresponding features, which are used to determine a position and an orientation of the camera.