Map Processing Device for Cross-Sensor Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current SLAM algorithms are limited to using either image-based or structure-based sensors, making it difficult for devices to localize using image-based sensors when only a structure-based map is available, as they cannot directly process the structure-based map for localization.
Innovation Solution
A map processing device that uses an image-based localization algorithm to localize a camera by detecting 2D features, densifying them, and matching them to a structure-based map created by a depth sensor, allowing for accurate and efficient localization even when only a structure-based map is available.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If image-based SLAM algorithms are used for localization, then the device can localize using image sensors, but it cannot process structure-based maps for localization
Solution Approach 1:
The patent introduces an intermediary conversion process that transforms structure-based map data into a format compatible with image-based SLAM algorithms. The system converts 3D structure data into 2D image projections or feature representations that image-based algorithms can process, enabling cross-format localization without requiring the device to natively support both sensor types.
Solution Approach 2:
The patent changes the parameters of the structure-based map data by transforming 3D spatial coordinates and depth information into 2D image space parameters. This parameter transformation allows the data to be processed by image-based SLAM algorithms while preserving the essential localization information.
2Measurement precision
If structure-based maps are used for localization, then depth information is available, but image-based algorithms cannot directly process this data
Solution Approach 1:
The patent creates a copied representation of the structure-based map in a format suitable for image-based processing. Instead of directly processing 3D depth data, the system generates 2D image projections or feature maps that copy the essential spatial information in a compatible format, allowing image-based algorithms to utilize depth information indirectly.
3Reliability
If existing SLAM approaches are used to convert images to depth images, then localization is possible, but the device position must be known up to several meters in advance
Solution Approach 1:
The patent performs preliminary conversion of structure-based map data into image-based representations in advance, creating a pre-processed map format that can be quickly queried during localization. This preliminary action eliminates the need for real-time image-to-depth conversion and reduces the acceptable position uncertainty from several meters to much smaller values.
4Measurement precision
If plane-based matching is used between sensor data, then correspondence can be found, but planes may not be available in all environments
Solution Approach 1:
The patent develops a universal feature extraction and matching approach that works across different environment types. Instead of relying on plane-specific features that may not exist in all scenes, the system uses general-purpose 2D image features and projections that can be applied universally, making the localization system adaptable to diverse environments including those without planar structures.
Data Source
AI summary
A map processing device obtains images from a camera transported with a mobile device, and detects 2D features in the images. At least some of the 2D features detected in the images are densified to generate a densified set of 2D features. Locations of the densified set of 2D features are determined in 3D space using an image-based localization algorithm. A structure-based map comprising depth information from a depth sensor transported by another mobile device is accessed, wherein the depth information includes a set of data points indicating locations in 3D space corresponding to features in the real-world sensed by the depth sensor. The camera transported with the mobile device is localized to the real-world based on matching the locations of the densified set of 2D features in 3D space to the depth information accessed in the structure-based map.