Mobile Device SLAM 3D Map Points for Server-Based Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional client-server based visual search methods for mobile devices suffer from high latency and increased data transmission due to the need to send large query images to servers, leading to outdated object pose calculations and inefficient data processing.
Innovation Solution
Implementing a simultaneous localization and mapping (SLAM) framework on mobile devices to generate a set of 3D map points representing environmental surfaces, which are then sent to a server for object detection, reducing data transmission and enabling live pose calculations and incremental updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If query images are sent to server for object recognition, then object detection accuracy is improved, but network transmission time and data size increase
Solution Approach 1:
The patent extracts only the essential feature descriptors and 3D map points from the query image, discarding the redundant pixel data. This allows the server to process compacted information while maintaining detection accuracy, thereby reducing transmission time and network bandwidth requirements.
Solution Approach 2:
The patent segments the image data into two parts: 2D feature descriptors for object recognition and 3D map points for spatial localization. This segmentation enables selective transmission of only the necessary components to the server, optimizing the balance between detection accuracy and transmission efficiency.
2Measurement precision
If high resolution images are used for query, then object recognition accuracy is improved, but data transmission size increases
Solution Approach 1:
The patent extracts only the essential feature descriptors and 3D map points from the query image, discarding the redundant pixel data. This allows the server to process compacted information while maintaining detection accuracy, thereby reducing transmission time and network bandwidth requirements.
Solution Approach 2:
The patent transforms the image data from high-resolution pixel format to a parameterized representation using feature descriptors and 3D coordinates. This parameter transformation maintains the semantic information needed for recognition while dramatically reducing the data size for transmission.
3Measurement precision
If server processes query images for object detection, then detection accuracy is improved, but pose information becomes outdated due to latency
Solution Approach 1:
The patent performs preliminary actions by pre-computing 3D map points and feature descriptors locally on the mobile device before transmission to the server. This preliminary processing reduces the processing time required on the server side and enables faster response, ensuring the pose information remains current despite network latency.
Solution Approach 2:
The patent creates a compressed copy of the essential spatial and feature information (3D map points and descriptors) that preserves the geometric relationships and object identification data. This copied representation enables accurate pose calculation while reducing transmission time to maintain information freshness.
Data Source
AI summary
Method, mobile device, computer program product and apparatus for performing a search are disclosed. The method of performing a search comprises receiving one or more images of an environment in view of a mobile device, generating a simultaneous localization and mapping of the environment using the one or more images, wherein the simultaneous localization and mapping of the environment comprises a plurality of map points representing a plurality of surfaces in a three dimensional coordinate system of the environment, sending a set of the plurality of map points as a search query to a server, receiving a query response from the server, and identifying an object in the environment based at least in part on the query response.