Mobile Augmented Reality Object Identification via Projection Masks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mobile augmented reality systems fail to provide precise information about specific objects within live views captured by mobile computing devices, especially when objects are located in buildings with multiple businesses, as they rely solely on 3D point cloud data which is insufficient for distinguishing between different points of interest.
Innovation Solution
The system processes 3D point cloud data in conjunction with image and location data to generate projection masks that accurately identify and highlight specific objects within a live view, using techniques like noise removal, confidence information, and color matching to differentiate between objects, and then overlays relevant information in real-time, ensuring the data is displayed three-dimensionally consistent with the device's position and orientation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D point cloud data is used for object identification, then basic location information can be provided, but precise identification of specific objects within buildings containing multiple businesses cannot be achieved
Solution Approach 1:
The patent combines 3D point cloud data with 2D image data from the camera to create a more comprehensive representation of the environment. By merging these different data types, the system can identify specific objects within buildings that contain multiple businesses, overcoming the limitation of using 3D point cloud data alone which lacks the detailed visual information needed for precise object identification.
Solution Approach 2:
The system transitions from relying solely on 3D spatial data to incorporating 2D image dimensions. This dimensional addition allows the system to distinguish between different points of interest within the same building by analyzing visual features in the 2D image plane, thereby achieving precise identification of specific objects that would be indistinguishable using only 3D point cloud information.
2Loss of information
If basic location information is displayed in the live view, then users can see general surrounding context, but precise information about specific businesses or objects cannot be conveyed
Solution Approach 1:
The patent segments the environment into distinct regions or zones based on the combined 3D point cloud and 2D image data. By dividing the scene into manageable segments corresponding to different objects or points of interest, the system can selectively process and display information about specific businesses or objects without having to analyze the entire scene, thus reducing computational complexity while maintaining information completeness.
Solution Approach 2:
The system applies different processing qualities and levels of detail to different regions of the live view based on their importance and the user's needs. High-priority areas such as identified businesses or objects of interest receive more detailed processing and information overlay, while less important areas use simpler processing, thereby balancing information completeness with computational efficiency.
3Measurement precision
If the system attempts to identify specific businesses in buildings with multiple businesses, then precise location information can be provided, but current solutions fail to differentiate between multiple points of interest
Solution Approach 1:
The patent utilizes color information from the 2D image data to differentiate between various objects and points of interest within the same building. By analyzing color features, textures, and visual characteristics in the image plane, the system can distinguish between multiple businesses or objects that occupy different spatial locations, enabling precise identification even when they are in close proximity or within the same building structure.
Data Source
AI summary
Systems, apparatuses and methods to provide image data, augmented with related data, to be displayed on a mobile computing device are disclosed. An example mobile device includes a camera to provide images of a scene from different angles to a server, at least one sensor to sense a position and an orientation of the camera, and a screen to present augmented reality data over the scene based on the position and the orientation of the camera and on a three-dimensional representation of the scene based on the images.