Indoor Image Localization Using Multi-Device Visual and IMU Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in effectively capturing, representing, and using building interior information, such as visual data, without physical presence, and constructing and maintaining accurate floor plans for navigation and analysis.
Innovation Solution
Automated techniques determine image acquisition locations within building interiors using data from multiple devices, including cameras and mobile computing devices, without depth sensors, by combining visual data and additional data from IMU sensors, SLAM, SfM, and MVS analysis to generate 3D room shapes and orientations, enabling accurate floor plan augmentation and navigation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If automated image acquisition location determination is performed without depth sensors, then device complexity is reduced, but measurement precision of image locations may deteriorate
Solution Approach 1:
The patent introduces mobile computing devices as intermediary carriers that hold camera devices. These mobile devices provide IMU sensors and processing capabilities that enable accurate image location determination without requiring depth sensors on the camera itself. The mobile device acts as a mediator between the camera and the processing system, providing alternative measurement channels through IMU data and visual SLAM algorithms.
Solution Approach 2:
The patent replaces mechanical depth sensing systems with computational geometry approaches. Instead of using physical depth sensors to measure distances, the system uses visual data from multiple images combined with IMU sensor information and geometric algorithms (visual SLAM, SfM, MVS) to computationally determine three-dimensional room shapes and image acquisition locations, substituting mechanical measurement with computational inference.
2Measurement precision
If multiple data capture devices are used to determine image acquisition locations, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent merges multiple data capture devices (camera devices and mobile computing devices) into an integrated system. The mobile computing devices serve dual purposes: they carry the camera devices for image capture while simultaneously providing IMU sensing and computational processing capabilities. This consolidation allows multiple functions to be achieved through coordinated operation of fewer physical devices.
Solution Approach 2:
The mobile computing devices perform multiple functions: they serve as carriers for camera devices, provide IMU sensing for motion tracking, execute visual SLAM algorithms for localization, and process visual data for three-dimensional room shape reconstruction. This multi-functionality reduces the need for separate dedicated devices for each function, thereby managing system complexity while maintaining high measurement precision.
3Manufacturing precision
If floor plans are manually constructed and maintained, then manufacturing precision can be controlled, but productivity decreases
Solution Approach 1:
The patent implements self-service automation where the system automatically generates and updates floor plans without human intervention. The automated image acquisition location determination system processes visual data from multiple images, reconstructs three-dimensional room shapes, and generates accurate floor plans autonomously. The system also automatically detects and tracks structural changes in buildings, updating floor plans in real-time without requiring manual surveying or redrawing.
Solution Approach 2:
The patent performs preliminary automated capture and processing of building interior data before floor plan generation is needed. By continuously acquiring images and processing them to determine acquisition locations and reconstruct room shapes in advance, the system prepares accurate floor plan data proactively. This preliminary action eliminates the need for manual floor plan construction when actually needed, significantly improving productivity while maintaining precision through automated processing.
Data Source
Figure 1A~1B
Figure 2A
Figure 2B~2C
AI summary
Techniques are described for using multiple devices to automatically determine the acquisition location of an image from a camera device, such as within a building interior and by using data from the multiple devices (e.g., visual data from the camera device's image, and additional data captured by a separate mobile computing device in the same area as the camera device), and for subsequently using determined image acquisition location information in one or more further automated manners. The image may be a panorama image or of another type, and the determined acquisition location for such an image may be at least a location on the building's floor plan - in addition, the automated image acquisition location determination may be further performed without having or using information from any depth sensors or other distance-measuring devices about distances from an image's acquisition location to walls or other objects in the surrounding building.