Mobile Device Localization by Fusing Camera and Inertial Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining the location of mobile devices, especially in GPS-denied environments, face challenges such as requiring precise camera calibration, failing in texture-poor environments, and needing time-synchronization of image and motion data, which can lead to significant errors and limitations in scale estimation and orientation determination.
Innovation Solution
A computer-implemented method using three neural networks to determine the location of a mobile device with a camera and inertial sensors, where the first and second neural networks extract features from images and motion data respectively, and a third neural network combines these features to determine the device's location, allowing for robust and reliable location determination without the need for precise calibration or time-synchronization, and enabling end-to-end training with raw data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional visual odometry techniques are used to determine location from camera images, then location can be determined in GPS-denied environments, but precise camera calibration is required and the system fails in texture-poor environments or when the camera is rotated rapidly
Solution Approach 1:
The patent replaces traditional mechanical calibration procedures with a neural network-based system that learns calibration parameters automatically from data. The neural network model is trained to determine camera calibration parameters and motion parameters without requiring manual calibration, thereby eliminating the need for precise mechanical calibration while maintaining location determination reliability
Solution Approach 2:
The patent changes the approach from fixed calibration parameters to dynamically learned parameters. The neural network learns calibration and motion parameters from training data, allowing the system to adapt to different environments and conditions without requiring manual recalibration, thus improving reliability in texture-poor environments and during rapid camera rotation
2Measurement precision
If image data and motion data are combined to determine location, then more accurate location information can be obtained, but the data needs to be time-synchronised which makes the system difficult to set up and can lead to significant errors with miscalibration
Solution Approach 1:
The patent introduces a neural network model as an intermediary that processes both image data and motion data together. The model learns to fuse these data sources and automatically handles the time-synchronisation issue by learning the temporal relationships during training, eliminating the need for complex manual time-synchronisation setup while maintaining high measurement precision
Solution Approach 2:
The neural network model performs self-calibration and self-synchronisation by learning from training data. The system automatically adjusts its internal parameters to optimally combine image and motion data without requiring external calibration procedures or manual time-synchronisation, thereby reducing device complexity while maintaining accurate location measurement
3Adaptability or versatility
If known neural network systems are used to process camera images for location determination, then location can be determined without GPS, but the systems require training for specific environments and cannot be used in new environments without suitable training
Solution Approach 1:
The patent designs a universal neural network model that can operate across multiple environments without requiring environment-specific training. The model is trained on diverse data during the initial training phase to learn generalizable features and relationships, enabling it to function effectively in new environments without additional training, thus improving environmental adaptability while minimizing training time requirements
Data Source
AI summary
A computer-implemented method of determining the location of a mobile device comprising a camera and at least one inertial sensor. A sequence of images are captured over a period of time, and a first neural network determines features indicative of the motion of the device from pairs of consecutive images. Data indicative of the motion of the device is captured, and a second neural network determines features indicative of the motion of the device from the data. A third neural network then determines features indicative of the location of the device from the features determined by the first neural network and the second neural network. The location of the device is the determined from the features determined by the third neural network.