Front-Facing Camera Pose Estimation for Smartphone Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current smartphone navigation systems using GNSS and MEMS sensors face performance issues due to low-cost sensors and poor GNSS conditions, and existing methods that incorporate rear-facing cameras are inconvenient for users as they require the camera to point towards the environment, failing to accurately determine the user's pose relative to the environment.
Innovation Solution
The integration of front-facing electro-optical (EO) sensors, such as cameras, with existing GNSS and INS systems to estimate the user's position, velocity, and orientation by matching feature points from these sensors with a user model, allowing for improved navigation performance and convenience by pointing the camera towards the user.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If rear-facing camera is used for navigation, then navigation accuracy is improved through environmental feature matching, but user convenience deteriorates as the camera must point toward the environment
Solution Approach 1:
The patent inverts the traditional camera orientation approach by using the front-facing camera instead of the rear-facing camera for navigation. The front-facing camera captures images of the user's face or body, and the system extracts feature points from these images to determine the user's pose and navigation status, eliminating the need for the camera to point toward the environment.
2Ease of manufacture
If low-cost MEMS sensors are used, then device cost is reduced, but navigation performance deteriorates under poor GNSS conditions
Solution Approach 1:
The patent combines multiple sensor types (GNSS, MEMS, and front-facing camera) into a unified navigation system. The camera-based pose estimation complements the GNSS and MEMS measurements, providing additional observability that improves navigation performance especially in poor GNSS conditions, while maintaining the use of cost-effective components.
Solution Approach 2:
The front-facing camera acts as an intermediary sensor that provides alternative measurement pathways. When GNSS signals are weak or unavailable, the camera-based system can independently estimate user pose and navigation status, bridging the gap between low-cost sensor limitations and required navigation reliability.
3Measurement precision
If feature points are extracted from camera observations, then user pose estimation is improved, but computational complexity increases
Solution Approach 1:
The patent applies local quality by focusing computational resources on extracting and matching only the most informative feature points from the camera images rather than processing the entire image. This selective approach maintains high pose estimation accuracy while reducing overall computational burden.
Data Source
AI summary
A method and an apparatus are provided that includes acquiring observations by one or more electro-optical (EO) sensors on a front-facing surface of a user equipment (UE). A processor of the UE extracts feature points from the observations expressed in a first coordinate frame of the UE. The processor matches the feature points to corresponding points of a model in a second coordinate frame of a user of the UE. The processor determines an integrated navigation solution in a third coordinate frame based on the matched feature points.