AR Navigation Camera Pose Fusion for Sensor Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vehicle navigation systems using Augmented Reality (AR) face challenges due to sensor noise from GPS and other navigation sensors, leading to unrealistic rendering of navigation routes as they sway, wobble, or bounce, making it difficult to provide a stable and realistic AR navigation display.
Innovation Solution
The implementation of computer vision techniques for real-time visual tracking on video data from vehicle cameras, combining sensor measurements with visual tracking results to produce a fused camera pose for accurate and stable path rendering, ensuring AR route indicators align realistically with the road.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sensor measurements (GPS, etc.) are used for AR navigation rendering, then navigation functionality is provided, but sensor noise causes unrealistic rendering (swaying, wobbling, bouncing)
Solution Approach 1:
The patent combines sensor measurements (GPS, IMU) with computer vision-based visual tracking to produce a fused camera pose. This merging of multiple data sources allows the system to leverage the complementary strengths of each source while mitigating their individual weaknesses, thereby achieving stable and realistic AR rendering without the swaying and wobbling caused by sensor noise alone.
Solution Approach 2:
The patent introduces an intermediary processing layer that includes a vision system and fusion algorithm. This intermediary layer receives raw sensor data and visual tracking data, processes them through fusion algorithms, and produces a corrected camera pose that is free from high-frequency noise, thereby enabling stable AR rendering.
2Measurement precision
If sensor noise is present in navigation measurements, then navigation data is available, but rendering accuracy deteriorates (route graphics placement errors)
Solution Approach 1:
The patent implements a feedback mechanism where the visual tracking system continuously monitors the actual camera pose and compares it with the sensor-based pose estimation. The difference (error) is fed back into the fusion algorithm, which adjusts the camera pose estimation in real-time, thereby correcting graphics placement errors caused by sensor noise.
Solution Approach 2:
The patent replaces the direct mechanical use of noisy sensor measurements with a computational approach that uses vision algorithms and fusion processing. Instead of directly translating raw sensor data into graphics placement, the system substitutes a multi-step computational pipeline that includes visual feature tracking and probabilistic fusion, thereby achieving higher placement accuracy.
Data Source
AI summary
Embodiments of the present disclosure are directed to providing an Augmented Reality (AR) navigation display in a vehicle. More specifically, embodiments are directed to rendering AR indications of a navigation route over a camera video stream in perspective. According to one embodiment, visual tracking can be performed on features in the video data and camera pose, i.e., a matrix encapsulating position and orientation, can be determined for each frame of video based on both the visual tracking and navigation sensor data. These separately determined camera poses can then be merged or fused into a single camera pose that is more accurate and more stable and which can then be used in rendering more realistic AR route indicators onto the video of the real-world route captured by the camera.