Vehicle AR Overlay Using Marker-Based Pose Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality (AR) systems in vehicles face challenges with localization accuracy and orientation precision due to limitations in GPS, gyroscope, magnetometer, and SLAM techniques, which result in errors and restricted AR experiences, especially for passengers who cannot access a high-level view of the vehicle's surroundings.
Innovation Solution
The system improves localization accuracy by using a vehicle-mounted GPS unit and calibrates the orientation of onboard cameras relative to the vehicle's reference points, allowing for the generation of accurate graphical renderings of virtual content to be overlaid on real-world images, using techniques like structure from motion and triangulation, and enabling AR experiences through various devices such as smartphones, tablets, and wearable technology.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If GPS, gyroscope, magnetometer, and SLAM techniques are used for localization and orientation in AR systems, then AR functionality is enabled, but localization accuracy and orientation precision deteriorate due to errors ranging from 10-70 degrees
Solution Approach 1:
The patent introduces vehicle-mounted cameras and markers as intermediary elements to establish a reference frame between the vehicle and the external environment. The cameras capture images of markers with known positions and orientations, creating a reliable reference system that mediates between the unreliable GPS/gyroscope data and the actual vehicle position and orientation. This intermediary reference frame allows for accurate calculation of vehicle pose despite the inherent errors in conventional sensors.
Solution Approach 2:
The patent creates a virtual copy of the real-world environment by rendering virtual objects that correspond to physical markers and their surroundings. This virtual representation is then overlaid on the camera feed to create the augmented reality experience. By copying the physical marker positions and orientations into the virtual environment, the system establishes an accurate reference frame that compensates for sensor errors.
2Adaptability or versatility
If conventional AR viewing devices with multiple orientation methods are used, then AR viewing capability is provided, but measurement accuracy deteriorates due to radial GPS errors of 10-30 meters and gyroscope drift
Solution Approach 1:
The patent uses markers mounted on the vehicle as intermediary reference objects. These markers have known positions and orientations relative to the vehicle, serving as a stable reference frame that mediates between the moving vehicle and the AR viewing device. By tracking these intermediary markers, the system can accurately determine vehicle position and orientation without relying on error-prone GPS and gyroscope data.
Solution Approach 2:
The system continuously captures images of markers, calculates the vehicle's position and orientation based on marker positions, and uses this feedback to update the virtual environment rendering. This closed-loop feedback mechanism allows the system to continuously correct for drift and maintain accurate localization and orientation estimation, compensating for the limitations of conventional sensors.
3Ease of operation
If stationary 2D or 3D targets are used for AR orientation, then orientation can be determined, but the AR experience is limited to the immediate vicinity of the stationary marker
Solution Approach 1:
The patent merges the stationary marker reference method with vehicle-mounted cameras and dynamic rendering. By combining the reliable orientation determination from markers with the mobility of vehicle-mounted cameras and the versatility of dynamic virtual object rendering, the system overcomes the limitation of being confined to the immediate vicinity of stationary markers. The vehicle becomes a mobile platform that can access AR experiences across different locations while maintaining accurate orientation through marker tracking.
Solution Approach 2:
The system transitions from static AR experiences tied to fixed markers to dynamic AR experiences that move with the vehicle. The vehicle-mounted cameras and dynamic rendering engine enable the AR content to adapt to different locations and viewing angles, allowing passengers to access AR experiences throughout the vehicle's journey rather than being limited to a single stationary location.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Various embodiments are described herein for allowing a user in a vehicle to view at least one AR image of a portion of the vehicle's surroundings. At least one real world camera may be used to obtain at least one real world image of the portion of the vehicle's surroundings and at least one display may be used to display the at least one AR image to the user. Location, orientation and field of view data for the at least one real world camera is obtained and a virtual world camera having similar characteristics is generated to obtain at least one virtual world image of a portion of the virtual world data that corresponds to the real world data. The at least one AR image is generated by combining the at least one virtual and real world images.