Vehicle AR Display Using 3D Mesh Maps Beyond Sensor Range
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time augmented reality systems in vehicles face challenges in rendering accurate and comprehensive environmental data due to limitations in on-board sensors, such as range, occlusions, and adverse weather conditions, which affect the quality and response time of the AR display.
Innovation Solution
The use of pre-generated stereographic 3D reconstruction and 3D mesh map data to enhance AR rendering capabilities, allowing the system to project 3D elements beyond sensor limitations, fill in occluded areas, and optimize queries by leveraging static imagery from multiple sources, including aerial and street photography.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If on-board sensors (LiDAR, cameras) are used to collect real-time environmental data, then the AR system can provide up-to-date information about nearby objects, but the sensors have limited range and are blocked by occluding objects, causing incomplete environmental data
Solution Approach 1:
The patent combines pre-generated 3D mesh map data from multiple sources (aerial photography, street-level imagery, LiDAR surveys) with real-time sensor data to create a comprehensive environmental model. This merging allows the system to overcome the limited range and occlusion problems of individual on-board sensors by integrating data from diverse sources that cover different areas and perspectives.
Solution Approach 2:
The system performs preliminary actions by pre-generating 3D mesh maps of the environment before the vehicle reaches those areas. These pre-generated maps are stored and then used to augment real-time sensor data, allowing the system to have information about distant or occluded areas before the sensors can actually observe them, thus extending the effective coverage range.
2Productivity
If the system queries on-board sensors frequently to update AR content, then the AR display can reflect current environmental conditions, but this increases the computational load and response time
Solution Approach 1:
The system performs preliminary actions by pre-generating 3D mesh maps of the environment before the vehicle reaches those areas. These pre-generated maps are stored and then used to augment real-time sensor data, allowing the system to have information about distant or occluded areas before the sensors can actually observe them, thus extending the effective coverage range.
Solution Approach 2:
The system applies local quality by using pre-generated 3D mesh map data for distant and occluded areas where real-time sensor data is unavailable or incomplete, while using real-time sensor data for nearby areas where the sensors have good coverage. This localized approach optimizes the use of different data sources based on spatial requirements, reducing the need for frequent comprehensive sensor queries.
3Reliability
If the system uses pre-generated 3D mesh map data from multiple sources, then it can fill in occluded and distant areas beyond sensor range, but this increases the complexity of data integration
Solution Approach 1:
The system segments the environmental data into different spatial zones: areas with good sensor coverage, areas with partial coverage, and areas beyond sensor range. For each zone, it selectively uses appropriate data sources (real-time sensor data, pre-generated 3D mesh maps, or combinations), reducing the complexity of integrating all possible data sources across the entire environment.
Solution Approach 2:
The system uses an intermediary processing layer that automatically reconciles and integrates data from multiple sources (aerial photography, street-level imagery, LiDAR surveys, real-time sensors). This intermediary layer handles the complex tasks of data alignment, coordinate transformation, and conflict resolution, shielding the rest of the system from the intricacies of multi-source data integration.
Data Source
AI summary
An AR system that leverages a pre-generated 3D model of the world to improve rendering of 3D graphics content for AR views of a scene, for example an AR view of the world in front of a moving vehicle. By leveraging the pre-generated 3D model, the AR system may use a variety of techniques to enhance the rendering capabilities of the system. The AR system may obtain pre-generated 3D data (e.g., 3D tiles) from a remote source (e.g., cloud-based storage), and may use this pre-generated 3D data (e.g., a combination of 3D mesh, textures, and other geometry information) to augment local data (e.g., a point cloud of data collected by vehicle sensors) to determine much more information about a scene, including information about occluded or distant regions of the scene, than is available from the local data.