Vehicle AR Display Using 3D Mesh Maps for Occluded Scene Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time augmented reality systems in vehicles face challenges in rendering accurate and comprehensive environmental data due to limitations of onboard sensors, such as range, occlusions, and adverse weather conditions, which affect the quality and responsiveness of AR displays.
Innovation Solution
The use of pre-generated stereographic 3D reconstruction and mesh map data, which complements sensor information to provide a more extensive and accurate representation of the environment, allowing for the rendering of occluded and distant areas without relying on real-time sensor queries, and enhancing AR content visibility by adjusting lighting and terrain responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If onboard sensors (LiDAR, cameras) are used to provide real-time environmental data for AR display, then the system can detect objects and provide depth information, but the sensing range is limited and occluded areas cannot be detected
Solution Approach 1:
The system pre-generates 3D mesh map data and stereographic reconstructions of the environment before the vehicle reaches those areas. This preliminary action allows the AR system to access environmental information beyond the real-time sensor range, eliminating blind spots caused by limited sensor detection distance.
Solution Approach 2:
The system creates copies of environmental data in the form of 3D mesh models and stereographic reconstructions that can be stored and referenced. These digital copies provide complete environmental information including occluded areas, supplementing the limited real-time sensor data and enabling comprehensive AR rendering.
2Productivity
If real-time sensor queries are performed to update AR content, then the system can provide current environmental information, but the response time and quality are impacted by query optimization requirements
Solution Approach 1:
The system pre-processes and structures environmental data into optimized 3D mesh formats and stereographic projections before they are needed. This preliminary organization of data allows for rapid retrieval and rendering during real-time operation, reducing the time required for sensor queries and AR content updates.
Solution Approach 2:
The system divides the environmental data into localized 3D mesh segments and stereographic tiles that can be independently queried and rendered. This local organization allows the system to process only the relevant portions of the environment at any given time, improving query efficiency and reducing overall processing time.
3Loss of information
If pre-generated 3D mesh map data is used to extend AR beyond sensor range, then occluded and distant areas can be visualized, but the system requires integration of multiple data sources
Solution Approach 1:
The system merges real-time sensor data with pre-generated 3D mesh map data and stereographic reconstructions into a unified AR environment. This combination allows the system to leverage the strengths of each data source: real-time accuracy from sensors and comprehensive coverage from pre-generated models, including occluded and distant areas.
Solution Approach 2:
The system uses 3D mesh maps and stereographic reconstructions as intermediary representations that bridge the gap between real-time sensor data and the final AR display. These intermediaries provide a structured framework for integrating multiple data sources, managing the complexity of data fusion while enabling comprehensive environmental visualization.
4Reliability
If sensor data is used for AR rendering, then real-time accuracy is achieved, but the system fails in poor visibility conditions (fog, rain, night)
Solution Approach 1:
The system prepares backup environmental information in the form of pre-generated 3D mesh maps and stereographic reconstructions that can be used when real-time sensor data becomes unreliable. This beforehand preparation cushions against the harmful effects of poor visibility conditions, ensuring continuous AR functionality regardless of weather or lighting.
Solution Approach 2:
The system creates duplicate representations of the environment through pre-generated 3D models that can be used as fallback when real-time sensing fails. These copies preserve environmental geometry and features independent of current weather conditions, maintaining AR reliability during fog, rain, or nighttime operations.
Data Source
AI summary
An AR system that leverages a pre-generated 3D model of the world to improve rendering of 3D graphics content for AR views of a scene, for example an AR view of the world in front of a moving vehicle. By leveraging the pre-generated 3D model, the AR system may use a variety of techniques to enhance the rendering capabilities of the system. The AR system may obtain pre-generated 3D data (e.g., 3D tiles) from a remote source (e.g., cloud-based storage), and may use this pre-generated 3D data (e.g., a combination of 3D mesh, textures, and other geometry information) to augment local data (e.g., a point cloud of data collected by vehicle sensors) to determine much more information about a scene, including information about occluded or distant regions of the scene, than is available from the local data.