Mixed Reality Route Guidance with Vehicle-Building Texture Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D maps used in mixed reality route guidance systems lack accurate building textures due to difficulties in capturing lateral building surfaces from satellite or aerial photos, resulting in a mismatch between digital twin maps and the real world, which affects the accuracy of route guidance information provided to drivers.
Innovation Solution
A system that includes a display device and route guide system capable of extracting building images from camera footage, generating texture images for building surfaces, and transmitting these to a cloud server for updating digital twin maps, using a combination of point cloud technology and artificial intelligence algorithms like Deep Neural Network (DNN)-Generative Adversarial Network (GAN) to synthesize accurate building textures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If satellite or aerial photos are used for building modeling, then 3D mapping can be provided, but accurate textures of lateral building surfaces cannot be obtained
Solution Approach 1:
The patent uses images captured by cameras mounted on vehicles to create copies of building lateral surfaces. Instead of attempting to capture all building surfaces directly through satellite or aerial photography, the system captures images from ground-level vehicle perspectives and uses these copies to generate accurate textures for the digital twin map's building models.
Solution Approach 2:
The patent introduces vehicle-mounted cameras as an intermediary means to obtain building lateral surface images. These cameras serve as a mediator between the building surfaces and the digital twin map creation process, enabling acquisition of lateral surface textures that satellite or aerial photos cannot capture.
2Manufacturing precision
If polygon data with footprint and height is used for 3D mapping, then building location and volume can be shown, but accurate building textures are missing
Solution Approach 1:
The patent segments the building texture acquisition process into distinct components: image capture by vehicle cameras, image processing to extract building lateral surfaces, and texture mapping to 3D building models. This segmentation allows each component to be optimized independently, improving overall texture accuracy while managing processing complexity.
Solution Approach 2:
The patent transitions from 2D satellite/aerial photos to 3D building models with accurate lateral surface textures by incorporating ground-level vehicle camera perspectives. This dimensional change enables capture of lateral surfaces that are invisible from top-down views, adding texture detail to the previously geometry-only polygon data.
3Measurement precision
If digital twin map is updated with accurate building textures, then matching rate with real world increases, but data collection and processing time increases
Solution Approach 1:
The patent performs preliminary actions by capturing building images during normal vehicle operation and pre-processing these images to extract building lateral surfaces. By preparing texture images in advance and storing them locally, the system reduces the time required for real-time digital twin map updates while maintaining high matching accuracy.
Solution Approach 2:
The patent implements a feedback mechanism where the digital twin map is continuously updated with newly captured building textures, and the matching rate between the digital twin and real world is monitored. This feedback loop allows the system to prioritize updating buildings that show larger discrepancies, optimizing the balance between accuracy improvement and time consumption.
Data Source
AI summary
The present invention relates to a display device comprising: a communication unit for communicating with a cloud server; an interface unit for obtaining an image of the periphery of a vehicle by means of a camera, and receiving sensing information collected from at least one sensor; an MR module for rendering MR information, comprising a virtual object, on the basis of the sensing information and map information received from the cloud server; and a processor extracting an image, corresponding to a lateral surface of a building around the vehicle, from the obtained image, generating a texture image corresponding to the lateral surface of the building from the extracted image and transmitting same to the cloud server, receiving, from the cloud server, map information comprising models of buildings to which building textures on the basis of texture images are mapped, and displaying MR information rendered on the basis of the map information on a display provided in the vehicle.