MR Route Guidance Using AI-Built Building Side Textures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D maps lack accurate building textures due to difficulties in capturing images of building side surfaces from satellite or aerial photos, leading to a mismatch between digital twin maps and the real world, which affects the accuracy of mixed reality-based route guidance.
Innovation Solution
A display device and route guide system that uses a camera to capture building side surfaces, generates texture images, and updates digital twin maps through a cloud server, synthesizing textures from multiple angles and directions using AI algorithms to enhance map accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If satellite or aerial photos are used for building modeling, then 3D mapping can be provided, but accurate textures of building side surfaces cannot be obtained
Solution Approach 1:
The patent transitions from top-down 2D satellite/aerial photography to multi-angle 3D photography by positioning cameras at ground level around buildings. This dimensional change enables capture of building side surfaces that were previously invisible, allowing accurate texture mapping of all building faces including lateral surfaces.
Solution Approach 2:
The patent introduces an intermediary processing system that collects images from multiple ground-based cameras, automatically identifies building side surfaces, generates texture maps, and integrates them with 3D building models. This intermediary process bridges the gap between raw multi-angle images and accurate building textures.
2Quantity of substance
If polygon data with footprint and height is used for 3D mapping, then building location and volume can be shown, but accurate building shapes and textures are lost
Solution Approach 1:
The patent merges two previously separate data streams: the 3D geometric information (footprint and height from polygon data) and the detailed texture information (from multi-angle ground-based photography). By combining these, the system creates comprehensive building models that retain both location/volume accuracy and detailed surface textures.
Solution Approach 2:
The patent performs preliminary actions by first establishing accurate 3D building models with correct geometry from existing polygon data, then subsequently adding texture information through ground-based photography. This sequential approach ensures that the foundation (geometric accuracy) is established before adding detailed surfaces.
3Productivity
If 3D maps with polygon modeling are provided, then building location and volume are displayed, but the matching rate with real world buildings is reduced
Solution Approach 1:
The patent implements a self-service system where the mapping infrastructure uses itself to improve accuracy. Ground-based cameras positioned along roads and pathways automatically capture building images, which are then processed to generate textures that enhance the existing 3D models. This self-enhancing process maintains productivity while improving reliability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to a display device comprising: a communication unit for communicating with a cloud server; an interface unit for obtaining an image of the periphery of a vehicle by means of a camera, and receiving sensing information collected from at least one sensor; an MR module for rendering MR information, comprising a virtual object, on the basis of the sensing information and map information received from the cloud server; and a processor extracting an image, corresponding to a lateral surface of a building around the vehicle, from the obtained image, generating a texture image corresponding to the lateral surface of the building from the extracted image and transmitting same to the cloud server, receiving, from the cloud server, map information comprising models of buildings to which building textures on the basis of texture images are mapped, and displaying MR information rendered on the basis of the map information on a display provided in the vehicle.