Real scene three-dimensional semantic reconstruction method and device based on deep learning and storage medium

Abstract

The invention discloses a real scene three-dimensional semantic reconstruction method and device based on deep learning and a storage medium, relates to the technical field of remote sensing surveying and mapping geographic information, and solves the problem of inaccurate multi-scene labeling in the prior art. The method comprises: obtaining anaerial image; carrying out semantic segmentation on the aerial image, and determining a pixel probability distribution diagram; performing motion structure recovery on the aerial image, and determining a camera pose of the aerial image; performing depth estimation on the aerial image, and determining a depth map of the aerial image; and performing semantic fusion on the pixel probability distribution map, the camera pose and the depth map to determine a three-dimensional semantic model. Thus, high-precision segmentation is realized under the conditions of more scene objects, serious stacking and the like is realized; and in a large-scale scene, the performance of the depth estimation network is not affected, stable and accurate estimation can be carried out in various scenes, and compared with other traditional three-dimensional reconstruction algorithms, the semantic three-dimensional reconstruction algorithm constructed by the invention has the advantage that the calculation speed is increased.

Description

technical field [0001] This application relates to the technical field of remote sensing surveying and mapping geographic information, and in particular to a method, device and storage medium for real-scene 3D semantic reconstruction based on deep learning. Background technique [0002] 3D reconstruction and scene understanding is a research hotspot in computer field. 3D models with correct geometry and semantic segmentation are crucial in areas such as urban planning, autonomous driving, machine vision, and more. In urban scenes, semantic labels are used to visualize objects such as buildings, vegetation, and roads. 3D point clouds with semantic labels make 3D maps easier to understand and facilitate subsequent research and analysis. Although 3D semantic modeling has been extensively studied, different ways of extracting semantic information often lead to inconsistent or wrong results during point cloud reconstruction. Compared with two-dimensional images, semantic segme...

AI Technical Summary

Problems solved by technology

[0005] The embodiment of the present application solves the problem of inaccurate labeling of multiple scenes in the prior art by providing a deep learning-based real-scene 3D semantic reconstruction method, and realizes high-precision segmentation when there are many scene objects and serious stacking; and in In large-scale scenarios, the performance of the depth estimation network is not affected, and stable and accurate estimation can be achieved in various scenarios

Images