Monocular 3D reconstruction method with depth prediction

A depth prediction, single-purpose technology, applied in the field of 3D reconstruction, can solve the problems of lack of shape details in the reconstructed scene, difficult to train the neural network multi-view geometric basic principle, blurring, etc., to improve tracking and 3D reconstruction accuracy, maintain reconstruction accuracy, The effect of increasing the frame rate

Active Publication Date: 2021-08-13
NAT UNIV OF DEFENSE TECH
View PDF5 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, these methods also have their drawbacks, and it is difficult to train neural networks directly using the fundamentals of multi-view geometry
In addition, the depth predicted by the network will be partially blurred, resulting in a lack of shape details in the reconstructed scene

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Monocular 3D reconstruction method with depth prediction
  • Monocular 3D reconstruction method with depth prediction
  • Monocular 3D reconstruction method with depth prediction

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0076] A monocular 3D reconstruction method with depth prediction, comprising the following steps:

[0077] A. Use the monocular depth estimation network to obtain the depth map and rough pose estimation of the RGB image;

[0078] B. Combining the ICP algorithm and the PnP algorithm to calculate the camera pose estimation, perform loop closure detection at the local and global levels to ensure the consistency of the reconstruction model, and use uncertainty to refine the depth map to improve the reconstruction quality ;

[0079] C. Convert the depth map into a global model, and then insert the random fern code of the current frame into the database.

[0080] In step A, in the forward propagation stage, iterative optimization between subnetworks can produce accurate depth predictions. Then, we correct the depth map according to the camera parameters and transfer the result to the pose estimation module.

[0081] In step A,

[0082] Transform the RGB image into a depth image...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a monocular 3D reconstruction method with depth prediction. The monocular 3D reconstruction method comprises the following steps: A, obtaining a depth map and rough pose estimation of an RGB image by using a monocular depth estimation network; b, calculating camera pose estimation in combination with an ICP algorithm and a PnP algorithm, and executing loopback detection on a local level and a global level to ensure the consistency of a reconstruction model; and C, converting the depth map into a global model, and then inserting the random fern code of the current frame into a database. According to the invention, the defects in the prior art can be overcome, and large-scale high-quality three-dimensional reconstruction is realized.

Description

technical field [0001] The invention relates to the technical field of three-dimensional reconstruction, in particular to a monocular 3D reconstruction method with depth prediction. Background technique [0002] In recent years, many researchers have paid attention to indoor dense 3D reconstruction with detailed information. Simultaneous localization and mapping technology aims to solve navigation and mapping problems in unknown environments, and has been proven to be a feasible method for 3D reconstruction. With the release of the depth camera, many excellent SLAM methods have emerged such as: KinectFusion, InfiniTAM, ElasticFusion, RGB-D SLAM, etc. These methods can be widely used in autonomous driving, model building, augmented reality, etc. But the insufficiency of depth cameras imposes insurmountable limitations on these methods. First, the depth camera has a limited detection range and is very sensitive to lighting conditions, which leads to poor reconstruction accu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06T7/55
CPCG06T7/55
Inventor 陈颖文段志敏胡博文于鹄杰陈晨
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap