A real-time monocular video depth estimation method

A technology of depth estimation and video, applied in neural learning methods, calculations, computer components, etc., can solve problems such as limiting the practicality of depth estimation, and achieve the effect of promoting practicality, less model parameters, and promoting development

Active Publication Date: 2022-07-19
NORTHWESTERN POLYTECHNICAL UNIV
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

These problems limit the practicality of depth estimation on certain problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A real-time monocular video depth estimation method
  • A real-time monocular video depth estimation method
  • A real-time monocular video depth estimation method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] The present invention will now be further described in conjunction with the embodiments and accompanying drawings:

[0042] The technical scheme of the present invention is to combine a two-dimensional convolutional neural network (2D-CNN) and a convolutional long-short-term memory (CLSTM) network to construct a pair of A model for real-time deep depth estimation from monocular video data. At the same time, a generative adversarial network (GAN) is used to constrain the estimated results to satisfy temporal consistency.

[0043] The specific measures of this technical solution are as follows:

[0044] Step 1: Data preprocessing. Data preprocessing includes RGB video normalization, depth map normalization and sample extraction.

[0045] Step 2: Divide the training set and the validation set. A small number of samples are extracted as the validation set, and all the remaining samples are used as the training set.

[0046] Step 3: Build the network model. In the pres...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a real-time monocular video depth estimation method, which combines a two-dimensional convolutional neural network 2D-CNN and a convolutional long-short-term memory network to construct a real-time depth estimation method that can simultaneously utilize spatial and time sequence information for monocular video data. 's model. At the same time, the generated adversarial network GAN is used to constrain the estimated results. In terms of evaluation accuracy, it is comparable to the current state-of-the-art model. In terms of usage overhead, the model runs faster, has fewer model parameters, and requires less computing resources. And the results estimated by this model have good temporal consistency. When depth estimation is performed on consecutive multiple frames, the changes of the obtained depth result map are consistent with the changes of the input RGB map, and there will be no sudden change or jitter.

Description

technical field [0001] The invention relates to a real-time depth estimation method for each pixel of each frame in a monocular video, belonging to the field of video processing and three-dimensional reconstruction. Background technique [0002] Depth estimation takes the RGB image as input data and estimates the distance between each pixel position in the image and the camera position. According to whether the processing object is the image collected by multiple cameras for the same scene or the image collected by the monocular camera, it can be divided into multi-eye depth estimation and monocular depth estimation. Among them, monocular depth estimation is more challenging and applicable to a wider range. Monocular depth estimation can be applied to 3D modeling, scene understanding, and depth perception. [0003] In recent years, thanks to the development of deep learning technology and the increase of available labeled data for depth estimation, monocular depth estimati...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06T7/55G06N3/04G06N3/08
CPCG06T7/55G06T2207/10028G06T2207/20221G06N3/045G06F18/214
Inventor 李映张号逵李静玉
Owner NORTHWESTERN POLYTECHNICAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap