Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Real-time monocular video depth estimation method

A technology of depth estimation and video, which is applied in computing, computer components, image analysis, etc., can solve the problems that limit the practicality of depth estimation, and achieve the effect of promoting practicality, good time consistency, and fast operation speed

Active Publication Date: 2019-09-17
NORTHWESTERN POLYTECHNICAL UNIV
View PDF8 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

These problems limit the practicality of depth estimation on certain problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time monocular video depth estimation method
  • Real-time monocular video depth estimation method
  • Real-time monocular video depth estimation method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] Now in conjunction with embodiment, accompanying drawing, the present invention will be further described:

[0042] The technical solution of the present invention is to combine two-dimensional convolutional neural network (2D-CNN) and convolutional long short-term memory network (convolutional long short-termmemory, CLSTM), construct a pair of spatial and temporal information that can simultaneously use A model for real-time deep depth estimation from monocular video data. At the same time, a generative adversarial network (GAN) is used to constrain the estimated results so that they meet time consistency.

[0043] The concrete measures of this technical scheme are as follows:

[0044] Step 1: Data preprocessing. Data preprocessing includes RGB video normalization, depth map normalization and sample extraction.

[0045] Step 2: Divide the training set and validation set. A small number of samples are extracted as a validation set, and all remaining samples are used...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a real-time monocular video depth estimation method, which combines a 2D-CNN (Two-Dimensional Convolutional Neural Network) and a convolutional long-short term memory network to construct a model capable of performing real-time depth estimation on monocular video data by utilizing space and time sequence information at the same time. A GAN (Generative Adversarial Network) is used for constraining an estimated result. In terms of evaluation precision, the method can be compared with a current state-of-th-art model. In the aspect of use overhead, the model operation speed is higher, the model parameter quantity is smaller, and fewer computing resources are needed. The result estimated by the model has good time consistency, and when depth estimation is carried out on multiple continuous frames, the change condition of the obtained depth result image is consistent with the change condition of the input RGB image, so that sudden change and jitter are avoided.

Description

technical field [0001] The invention relates to a method for real-time depth estimation of each pixel of each frame in a monocular video, belonging to the field of video processing and three-dimensional reconstruction. Background technique [0002] Depth estimation, using RGB images as input data, estimates the distance between each pixel position in the image and the camera position. According to whether the object to be processed is multiple cameras collecting images of the same scene or images collected by a monocular camera, it can be divided into multi-camera depth estimation and monocular depth estimation. Among them, monocular depth estimation is more challenging and applicable to a wider range. Monocular depth estimation can be applied in fields such as 3D modeling, scene understanding, and depth perception. [0003] In recent years, thanks to the development of deep learning technology and the increase in the available labeled data for depth estimation, monocular ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06T7/55G06K9/62G06N3/04
CPCG06T7/55G06T2207/10028G06T2207/20221G06N3/045G06F18/214
Inventor 李映张号逵李静玉
Owner NORTHWESTERN POLYTECHNICAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com