Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Full-automatic 2D video-to-3D video conversion method based on spatiotemporal information modeling

A conversion method, a fully automatic technology, applied in the field of conversion from 2D video to 3D video, to achieve the effect of reducing the amount of calculation, improving conversion efficiency, and improving conversion quality

Inactive Publication Date: 2020-02-07
NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In order to solve the problems in the 2D video to 3D video conversion algorithm in the prior art, the present invention proposes a fully automatic 2D video to 3D video conversion method based on spatio-temporal information modeling. This method uses spatio-temporal information as a video representation method , while improving the feature extraction model, improving the video conversion quality and conversion efficiency

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Full-automatic 2D video-to-3D video conversion method based on spatiotemporal information modeling
  • Full-automatic 2D video-to-3D video conversion method based on spatiotemporal information modeling
  • Full-automatic 2D video-to-3D video conversion method based on spatiotemporal information modeling

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] A method for converting fully automatic 2D video to 3D video based on spatio-temporal information modeling, comprising the following steps:

[0027] Step 1: Extract multiple video frames using the encoder network time information f t with spatial information f s , for the encoder network, we use a densely connected neural network and replace the 2D convolutions in the densely connected neural network with 3D convolutions;

[0028] 1.1 Densely connected neural network

[0029] Suppose the input is an image X 0 , after an L-layer neural network, the input of the jth layer of the densely connected neural network is not only related to the output of the j-1 layer, but also related to the output of all previous layers, denoted as:

[0030] x j =H j ([X 0 ,X 1 ,...,X j-1 ])

[0031] where: X j is the output of the i-th layer of the neural network, H j () is the nonlinear transformation of the jth layer of the neural network,

[0032] A densely connected neural n...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a full-automatic 2D video-to-3D video conversion method based on spatiotemporal information modeling, and the method comprises the steps: firstly extracting the spatial information of a 2D video through an encoder network in a neural network; extracting time information among multiple frames of the video, and taking space information and the time information as representation modes of the video; decoding the space information and the time information of the video into displacement information by utilizing a decoder network in the neural network; and combining the displacement information with the pixel information of the video frame by using a space converter to obtain a video frame of another view angle corresponding to the video frame; and finally, splicing the video frames of the two visual angles into a 3D video. The method is applied to conversion from the 2D video to the 3D video, and by adopting the technical scheme of the invention, the video conversionquality and conversion efficiency can be effectively improved.

Description

technical field [0001] The invention belongs to the technical field of video processing, and in particular relates to a fully automatic conversion of 2D video to 3D video using spatio-temporal information modeling. Background technique [0002] Existing methods for converting 2D video to 3D video are divided into two steps: 1) extracting a depth map from the input image; 2) generating stereoscopic image pairs using virtual view synthesis techniques. The extraction of the depth map can be divided into two categories: semi-automatic and fully automatic according to whether the operator participates in it. The semi-automatic method requires a lot of time and cost due to manual participation. The fully automatic method saves labor costs and greatly improves the conversion speed, but it cannot meet people's needs in terms of conversion quality. ; At the same time, due to the need for subsequent virtual viewpoint synthesis, the conversion efficiency of the video is limited. [0...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): H04N13/161H04N13/261H04N13/275
CPCH04N13/161H04N13/261H04N13/275
Inventor 陈蓓袁家斌包秀平
Owner NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com