Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

2D-to-3D conversion method for video

A conversion method and video technology, applied in machine learning, instrumentation, computing, etc., can solve the problems of large black hole areas that cannot be filled, inaccurate depth estimation, and slow three-dimensional conversion speed, etc., to achieve excellent rendering effects, improve detailed textures, reduce The effect of object deformation

Active Publication Date: 2020-12-01
上海网达软件股份有限公司
View PDF4 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The 2D / 3D conversion method of the present invention solves the problems of inaccurate depth estimation in complex scenes, resulting in jittering pictures and subtitles, deformation and distortion of objects, etc.
Fixed an issue where large black hole regions formed by source image distortions could not be filled
Solved the problem of slow three-dimensional conversion of images with 2K resolution and above

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • 2D-to-3D conversion method for video
  • 2D-to-3D conversion method for video
  • 2D-to-3D conversion method for video

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0064] Such as figure 1 As shown, this embodiment provides a specific 2D-to-3D video conversion method, using a deep learning model to replace the depth map estimation algorithm and black hole filling algorithm in the traditional DIBR method, using the high generalization and high precision of the deep learning model, etc. characteristics, making up for the deficiencies in traditional algorithms. Based on the Tensorflow framework, an efficient and high-quality conversion technology is realized.

[0065] The invention is a 2D / 3D conversion method aimed at 2K resolution and above. Based on the Tensorflow open source framework, the CUDA engine is used to accelerate parallel computing, and the 2D / 3D conversion on the ultra-high resolution image is realized. The original image and mask are sent to the depth estimation model to generate the left and right projection images, such as figure 2 As shown, the depth image containing the black hole is repaired into a complete image as ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of video dimension conversion, and provides a 2D-to-3D conversion method for a video, and the method comprises the steps: S1, collecting an open-source RGB-D image data set, carrying out the expansion, forming a depth estimation data set, constructing a depth estimation model through the depth estimation data set, and training the depth estimation model; S2, collecting 4K high-definition pictures to make an image restoration data set, expanding the image restoration data set, constructing an image restoration model through the image restoration dataset, and training the image restoration model; and S3, extracting an original image mask by using the pre-trained Mask-RCNN model, adjusting the resolution of the original image and the mask, sendingthe original image and the mask to a depth estimation model, calculating original left and right projection images according to the depth image, and sending the left and right projection images to animage restoration model to restore a black hole area. A deep learning algorithm and a traditional algorithm are combined, a deep learning model is used for replacing a depth map estimation algorithmand a black hole filling algorithm in a traditional DIBR method, and 2D / 3D conversion on an ultrahigh-resolution image is achieved.

Description

technical field [0001] The present invention relates to the technical field of video dimension conversion, in particular to a method for converting video from 2D to 3D, including processing methods such as computer image processing, computer vision, deep learning, and CUDA high-performance programming. Background technique [0002] At present, high-speed, large-capacity, and low-latency 5G communications are gradually becoming popular, making it possible for the Internet of Everything. Compared with 2D video, 3D video is rich in scene depth information. The parallax composite image generated by 3D algorithm conversion conforms to the 3D stereoscopic perception of human eyes in the real world, and immersive experience can be obtained with the help of VR equipment. [0003] The current 2D / 3D conversion methods are mainly divided into end-to-end methods based on deep learning and traditional methods based on DIBR. [0004] (1) End-to-end approach to deep learning [0005] At ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): H04N13/111H04N13/128G06N20/00
CPCH04N13/111H04N13/128G06N20/00
Inventor 唐杰李进李庆瑜戴立言
Owner 上海网达软件股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com