Video frame rate up-conversion system and method based on scene depth estimation

A technology of scene depth and video frame rate, which is applied in the field of video frame rate up-conversion system based on deep learning technology, can solve the problems of not obtaining the occlusion relationship of different objects, the inability to know the pixel brightness, exposure, and occlusion problems, etc. Achieve the effect of high system reliability, strong versatility, and good transformation effect

Inactive Publication Date: 2019-10-18
SHANGHAI JIAO TONG UNIV
View PDF5 Cites 26 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, a serious defect of such video interpolation methods based on motion estimation is that they are not capable of dealing with exposure and occlusion problems between different moving objects.
Since these methods do not obtain t

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Video frame rate up-conversion system and method based on scene depth estimation
  • Video frame rate up-conversion system and method based on scene depth estimation
  • Video frame rate up-conversion system and method based on scene depth estimation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] The present invention will be described in detail below in conjunction with specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0049] According to a kind of video frame rate up-conversion system based on scene depth estimation provided by the present invention, comprise following module:

[0050] Module 1: Optical flow estimation network;

[0051] Module 2: Scene Depth Estimation Network;

[0052] Module 3: Context Extraction Network;

[0053] Module 4: Interpolation Kernel Estimation Network;

[0054] Module 5: Optical flow field mapping for depth perception;

[0055] Module 6: Adaptive interpolation mapping;

[0056] Modul...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a video frame rate up-conversion system and method based on scene depth estimation. The system comprises a module 1: an optical flow estimation network; a module 2: a scene depth estimation network; a module 3: a context extraction network; a module 4: an nterpolating kernel estimation network; a module 5: an optical flow field mapping of depth perception; a module 6: a self-adaptive interpolation mapping; a module 7: an intermediate frame generation network. According to the method, scene depth estimation is utilized, and a solution for the exposure shielding problem in video frame rate up-conversion is provided; a neural network system capable of being trained end to end by utilizing a deep learning technology, so that the system can be trained and optimized on alarge amount of label-free video data.

Description

technical field [0001] The present invention relates to the technical field of video frame rate conversion, in particular to a video frame rate up-conversion system and method based on scene depth estimation, especially to a video frame rate up-conversion system and method based on deep learning technology. Background technique [0002] Video frame rate up-conversion is a technology for up-converting low frame rate videos such as 24fps and 30fps to high frame rate videos such as 48fps and 60fps. The technology is implemented by inserting a visually reasonable new image between every two frames of the original low frame rate video, thereby improving the viewing experience of the video. [0003] For example, a video frame interpolation processing method disclosed in patent document CN109640117A uses a Lagrange interpolation method to perform frame interpolation processing on a video. Read digital video, obtain video information such as duration, rate, total frame number, etc....

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04N7/01
CPCH04N7/0127H04N7/0135
Inventor 张小云包文博高志勇陈立
Owner SHANGHAI JIAO TONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap