Stereoscopic video quality evaluation method based on binocular fusion network and two-step training framework

A technology that integrates network and stereoscopic video, applied in stereoscopic systems, neural learning methods, biological neural network models, etc., can solve problems such as difficult to accurately evaluate asymmetric distortion and unreasonableness, achieve excellent accuracy and reliability, and improve performance Effect

Inactive Publication Date: 2021-03-02
TIANJIN UNIV
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

But for asymmetric distortion, it is unreasonable to use the same label to label different views and different regions of the sam

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Stereoscopic video quality evaluation method based on binocular fusion network and two-step training framework
  • Stereoscopic video quality evaluation method based on binocular fusion network and two-step training framework

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] In order to solve the problems existing in the prior art, this patent embeds the "weighting module" into the fusion network to imitate binocular competition as much as possible, and adopts a two-step training framework. In the first step, the quality scores of the blocks are generated by the FSIM algorithm [17] and used as labels to regress the local network. In the second step, based on the weight model of the first step, the MOS value is used for global regression.

[0023] In order to reflect the temporal and spatial correlation of the video, the present invention selects the spatiotemporal saliency feature stream as the input of the binocular fusion network. The theory that time is not independent of each other is consistent. Because changes in spatial pixels provide motion information and attention mechanisms to the temporal domain, and in turn, the temporal flow reflects the spatial saliency in the video.

[0024] Therefore, the main contributions of the present i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the field of video and image processing, and provides a stereoscopic video quality evaluation method which is more accurate and efficient and fits the perception of human eyeson stereoscopic vision, and the stereoscopic video quality evaluation method based on a binocular fusion network and a two-step training framework comprises the following steps: firstly, calculatingthe time significance and the space significance of a stereoscopic video; generating saliency pictures for each frame of the three-dimensional video, and calling the obtained saliency pictures arranged in sequence as a space-time saliency characteristic flow of the left video and a space-time saliency characteristic flow of the right video and serve as two inputs of a binocular fusion network; secondly, training a binocular fusion network in two steps, i.e., local regression and global regression, and in the first step, pre-training a left channel CNN and a right channel CNN of the proposed network by adding a full connection layer and using small tags; and in the second step, based on the weight model in the first step, using the MOS value to train the whole network. The method is mainlyapplied to video and image processing occasions.

Description

technical field [0001] The invention belongs to the field of video and image processing, and relates to the calculation of video spatiotemporal salience, the calculation of quality scores of different distorted blocks, and the application of deep learning in stereoscopic video quality evaluation. It specifically involves a stereoscopic video quality evaluation method based on a binocular fusion network and a two-step training framework. Background technique [0002] At present, stereoscopic video has been widely used in various fields of human life. At the same time, a series of stereoscopic video processing techniques have also been produced. However, any processing technology may cause varying degrees of distortion to stereoscopic content, thereby affecting people's perception of stereoscopic video. Therefore, it is very necessary to find an effective stereoscopic video quality evaluation method. An effective stereoscopic video quality evaluation method can not only mea...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04N17/00H04N13/106G06N3/08G06N3/04G06K9/62
CPCH04N17/004H04N13/106G06N3/084G06N3/045G06F18/213G06F18/253
Inventor 李素梅刘安琪马帅
Owner TIANJIN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap