No-reference video quality evaluation method based on deep spatio-temporal information

Abstract

The invention discloses a non-reference video quality evaluation method based on deep spatio-temporal information, and the method comprises the following steps: S1, extracting content perception features, extracting semantic layer features of a top layer through a Resnet-50 pre-trained deep neural network, carrying out the aggregation, and carrying out the mean aggregation and standard deviation aggregation of a feature map; S2, carrying out modeling on the time memory effect, wherein in the aspect of feature integration, modeling is carried out on the long-term dependency relationship by adopting a GRU network, and in the aspect of quality aggregation, a time pooling model of subjective inspiration is proposed and embedded into the network. An existing NR-VQA method cannot well model a long-term dependency relationship in a VQA task; in order to solve the problem, GRU which is a recurrent neural network model with gate control and can integrate features and learn the long-term dependency relationship is used, and the GRU is used for integrating a content awareness function and predicting a frame-by-frame quality score.

Description

technical field [0001] The invention relates to the technical field of image and video processing, in particular to a no-reference video quality evaluation method based on a deep convolutional network. Background technique [0002] With the popularization of wearable devices, smart phones and tablet computers with camera and video recording functions, the acquisition and storage of video information has become easier and easier. Greatly affect the audience's visual experience. In the entire video link, most modules can be accurately measured, such as capture, upload, preprocessing, transcoding, distribution, etc. However, the unknown part is precisely the most critical part, that is, how the user's video viewing experience is. At present, video quality assessment methods in the industry are divided into two categories: objective quality assessment and subjective quality assessment. The subjective evaluation method is evaluated by the human visual system, which is more acc...

AI Technical Summary

Problems solved by technology

As a result, the information obtained during the evaluation is not comprehensive enough

[0008] 2. For global timing information, only one-way GRU is considered

Therefore, the results obtained by simply considering the one-way GRU are inaccurate

[0009] 3. The window sliding in the temporal memory model does not consider the frame rate, and only uses a fixed window

Images