Blind video quality evaluation method based on space-time feature analyzer

Abstract

The invention provides a blind view screen quality evaluation method based on a space-time feature analyzer. Feature extraction serves as an important component of the BVQA and has an important influence on the performance and speed of the BVQA. In the process of designing the BVQA method, the feature extraction process is independent of the whole quality evaluation method. The method comprises the following steps: firstly, obtaining a spatial-temporal feature analyzer (STFR) through training; then, the STFR is utilized to directly extract the spatio-temporal characteristics of the video sequence; finally, the extracted features are mapped to a quality score using support vector regression (SVR). The STFR only needs to be trained in an undistorted video sequence, can be directly applied to video sequences of various scenes, and has universality. Experimental results show that compared with other BVQA methods, the method provided by the invention not only obtains higher precision than the existing BVQA method, but also has remarkable competitiveness in calculation speed.

Description

Technical Field

[0001] The present invention relates to the technical problem of blind video quality evaluation in the field of video processing, and in particular to designing a spatio-temporal feature resolver (STFR) for extracting spatio-temporal features of a video, and then extracting spatio-temporal features of the video based on STFR to achieve blind video quality evaluation. Background Art

[0002] With the rapid development of Internet technology and the widespread use of multimedia devices, digital multimedia information is becoming more and more common in our daily lives. As an important branch of digital media, video contains more appropriate and comprehensive information than other forms of digital media. However, video information undergoes lossy compression during processing and is also susceptible to various distortions during storage, transmission, and reproduction. These distortions will have an adverse effect on video quality. Therefore, video quality assessment (VQA) is crucial in the field of video processing.

[0003] Existing VQA methods are mainly divided into subjective VQA and objective VQA. Subjective VQA methods directly score video content based on human subjective perception. Although subjective VQA is accurate and reliable, it is time-consuming and laborious. Therefore, in the study of video processing, it is very necessary to design an objective VQA method that conforms to the characteristics of human visual perception.

[0004] According to the degree of utilization of reference information by VQA methods, objective VQA can be divided into full-reference VQA (Full-Reference VQA), reduced-reference VQA (Reduced-Reference VQA), no-reference VQA (NR-VQA) or blind reference VQA (Blind VQA). Since perfect videos cannot be obtained in real scenes, and unknown distortions will be generated in video processing stages such as video shooting and compression, FR-VQA and RR-VQA methods are not applicable. In contrast, BVQA does not rely on reference video information and has more practical research significance and value.

[0005] The performance of the BVQA method depends largely on the effective extraction of video features, and the use of effective features is conducive to improving performance. In the early days of the BVQA method, video frames were regarded as independent images, and the blind image quality assessment (BIQA) method was used to calculate the quality score of a single video frame. Finally, all video frames were statistically calculated to obtain the final BVQA result. BVQA and BIQA have both similarities and differences. Unlike BIQA, which only considers spatial information, BVQA needs to consider both spatial and temporal information, because temporal information is unique and critical to video content. Simply applying the BIQA method to BVQA without correctly extracting temporal features may have an adverse effect on the quality assessment results of the video.

[0006] In order to further improve the performance of the BVQA method, the spatial features extracted from the BIQA method are combined with the temporal features manually extracted from the video sequence. Although the manual extraction of spatial and temporal features has been explored and designed in various scenarios, these BVQA methods have poor generalization capabilities and limitations due to the diversity and complexity of video information content.

[0007] In recent years, deep learning-based BVQA methods have gained popularity and have achieved significant performance improvements over traditional BVQA methods. However, these deep learning-based BVQA methods require large-scale labeled datasets and a lot of training time. Some of these methods leverage weak labels from FR-VQA or RR-VQA to augment large-scale labeled data, and others speed up computation by designing lightweight networks. In addition, these methods may have biases and deficiencies. This can be a challenge, especially when BVQA is used as a performance evaluation method in other video processing applications, as the computational complexity and time-consuming nature may affect widespread adoption. Summary of the invention

[0008] In view of the problem that there is still room for improvement in feature extraction of blind video quality evaluation and the calculation accuracy and speed can be further improved, the present invention aims to propose a blind video quality evaluation method based on a space-time feature parser.

[0009] The basic idea of ​​the present invention is to effectively extract spatial and temporal features from video sequences of different scenes and accurately calculate the video quality score. A feature extractor is designed to extract video features, and the generalization of the blind video quality evaluation method based on traditional manual feature extraction and the calculation speed of the blind video quality evaluation method based on deep learning are solved. The method mainly includes two steps, namely:

[0010] Step 1: Propose a space-time feature parser. This model can parse the space-time features of the video by training a distortion-free video, complete the rapid extraction of video features, and help speed up the calculation of the blind video quality evaluation method.

[0011] Step 2: A blind video quality assessment algorithm based on a spatio-temporal feature parser is proposed. By preprocessing the video, the scale of the input video sequence and the size of the spatio-temporal feature extractor are kept consistent. The spatio-temporal feature extractor is used to complete the feature extraction of the video sequence, and finally the regression model is used to realize the mapping of features to quality scores.

[0012] Step 3: Take the video sequence as input, and sequentially pass it through the trained spatial-temporal feature parser and regression network model to finally obtain the quality score of the video;

[0013] The specific process is as follows:

[0014] (1) The parser for extracting space-time features of the present invention is as shown in the attached figure. Figure 2 As shown, first, in order to reduce the complexity of feature extraction and speed up the extraction of spatiotemporal features, we trained a set of visual models for extracting spatiotemporal features of videos, called STFR;

[0015] (2) The blind video quality evaluation method designed by the present invention consists of preprocessing and a feature extraction regression model based on STFR. Figure 1 As shown in the figure, first, the video sequence is preprocessed to keep the scale of the video sequence and the size of the spatiotemporal feature extractor consistent; then STFR is applied to each frame of the video to calculate the spatiotemporal features of the entire video sequence by analyzing the similarities and differences between video frames. Finally, the regression model is used to map the spatiotemporal features to the quality score;

[0016] In process (1), the design of STFR requires a training process. At the beginning of the training process, a distortion-free video sequence is randomly selected from the video library as a sample source, and the YUV video is converted into an RGB video signal. Then, each frame of the video sequence is divided into video blocks of size 16×16×3, and 10,000 video blocks are randomly selected. Finally, these video blocks together form a matrix, denoted as M, with a size of 768×10,000;

[0017] Then, the covariance matrix of M is decomposed into eigenvalues ​​and eigenvectors using Zero-phase Component Analysis (ZCA). The sample data is whitened to complete the preprocessing. ZCA not only reduces the redundancy of the input, but also maps the whitened data back to the source space closer to the original data. The whitening matrix is ​​given by:

[0018]

[0019] Where D represents the eigenvalue of the covariance matrix, V represents the eigenvector, and V T represents the transpose of the eigenvector.

[0020] The whitened data Z is used for iterative training of STFR. A new iteration of the model starts with a random matrix and orthogonalizes the matrix:

[0021]

[0022] Then, the independent component analysis (ICA) algorithm is used to separate the data into statistically independent non-Gaussian linear combinations. The basic iterative form of the ICA algorithm in the non-Gaussian direction is as follows:

[0023]

[0024] Where g(·) is the hyperbolic tangent function and g'(·) is the derivative of g(·).

[0025] Each iteration involves updating the STFR ω And make them orthogonal. This process continues by repeating formulas (2)-(3) until the final result converges. The convergence criterion is as follows:

[0026]

[0027] where ||·|| represents the Frobenius norm and I represents the identity matrix.

[0028] Finally, the resulting matrix is ​​converted from the whitened space to the original video space to obtain STFR, which has a size of 768 × 768. Each row vector in STFR represents a matrix block of size 16 × 16 × 3.

[0029] In order to extract the spatiotemporal features of the video sequence more quickly and conveniently in process (2), STFR is directly applied to the input video sequence. This enables comprehensive spatiotemporal feature analysis and extracts them directly from the video sequence.

[0030] First, for the input video sequence F = (f 1 ,f 2 ,…,f M ) for preprocessing. The YUV video is converted to RGB video signal, and the average value of each frame is removed and divided into 16×16×3 blocks.

[0031]

[0032] Where j∈(1,2,…,N), N represents the total number of blocks in the current video frame, i∈(1,2,…,M), M represents the total number of video frames. F_j^i represents the jth block of the i-th video frame, and μ() represents the average value of the current video frame. In order to facilitate calculation and stretch each video block into a column vector, the current video frame is represented as B i The size is 768×N.

[0033] Then, STFR is directly applied to each frame of the video sequence, and the spatiotemporal feature information is extracted from the video in the form of the product of STFR and the video sequence.

[0034] f_stfr i ′=B i ×STFR (6) where f_stfr i ′ represents the visual perception features of the current video frame obtained by STFR, f_stfr i The size of ′ is 768×N.

[0035] By outputting the signal f_stfr i ′, calculate f_stfr i The row standard deviation of ′ is f_stfr i .f_stfr i Represents the video frame f i A feature of size 768×1.

[0036] Finally, the space-time feature information F_STFR of the current video sequence F is expressed as:

[0037] F_STFR=(f_stfr 1 ,f_stfr 2 ,…,f_stfr M ) (7)

[0038] The size of M represents the total number of video frames. F_STFR represents the spatial-temporal feature information of the video sequence F, and its size is 768×M.

[0039] As attached Figure 3 As shown in FIG. 1 , based on the video F_STFR obtained by STFR, a line graph of two different distorted video sequences is given. Figure 3 As shown in Figure .a, when the content between adjacent video frames does not change dramatically, the distribution of the line graphs of adjacent video frames is similar. However, when one of the adjacent video frames is severely distorted, the distribution of the distortion line graph will change dramatically, but the overall distribution remains the same, as shown in the attached figure. Figure 3To provide a more detailed explanation, we selected five adjacent video frames and extracted 20 f_stfr from each video frame. i At the same time, in order to observe the distortion changes of the curve, the third frame is synthesized by manually adding four distortion blocks.

[0040] Based on this observation, the f_stfr of the adjacent video i The features are similar in spatial feature information. Therefore, the row mean of F_STFR is calculated to represent the spatial features of the entire video sequence.

[0041]

[0042] The size of M represents the total number of video frames. f_m is the average number of rows in F_STER, and its size is 768×1.

[0043] At the same time, HVS has a time lag effect. According to the feature differences between adjacent video frames, the frame with the largest difference is identified as the representative frame and used as the feature of the video sequence.

[0044] f_r=f_stfr r (9)

[0045] where f_stfr r The representative frame with the largest difference from f_m in the video sequence is selected as the representative frame.

[0046] In the temporal feature information, when there is distortion in a certain frame of the video, there will be a big difference. Korhonoe et al. also mentioned that the vector obtained by the standard error is the video consistency temporal feature vector. The consistency feature effectively captures the motion consistency of the video, which is a significant feature of the video information and an important component of the temporal information. Therefore, the row standard deviation of F_STFR is calculated as the representative temporal feature of the entire video sequence.

[0047]

[0048] where i∈(1,2,…,768), i is the number of rows in F_STFR, μ() represents the mean of the i-th row in F_STFR. f_s is the standard deviation of the rows in F_STFR, and its size is 768×1.

[0049] Therefore, based on F_STFR, f_m, f_r and f_s are taken as the space-time feature vectors of the entire video sequence.

[0050] f all =f_m+f_r+f_s (11)

[0051] In the existing BIQA and BVQA methods, commonly used regression models include back propagation neural network (NN), random forest regression (RFR) and support vector regression (SVR). These regression models use the spatial and temporal features of video sequences as input and the corresponding Mean Opinion Score (MOS) as model output to train model parameters;

[0052] (3) In the training phase, the present invention first uses the trained space-time feature parser and regression model parameters. Then, the video sequence is used as the input of the quality evaluation framework, and then the space-time feature parser is used to obtain the video features. Finally, the regression model is used to realize the mapping of video features to quality scores, and the quality score of the video is obtained.

[0053] The advantages and beneficial technical effects of the present invention compared with the prior art are as follows:

[0054] (1) The present invention proposes a STFR algorithm for automatically extracting video spatiotemporal features. STFR is not only universal, but also can be obtained with only one video sequence. At the same time, using STFR can greatly save the time of extracting video spatiotemporal features;

[0055] (2) The present invention proposes a blind video quality assessment method based on STFR, which uses STFR as an independent model to extract video spatial-temporal features. This method fully considers the impact of similarity and difference of video features on video perception quality;

[0056] (3) The present invention proposes a blind video quality assessment method based on STFR. Compared with the existing blind video quality assessment method, the method proposed by us is superior to the existing method in terms of performance and calculation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Diagram of the training process of spatial-temporal feature parsing features.

[0058] Figure 2 Flowchart of the adaptive template matching algorithm.

[0059] Figure 3 The F_STFR value curve in the video sequence. (a) The F_STFR similarity curve of five adjacent frames. (b) The F_STFR change curve of five frames. DETAILED DESCRIPTION

[0060] The present invention is further described in detail below in conjunction with the embodiments. It is necessary to point out that the following embodiments are only used to further illustrate the present invention and cannot be understood as limiting the scope of protection of the present invention. Technical personnel familiar with the relevant field may make some non-essential improvements and adjustments to the present invention according to the above invention content for specific implementation, which should still fall within the scope of protection of the present invention.

[0061] (1) The proposed method was verified on four databases: LIVE-Qualcomm Mobile In-Capture Video Quality Database (LIVE-Q), LIVE Video Quality Challenge Database (LIVE-VQC), Konstanz Natural Video Database (KoNViD-1K), and the resulting Compressed Video quality database with Semi-Automatic Ratings (CVSAR);

[0062] (2) To provide quantitative performance evaluation for the target VQA, the Spearman rank correlation coefficient (SRCC) and Pearson linear correlation coefficient (PLCC) are used as two commonly used standards for evaluating the performance of objective VQA methods. Therefore, the experiment also uses these two standards for verification. SRCC represents the monotonic relationship between MOS and the predicted score, and PLCC represents the correlation between MOS and the predicted score. The closer the absolute values ​​of SRCC and PLCC are to 1, the better the linear rank correlation of the model and the higher the accuracy.

[0063] (3) Calculating the performance under the above conditions using the method of the present invention;

[0064] (4) Table 1 shows the median of 10 SRCC and PLCC results on LIVE-Q, LIVE-VQC, KoNViD-1K and CVSAR databases. As shown in Table 1, the present invention performs well in all databases. The SRCC results of the present invention rank in the top two of all BVQA methods on LIVE-VQC, KoNViD-1K and CVSAR, and the SRCC results on LIVE-Q rank third. In addition, other regression models (NN and RFR) are used instead of SVR as the regression model. It can be seen from Table 1 that the present invention selects SVR with slightly better performance as the regression model, but the performance of the two is not much different, which proves that the versatility of the model of the present invention in feature extraction has been verified;

[0065] (5) Table 2 shows the generalization ability of different BVQA methods. First, training was performed on the KoNViD-1K database, and then evaluation was performed on the LIVE-Q, LIVE-Q and CVSAR databases. As shown in Table 2, the present invention achieved the top three performance among all BVQA methods. Due to the diversity of video content and quality levels in subjective experiments, the BVQA method based on manual feature extraction lacks strong generalization ability. On the other hand, the BVQA method based on deep learning requires a lot of training on large-scale datasets to effectively extract quality perception features using deep models. However, the feature extraction process of the present invention will not affect the judgment of video quality as the video content changes. Therefore, the present invention can be applied to a wider range of video content and distortion types. Moreover, during the evaluation process, it is only necessary to ensure that the block size of the test video sequence is consistent with the block size of the STFR training process. Therefore, STFR does not need to be retrained during the cross-database evaluation process, nor does it need to adjust the video sequence to the same size. This highlights the convenience and versatility of the present invention;

[0066] (6) Table 3 shows the computational efficiency of the present invention compared with other BVQA methods. By running the algorithms on the same computer and then comparing the execution time, we can roughly understand the complexity differences between them. The present invention selected two video sequences with different resolutions from the CVSAR database, with resolutions of 320×240 and 1920×1080 respectively. The length and frame rate of the video sequences are set to the same size, 10 and 30fps respectively. It can be seen from Table 3 that compared with other BVQA methods, the present invention has the fastest calculation speed in video sequences with different resolutions. Moreover, the higher the resolution of the video sequence, the more obvious the calculation speed;

[0067] (7) Table 4 further explores the performance results and calculation time of different video segment lengths (video segments are 10 frames, 30 frames, 60 frames, 100 frames, 150 frames, and 300 frames). These video segments of different lengths are selected from the same video sequence. As shown in Table 4, as the number of frames in the video segment increases, the results of the SRCC and PLCC of the present invention are getting better and better. At the same time, longer video segments increase the calculation time;

[0068] (8) Table 5 A series of ablation experiments are conducted to evaluate the contribution of key components in the present invention. The present invention contains three features: f_m, f_r and f_s. Table 5 lists the SRCC results of the feature subset of the present invention on the KoNViD-1K database. We first select f_m (f_r or f_s) as an independent feature for the experiment. The present invention first selects f_m (f_r or f_s) as an independent feature for the experiment. The performance is not satisfactory and hardly reflects significant improvement. These results show that for complex VQA tasks, it is not enough to rely solely on spatial information, temporal information or special video frame information. In addition, when two of the three features are combined, the performance is relatively poor compared to the combination of all features. This highlights the importance of combining all three features in achieving the best overall performance. In summary, all features in the present invention contribute to its overall performance, and the fusion of these features produces the best results.

[0069] The comparison method of the present invention is:

[0070] Method 1: The method proposed by Mittal et al., reference “A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-Reference Image Quality Assessment in the Spatial Domain,” IEEE Trans. on Image Process., vol. 21, no. 12, pp. 4695–4708, Dec. 2012, doi: 10.1109 / TIP.2012.2214050.”

[0071] Method 2: The method proposed by Mittal et al., reference “A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a 'Completely Blind' Image Quality Analyzer,” IEEE Signal Process. Lett., vol. 20, no. 3, pp. 209–212, Mar. 2013, doi: 10.1109 / LSP.2012.2227726.”

[0072] Method 3: The method proposed by Saad et al., reference “MASaad, A.C.Bovik, and C.Charrier, “Blind Prediction of Natural Video Quality,” IEEE Trans. on Image Process., vol. 23, no. 3, pp. 1352–1365, Mar. 2014, doi: 10.1109 / TIP.2014.2299154.”

[0073] Method 4: The method proposed by Mittal et al., reference “A. Mittal, M. A. Saad, and A. C. Bovik, “A Completely Blind Video Integrity Oracle,” IEEE Trans. on Image Process., vol. 25, no. 1, pp. 289–300, Jan. 2016, doi: 10.1109 / TIP.2015.2502725.”

[0074] Method 5: The method proposed by Korhonen et al., reference “J. Korhonen, “Two-Level Approachfor No-Reference Consumer Video Quality Assessment,” IEEE Trans. on Image Process., vol. 28, no. 12, pp. 5923–5938, Dec. 2019, doi: 10.1109 / TIP.2019.2923051.”

[0075] Method 6: The method proposed by Reddy et al., reference “SVReddy Dendi andS.S.Channappayya,“No-Reference Video Quality Assessment Using NaturalSpatiotemporal Scene Statistics,”IEEE Trans.on Image Process.,vol.29,pp.5612–5624,2020,doi:10.1109 / TIP.2020.2984879.”

[0076] Method 7: The method proposed by Tu et al., reference “Z.Tu, Y.Wang, N.Birkbeck, B.Adsumilli, and ACBovik, “UGC-VQA: Benchmarking Blind Video Quality Assessment for UserGenerated Content,” IEEE Trans. on Image Process., vol. 30, pp. 4449–4464, 2021, doi: 10.1109 / TIP.2021.3072221.”

[0077] Method 8: The method proposed by Li et al., reference “D.Li, T.Jiang, and M.Jiang, “Quality Assessment of In-the-Wild Videos,” in Proceedings of the 27th ACMInternational Conference on Multimedia, Nice France: ACM, Oct. 2019, pp. 2351–2359. doi: 10.1145 / 3343031.3351028.”

[0078] Method 9: The method proposed by Korhonen l et al., reference “J. Korhonen, Y. Su, and J. You, “Blind Natural Video Quality Prediction via Statistical Temporal Features and Deep Spatial Features,” in Proceedings of the 28th ACM International Conference on Multimedia, Seattle WAUSA: ACM, Oct. 2020, pp. 3311–3319. doi: 10.1145 / 3394171.3413845.”

[0079] Method 10: The method proposed by You et al., reference “J. You and Y. Lin, “Efficient Transformer with Locally Shared Attention for Video Quality Assessment,” in 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France: IEEE, Oct. 2022, pp. 356–360. doi: 10.1109 / ICIP46576.2022.9898025.”

[0080] Method 11: The method proposed by Mitra et al., reference "S.Mitra and R.Soundararajan, "Multiview Contrastive Learning for Completely Blind Video Quality Assessment of User Generated Content," in Proceedings of the 30th ACM International Conference on Multimedia, Lisboa Portugal:ACM,Oct.2022,pp.1914–1924.doi:10.1145 / 3503161.3548064."

[0081] Method 12: The method proposed by Kancharla et al., reference “P. Kancharla and SS Channappayya, “Completely Blind Quality Assessment of User Generated Video Content,” IEEE Trans. on Image Process., vol. 31, pp. 263–274, 2022, doi: 10.1109 / TIP.2021.3130541.”

[0082] Method 13: The method proposed by Zheng et al., reference “Q. Zheng, Z. Tu, X. Zeng, A. C. Bovik, and Y. Fan, “A Completely Blind Video Quality Evaluator,” IEEE Signal Process. Lett., vol. 29, pp. 2228–2232, 2022, doi: 10.1109 / LSP.2022.3215311.”

[0083] Table 1 SRCC and PLCC results for LIVE-Q, LIVE-VQC, KoNViD-1, and CVSAR. The first two results are highlighted in bold

[0084]

[0085] Table 2 The best methods and their PLCC and SRCC results in cross-database scenarios

[0086]

[0087]

[0088] Table 3 Average computation time (in seconds) for selected videos in the original database. (resolution / frame / second)

[0089] METHOD 320×240 / 30 / 10 1920×1080 / 30 / 10 2012-BRISQUE 15.1 103.2 2014-NIQE 17.3 234.6 2014-VBLIINDS 148.9 2024.9 2016-VIIDEO 52.8 683.0 2019-VSFA 102.6 1387.9 2019-TLVQM 21.4 256.5 2020-CNN-TLVQM 50.0 123.4 2020-NSTSS 45.8 263.8 2021-VIDEVAL 149.9 1798.6 2022-LAGT-VQA 532.1 1066.7 2022-VISION 42.1 134.5 2022-STEM 127.1 318.0 2023-VIQE 16.7 200.6 STFR-BVQA 14.3 19.1

[0090] Table 4 Performance and computation time (in seconds) for five video clips of different lengths. The resolution of the video clip is 1920×1080.

[0091]

[0092] Table 5 Median of 10 SRCC and PLCC results in ablation experiments

[0093]

[0094]

Claims

1. A blind video quality assessment method based on a space-time feature analyzer, characterized in that The following steps are involved: Step 1: A spatiotemporal feature parser for extracting spatiotemporal features of videos. A spatiotemporal feature parser is designed to realize fast spatiotemporal feature extraction of videos and finally realize blind video quality evaluation. The specific process is as follows: The design of STFR requires a training process. At the beginning of the training process, a distortion-free video sequence is randomly selected from the video library as the sample source, the YUV video is converted into an RGB video signal, each frame of the video sequence is divided into video blocks of size 16×16×3, 10,000 video blocks are randomly selected, and the covariance matrix of M is decomposed into eigenvalues ​​and eigenvectors using zero-phase component analysis (ZCA). Then, the sample data is whitened to complete the preprocessing. The whitened data Z is used for iterative training of STFR. This process is continued by repeating formulas (2)-(3) until the final result converges. Finally, the obtained matrix is ​​converted from the whitened space to the original video space to obtain STFR, whose size is 768×768. Step 2: Use the trained spatiotemporal feature parser to quickly extract the spatiotemporal features of the video to implement a blind video quality assessment method. In order to extract the spatiotemporal features of the video sequence more quickly and conveniently, STFR is directly applied to the input video sequence, which enables comprehensive spatiotemporal feature analysis and extracts them directly from the video sequence. First, the input video sequence F = (f 1 ,f 2 ,…,f M ) for preprocessing; convert the YUV video into RGB video signal, remove the average value of each frame and divide it into 16×16×3 blocks, then use STFR to directly apply it to each frame of the video sequence, extract spatiotemporal feature information from the video in the form of the product of STFR and the video sequence, and use the output signal f_stfr i′ Analysis, calculate f_stfr i′ The row standard deviation is f_stfr i , f_stfr i Represents the video frame f i A feature of the adjacent video is calculated based on f_stfr i The feature mean feature,represents the frame and standard deviation to represent the spatial and temporal features of the,whole video sequence.,Finally, the model parameters are trained by using the,Support Vector Regression (SVR) in the regression model,using the spatial and temporal features of the video sequence as input and the,corresponding Mean Opinion Score (MOS) as the model output; Step 3: Take the video sequence as input, and sequentially train the spatial-temporal feature parser obtained in step 1 and the regression network model obtained in step 2 to finally obtain the quality score of the video.

Images