Video quality analysis method and device, equipment and medium

Abstract

The invention provides a video quality analysis method, device and equipment and a medium, and belongs to the technical field of video quality evaluation. The method comprises the following steps: acquiring original video data, and generating degraded video data with noise intensity, definition score and compression damage degree three-dimensional real labels through composite degradation processing; training a multi-task deep joint prediction network by using the data set; and inputting a real world degraded video to be analyzed into the trained network model, and outputting a corresponding noise intensity predicted value, a definition score predicted value and a compression damage degree predicted value. Compared with the prior art, the method has the advantages that high-fidelity simulation and automatic accurate evaluation of video multi-dimensional quality degradation factors can be realized, and the accuracy of a noise intensity prediction value, a definition score prediction value and a compression damage degree prediction value is improved; the method can be used for video coding and decoding optimization, network transmission quality control, video enhancement processing and other scenes.

Description

Technical Field

[0001] This invention relates to the field of video quality assessment technology, specifically to a video quality analysis method, apparatus, device, and medium. Background Technology

[0002] With the popularization of video acquisition equipment and network transmission technology, video data often suffers from image quality degradation due to multiple degradation factors such as imaging noise, motion blur, and encoding / decoding impairment during acquisition, compression, and transmission, which seriously affects the subsequent analysis and application effects.

[0003] Current video quality assessment methods and their shortcomings mainly include the following: Traditional video quality assessment methods typically employ independent modeling strategies, such as designing separate detection algorithms for noise, blur, or compression artifacts, lacking modeling of the coupling relationships between degradation types. Evaluation indicators are constructed based on frequency domain analysis or spatial domain statistical features (such as block effect detection and gradient histograms), which are difficult to adapt to complex degradation superposition scenarios. They rely on training with datasets of specific degradation types, resulting in insufficient accuracy in predicting parameters under the combined effects of multiple degradations. In other words, existing video quality assessment methods have low accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a video quality analysis method, apparatus, device, and medium to improve the accuracy of noise intensity prediction, sharpness score prediction, and compression impairment prediction, thereby improving the accuracy of video quality assessment. This invention achieves its purpose through the following methods: In a first aspect, the present invention provides a video quality analysis method, comprising: acquiring undegraded raw video data; performing composite degradation processing on the raw video data to generate degraded video data with three-dimensional real-world labels; training a multi-task deep joint prediction network based on a dataset containing the degraded video data and its corresponding three-dimensional real-world labels to obtain a trained network model; acquiring real-world degraded video data to be analyzed; inputting the real-world degraded video data to be analyzed into the trained network model; processing the input real-world degraded video data to be analyzed through the trained network model, and outputting a noise intensity prediction value, a sharpness score prediction value, and a compression impairment prediction value corresponding to the real-world degraded video data to be analyzed.

[0005] According to one embodiment of the present invention, the step of performing composite degradation processing on the original video data to generate degraded video data with three-dimensional real labels includes: superimposing Gaussian-Poisson mixed noise on video frames in the original video data to obtain noisy video frames, and recording the noise intensity parameters of the current noise injection processing; constructing a non-uniform blur kernel based on randomly generated motion trajectories, performing convolution processing on the noisy video frames through the non-uniform blur kernel to obtain motion-blurred simulated video frames, and recording the blur range parameters of the current motion-blurred simulated processing; calculating corresponding compression quantization parameters based on the noise intensity parameters and the blur range parameters through a dynamic correlation model; performing encoding and compression processing on the motion-blurred simulated video frames based on the compression quantization parameters to obtain the final degraded video frames, and combining the noise intensity parameters, blur range parameters, and compression quantization parameters into a three-dimensional real label corresponding to the current video frame.

[0006] According to one embodiment of the present invention, training a multi-task deep joint prediction network based on a dataset containing the degraded video data and its corresponding 3D ground truth labels to obtain a trained network model includes: inputting the degraded video data into a shared feature extraction layer of the multi-task deep joint prediction network to extract a multi-scale fused shared feature map; inputting the shared feature map into a noise intensity prediction branch, a sharpness score prediction branch, and a compression impairment prediction branch for parallel processing; in the noise intensity prediction branch, predicting a noise intensity value based on the shared feature map; in the sharpness score prediction branch, predicting a sharpness score value based on the shared feature map through gradient-sensitive feature selection and dilated convolution processing; in the compression impairment prediction branch, predicting a compression impairment value based on the shared feature map by focusing on high-frequency distortion regions through a channel attention mechanism; calculating a multi-task joint loss based on the predicted noise intensity value, the predicted sharpness score value, and the predicted compression impairment value, and the corresponding 3D ground truth labels, and updating the network parameters of the network model based on the multi-task joint loss.

[0007] According to one embodiment of the present invention, in the sharpness score prediction branch, based on the shared feature map, predicting the sharpness score value through gradient-sensitive feature selection and dilated convolution processing includes: performing gradient calculation on the shared feature map to extract high-frequency edge features; based on the high-frequency edge features, filtering out features related to blurred and non-blurred regions in the motion-blurred simulated video frame; using dilated convolutional layers with different dilation rates to capture multi-scale contextual information of the filtered features; and regressing to obtain the sharpness score value based on the captured multi-scale contextual information.

[0008] According to one embodiment of the present invention, in the compression damage prediction branch, based on the shared feature map, focusing on the high-frequency distortion region through a channel attention mechanism to predict the compression damage value includes: calculating the channel attention weights of the shared feature map; performing feature fusion on the feature map after channel attention weighting; locating the spatial region where the block effect or ringing effect is located from the fused features; and regressing the compression damage value based on the feature intensity of the located region.

[0009] According to one embodiment of the present invention, the step of calculating a multi-task joint loss based on the noise intensity prediction value, the sharpness score prediction value, and the compression damage prediction value, and the corresponding three-dimensional ground truth label, and updating the network parameters based on the multi-task joint loss, includes: calculating a first loss value for the noise intensity prediction task, a second loss value for the sharpness score prediction task, and a third loss value for the compression damage prediction task, respectively; assigning learnable dynamic weights to the first loss value, the second loss value, and the third loss value, respectively, wherein the dynamic weight of each task is negatively correlated with the uncertainty of the prediction for that task; and summing the weighted first loss value, the second loss value, and the third loss value to obtain the multi-task joint loss.

[0010] According to one embodiment of the present invention, acquiring undegraded raw video data includes: reading a video file from a preset video dataset; decoding the read video file to obtain an uncompressed raw video frame sequence; performing format unification processing on the raw video frame sequence, and outputting the processed raw video frame sequence as the undegraded raw video data; wherein, the format unification processing includes adjusting the resolution to a preset size and converting the color space to a preset format.

[0011] Secondly, the present invention also provides a video quality analysis device, comprising: a raw data acquisition module for acquiring undegraded raw video data; a composite degradation processing module for performing composite degradation processing on the raw video data to generate degraded video data with three-dimensional real-world labels; a network training module for training a multi-task deep joint prediction network based on a dataset containing the degraded video data and its corresponding three-dimensional real-world labels to obtain a trained network model; a test data acquisition module for acquiring real-world degraded video data to be analyzed; a data input module for inputting the real-world degraded video data to be analyzed into the network model trained by the network training module; and a quality prediction module for processing the real-world degraded video data input by the data input module through the trained network model, and outputting a noise intensity prediction value, a sharpness score prediction value, and a compression impairment prediction value corresponding to the real-world degraded video data to be analyzed.

[0012] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a video quality analysis method as described in the first aspect.

[0013] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements a video quality analysis method as described in the first aspect.

[0014] This invention, through the introduction of a composite degradation modeling method that conforms to the real physical order and correlation rules, generates high-quality and diverse training data, enabling deep networks to learn the intrinsic relationships between noise, blur, and compression. A multi-task deep joint prediction network trained on this data can more effectively decouple the unique information of various degradation types from complex mixed degradation features (e.g., focusing on blur through gradient-sensitive features and dilated convolutions, and focusing on compression artifacts through channel attention networks), thereby achieving more accurate collaborative prediction of the intensity of various degradation types. Compared to traditional methods of independent modeling, this invention significantly improves the accuracy of parameter prediction on real, complex degraded videos, resulting in higher accuracy for noise intensity prediction, sharpness score prediction, and compression impairment prediction. Furthermore, this invention directly outputs a structured three-dimensional quality parameter vector (noise intensity prediction, sharpness score prediction, and compression impairment prediction), rather than a single abstract score. This decomposed evaluation result has clear physical meaning, clearly indicating the specific causes and extent of video quality degradation. For example, the output parameters can be directly used as input to subsequent video restoration algorithms, guiding them to adaptively select or adjust denoising intensity, deblurring algorithms, or compression artifact elimination strategies. This achieves a closed-loop linkage from evaluation to restoration, effectively improving the targeting and efficiency of restoration and avoiding resource waste. This invention also provides a video quality analysis device, a computer device, and a computer-readable storage medium, all possessing the aforementioned beneficial effects, which will not be elaborated upon further here. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of the video quality analysis method according to an embodiment of the present invention; Figure 2 yes Figure 1A schematic diagram of a specific implementation method for step S10; Figure 3 yes Figure 1 A schematic diagram of a specific implementation method for step S20; Figure 4 yes Figure 1 A schematic diagram of a specific implementation method for step S30; Figure 5 yes Figure 3 A flowchart illustrating a specific implementation of step S34; Figure 6 yes Figure 3 A schematic diagram of a specific implementation method for step S35; Figure 7 yes Figure 3 A schematic diagram of a specific implementation method for step S36; Figure 8 This is a schematic block diagram of a video quality analysis device according to an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be understood that, when used in this specification and the appended claims, the terms “comprising” and “including” indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more of its features, integrals, steps, operations, elements, components and / or collections thereof.

[0019] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0020] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0021] To facilitate understanding of the technical solution of this invention, the key terms used herein will first be explained.

[0022] Multi-task Deep Joint Prediction Network: A deep learning-based neural network architecture whose core design involves sharing a single low-level feature extraction module and simultaneously connecting multiple (three in this embodiment) prediction branches (tasks) with different learning objectives. These branches share low-level features and collaboratively optimize during training, aiming to jointly predict multiple related but different target variables (noise intensity, sharpness score, and compression impairment in this embodiment) from the same input data.

[0023] Gaussian-Poisson hybrid noise: a model used to simulate the composite noise introduced by image sensors during the imaging process. Gaussian noise primarily simulates additive noise such as thermal noise from the sensor circuitry; Poisson noise (also known as shot noise) primarily simulates signal-dependent noise generated by the randomness of photons arriving at the sensor. This hybrid model more realistically reflects the noise characteristics in actual shooting scenarios.

[0024] Non-uniform blur kernel: A type of convolution kernel used to simulate motion blur, whose shape and intensity distribution are not uniform or symmetrical (such as the common Gaussian blur kernel), but are determined by the actual motion trajectory of the object or camera during the exposure time. This type of blur kernel can more accurately simulate the blur effect caused by complex motion (such as variable speed or curvilinear motion).

[0025] Dynamic correlation model: In the context of this invention, it specifically refers to a mathematical model used to describe the physical or statistical correlation between intensity parameters of different degradation types (such as noise, blur, and compression) during video degradation. This model is used to constrain the sampling combinations of different degradation parameters when synthesizing training data, making them more consistent with the degradation patterns in the real world and avoiding the generation of unreasonable, unrealistic samples such as high noise, extremely clear, and low compression.

[0026] This embodiment provides a video quality analysis method, apparatus, device, and medium. This embodiment aims to address the problems of insufficient accuracy and poor physical interpretability of traditional quality assessment methods in real-world video scenarios due to the coupling of multiple degradation factors such as noise, motion blur, and compression. It seeks to improve the accuracy of noise intensity prediction, sharpness score prediction, and compression impairment prediction, thereby enhancing the accuracy of video quality assessment. (Refer to...) Figure 1 A video quality analysis method includes: S10: Obtain the undegraded raw video data.

[0027] Specifically, this step is the starting point of the entire method, aiming to provide high-quality seed data for subsequent degradation data synthesis. In one embodiment, video files can be read from publicly available high-quality video datasets (such as YouTube-8M, Kinetics, etc.). The read video files are then decoded to obtain an uncompressed or lossless compressed raw video frame sequence (such as YUV or RGB format). To ensure consistency in subsequent processing, the raw video frame sequence needs to undergo format unification preprocessing. This preprocessing includes, but is not limited to: adjusting the resolution of the video frames to a preset fixed size (e.g., 256×256 pixels), and uniformly converting the color space to a preset format (e.g., RGB three-channel format). The processed video frame sequence is then output as the undegraded raw video data.

[0028] S20: Perform composite degradation processing on the original video data to generate degraded video data with three-dimensional real-world labels.

[0029] This step is used to construct a high-quality training dataset. Its purpose is to generate a large amount of degraded video frame-to-real-degrade parameter label pairing data covering various combinations of real-world degradation through controlled, physically-compliant degradation simulations.

[0030] S30: Based on the dataset containing the degraded video data and its corresponding 3D real labels, train the multi-task deep joint prediction network to obtain the trained network model.

[0031] This step uses the large-scale paired dataset generated in step S20 to train a deep neural network, enabling it to learn to accurately decouple from degraded video frames and predict the strength of the three degradation parameters.

[0032] S40: Acquire real-world degraded video data to be analyzed.

[0033] During the model application phase, users provide real-world videos whose quality needs to be evaluated. These videos are black-box inputs whose specific degradation type and degree are unknown. Similarly, these videos can be decoded and preprocessed in a format-unified manner similar to step S10 to obtain a frame sequence that meets the network input requirements.

[0034] S50: Input the real-world degraded video data to be analyzed into the trained network model.

[0035] The preprocessed video frames to be analyzed are input one by one into the network model trained in step S30.

[0036] S60: The trained network model processes the input real-world degraded video data to be analyzed, and outputs the predicted noise intensity, sharpness score, and compression impairment values ​​corresponding to the real-world degraded video data. That is, a three-dimensional prediction vector [σ_noise_pred, blur_size_pred, QP_compression_pred] is finally output as the quantitative analysis result of the video frame's quality.

[0037] This invention achieves end-to-end, multi-dimensional quality analysis of complex degradation-coupled videos. Specifically: Step S20 generates physically meaningful and widely covered training data through a composite degradation chain modeling process, improving the model's generalization ability and interpretability of prediction results in real-world scenarios. Step S30, based on training a multi-task deep joint prediction network, enables the model to effectively decouple the intensity of different degradation components such as noise, blur, and compression from mixed degradation features, solving the problem of inaccurate evaluation under multiple degradation conditions in traditional methods. The final output structured three-dimensional parameters not only provide a comprehensive quality score but also indicate the specific reasons for quality degradation (which degradation is dominant and its intensity), directly and accurately guiding downstream video restoration algorithms (such as selecting denoising filters, deblurring algorithms, or post-processing filters with specific intensities), forming an analysis-restoration closed loop. This improves restoration effectiveness while avoiding the waste of computational resources caused by blind processing.

[0038] This invention can be applied to: video surveillance system quality assessment: automatically diagnosing quality degradation in surveillance footage caused by low light (e.g., high noise), jitter (e.g., blur), and bandwidth limitations (e.g., high compression), and triggering alarms or adaptive adjustments; streaming media service platforms: real-time analysis of the objective quality of user-end video streams for QoE (Quality of Experience) monitoring, bitrate adaptive decision-making, or fault location; preprocessing modules for video restoration and enhancement algorithms: providing prior degradation parameter guidance for various video denoising, super-resolution, and decompression artifact algorithms to achieve adaptive and accurate restoration; video encoding and transmission optimization: dynamically adjusting encoding strategies or network transmission parameters based on analyzed quality bottlenecks (e.g., severe blur).

[0039] In some embodiments, refer to Figure 2 Step S10 specifically includes: S11: Read video files from a preset video dataset. The preset video dataset can be stored on a local disk or network storage and contains multiple high-quality, undegraded or slightly degraded original video files.

[0040] S12: Decode the read video file to obtain an uncompressed raw video frame sequence. Use a corresponding video decoding library (such as FFmpeg) to decode the video file, extract the pixel data of each frame, and form the raw video frame sequence. This step aims to remove any possible lossy compression.

[0041] S13: Perform format unification processing on the original video frame sequence, and output the processed original video frame sequence as the undegraded original video data; wherein, the format unification processing includes adjusting the resolution to a preset size and converting the color space to a preset format. Specifically, the resolution of each frame is adjusted (e.g., by downsampling or upsampling through bilinear interpolation) to the preset input size of the neural network (e.g., 224×224 pixels); and the color space is converted to the preset format of the neural network (e.g., from YUV to RGB, or unified to grayscale). This step ensures that all input data have consistent size and format, facilitating subsequent batch processing.

[0042] In some embodiments, refer to Figure 3 Step S20 specifically includes: S21: Gaussian-Poisson mixed noise is superimposed on the video frames in the original video data to obtain noisy video frames, and the noise intensity parameters of the current noise injection process are recorded. Specifically, for each clean video frame input, the following formula is used: I_noisy = I_clean + N_gaussian(0,σ_noise^2) + N_poisson(I_clean) adds mixed noise.

[0043] Where I_noisy represents a noisy video frame, I_clean represents each clean video frame, σ_noise represents the noise intensity parameter, N_gaussian is Gaussian noise with zero mean and variance σ_noise^2, and σ_noise is randomly sampled from a preset range (e.g., [0.5, 5.0]); N_poisson is Poisson noise, the intensity of which is related to the original pixel value I_clean. Through this operation, the noisy video frame I_noisy is obtained, and the noise intensity parameter σ_noise used in this operation is recorded.

[0044] S22: Construct a non-uniform blur kernel based on randomly generated motion trajectories. Perform convolution processing on the noisy video frame using this non-uniform blur kernel to obtain a motion-blurred simulated video frame, and record the blur range parameters of the current motion-blurred simulation. Specifically, a non-uniform blur kernel is generated based on random motion trajectories to simulate blur caused by object or camera movement. Blur kernel K blur It can be calculated using the following formula: in, v x , v y The motion velocity component is discretized over time, where T is the duration. The discretized motion velocity component ( v x, v y The non-uniform blur kernel is generated by setting the duration T to simulate the blurred trajectory caused by camera or object movement.

[0045] S23: Based on the noise intensity parameter and the fuzzy range parameter, the corresponding compression quantization parameter is calculated through a dynamic correlation model.

[0046] To avoid generating unrealistic degenerate combinations, this invention introduces a dynamic correlation model. This model establishes a correlation between noise, fuzziness, and compression intensity. For example, compression quantization parameters can be calculated using the following formula: Where σ_noise represents the noise intensity parameter, blur_size represents the blur range parameter, QP represents the compression quantization parameter, and k1, k2, and b are the fitting coefficients. The perturbation is random. This modeling approach avoids unreasonable combinations caused by independent parameter sampling (such as the coexistence of high noise and low compression), and is closer to the real degradation mechanism.

[0047] S24: Based on the compression quantization parameters, the motion-blurred simulated video frame is encoded and compressed to obtain the final degraded video frame. The noise intensity parameter, blur range parameter, and compression quantization parameters are combined to form the three-dimensional ground truth label corresponding to the current video frame. Using a standard video or image encoder (such as the intra-frame coding mode of an H.264 encoder, or a JPEG encoder), the motion-blurred simulated video frame is compressed and encoded using the QP calculated in step S23 as the quantization parameter, and then immediately decoded to obtain the final degraded video frame I_degraded containing compression artifacts. The σ_noise, blur_size, and QP used in this process are combined into a three-dimensional vector y=[σ_noise, blur_size, QP], which serves as the ground truth label for I_degraded. By repeatedly executing S21-S24 on a large number of video frames, a large-scale training dataset {I_degraded,y} with rich combinations of degradation types and intensities and conforming to real physical relationships can be constructed.

[0048] In some embodiments, refer to Figure 4 Step S30 specifically includes: S31: The degraded video data is input into the shared feature extraction layer of the multi-task deep joint prediction network to extract a multi-scale fused shared feature map F. The shared feature extraction layer can use a deep convolutional neural network (such as ResNet, VGG variants, etc.) as its backbone. After the input single-frame degraded image I_degraded is processed by a series of convolutions, pooling, non-linear activations, and possibly attention modules of the backbone network, a multi-channel feature map F rich in semantic information is output. This feature map F fuses low- to high-level features in the image that are related to noise, blur, and compression artifacts.

[0049] S32: Input the shared feature map F into the noise intensity prediction branch, the sharpness score prediction branch and the compression damage prediction branch respectively for parallel processing.

[0050] The shared feature map F is copied three times and fed into three independently designed prediction branches.

[0051] S33: In the noise intensity prediction branch, the noise intensity value σ_noise_pred is predicted based on the shared feature map F. The noise intensity prediction branch has a relatively simple structure. After receiving the shared feature map F, it typically compresses the spatial feature map into a feature vector through a global average pooling layer, and then performs a nonlinear transformation through several fully connected layers, finally outputting a scalar value σ_noise_pred as the predicted noise intensity.

[0052] S34: In the sharpness score prediction branch, based on the shared feature map F, the sharpness score value blur_size_pred is predicted through gradient-sensitive feature selection and dilated convolution processing. The sharpness score branch is specifically designed to capture blurry features. It first performs gradient calculation on the feature map F or uses a learnable edge detection module to extract high-frequency edge features. Then, a feature selection mechanism is used to strengthen feature channels related to blurry / unblurry region discrimination. Next, multiple dilated convolutional layers with different dilation rates are used to process the selected features in parallel to capture contextual information at different scales (a large receptive field helps to judge large-scale blur, while a small receptive field helps to judge detail blur). Finally, the multi-scale features are fused and the sharpness score blur_size_pred is output through a regression head (a smaller value generally indicates greater blur).

[0053] S35: In the compression damage prediction branch, based on the shared feature map F, a channel attention mechanism is used to focus on high-frequency distortion regions and predict the compression damage value QP_pred. The compression damage branch focuses on high-frequency distortions such as block effects and ringing effects caused by compression. This branch first applies a channel attention mechanism (such as the ECA-Net module) to the shared feature map F. This mechanism automatically generates channel weights by analyzing the feature importance of each channel and reweights the feature map according to the channel dimension, thereby allowing the network to focus on feature channels containing compression artifact information and suppressing irrelevant information. The weighted features are then processed through convolutional layers and fully connected layers for information integration and regression, finally outputting the compression damage prediction value QP_pred.

[0054] S36: Based on the predicted noise intensity, sharpness score, and compression impairment, calculate the multi-task joint loss with the corresponding 3D ground truth labels, and update the network parameters of the network model based on the multi-task joint loss. For a batch of training data, calculate the loss value for each of the three prediction tasks separately. For example, Smooth L1 Loss or Mean Squared Error (MSE) can be used as the loss function L_noise, blur_size, and L_qp for each task. Then, using a weighted strategy based on task uncertainty, sum the three losses to obtain the total loss. L_total=w1*L_noise+w2*blur_size+w3*L_qp In this context, * indicates multiplication. (i=1,2,3) represent the weights of the losses for different tasks, σ i Let L_total represent the learnable uncertainty parameters for task i. During training, the model simultaneously optimizes the damage function and the uncertain parameters, achieving adaptive weight allocation. Finally, a gradient descent algorithm (such as Adam) is used to calculate the gradient of L_total with respect to all network parameters, and the parameters are updated. Through iterative optimization, the network gradually learns to accurately predict 3D degradation parameters from degraded images.

[0055] In some embodiments, refer to Figure 5 Step S34 specifically includes: S341: Perform gradient calculation on the shared feature map to extract high-frequency edge features. Specifically, perform gradient calculation on the shared feature map F using the Sobel operator or a learnable convolutional kernel to extract high-frequency edge features in the horizontal and vertical directions of the image, obtaining a gradient feature map G. Gradient features can effectively highlight edge and texture information in the image, which is sensitive to blur.

[0056] S342: Based on the high-frequency edge features, select features related to the blurred and unblurred regions in the motion-blurred simulated video frame. Specifically, based on the high-frequency edge feature map G, design a feature selection module. This module can be a lightweight convolutional layer or attention mechanism, whose function is to analyze the distribution of gradient features, automatically select the feature channels that contribute the most to distinguishing blurred and unblurred regions, and suppress feature channels that are not sensitive to blur discrimination, thereby obtaining the selected enhanced feature map G_selected.

[0057] S343: Use dilated convolutional layers with different dilation rates to capture multi-scale contextual information from the selected features. Specifically, G_selected is processed in parallel using dilated convolutional layers with different dilation rates (e.g., dilation rates r=1, 2, 4). A dilated convolution with a dilation rate of 1 is the standard convolution, which has a small receptive field and focuses on local detail blur; a dilated convolution with a larger dilation rate has a larger receptive field while maintaining the number of parameters, which helps to capture a wider range of blurry information. The output features of dilated convolutional layers with different dilation rates are concatenated and fused to obtain a fused feature F_multi_scale containing multi-scale contextual information.

[0058] S344: Based on the captured multi-scale contextual information, the sharpness score is obtained through regression. Specifically, the fused feature F_multi_scale is input into a regression head consisting of fully connected layers, and the final sharpness score, blur_size_pred, is obtained through regression.

[0059] By focusing on key features through gradient-sensitive selection and combining multi-scale dilated convolution to capture blur context of different ranges, this branch can more accurately assess the degree of blur in an image.

[0060] In some embodiments, refer to Figure 6 Step S35 specifically includes: S351: Calculate channel attention weights for the shared feature map; specifically, firstly, apply the Efficient Channel Attention (ECA) module to the shared feature map F. This module first performs global average pooling on the feature map F in its spatial dimensions (height and width), compressing the features of each channel into a scalar to obtain a channel description vector. Then, a one-dimensional convolution (kernel size k, which can be adaptively determined) is used to perform cross-channel information interaction on the description vector, learning the importance weight of each channel. Finally, the weights are normalized to between 0 and 1 using the Sigmoid function to obtain the channel attention weight vector α. The original feature map F is multiplied by the weight vector α through channels to achieve feature reweighting, resulting in the weighted feature map F_attended. This process enables the network to adaptively focus on feature channels containing compressed artifacts such as block artifacts and ringing artifacts.

[0061] S352: Perform feature fusion on the feature maps after channel attention weighting. Specifically, the feature map F_attended after channel attention weighting is input into a feature fusion module. This module may contain a 1×1 convolutional layer and subsequent convolutional layers to fuse and further refine the weighted multi-channel features to obtain the fused feature F_fused.

[0062] S353: Locate the spatial regions containing block artifacts or ringing artifacts from the fused features. Specifically, the network locates the spatial regions containing block artifacts or ringing artifacts from the fused features F_fused. This is typically accomplished automatically through deep convolutional layers in the network, whose kernels respond to regular blocky boundaries or ringing textures in the image resulting from compression.

[0063] S354: Based on the feature intensity of the located region, the compression damage value is obtained by regression. Specifically, based on the feature intensity of the located region, for example, the feature responses of these regions are summarized by global pooling. The summarized feature vector is input into a fully connected regression layer to obtain the compression damage value QP_pred. A higher value generally indicates more severe compression damage.

[0064] In some embodiments, refer to Figure 7 Step S36 specifically includes: S361: Calculate the first loss value for the noise intensity prediction task, the second loss value for the sharpness score prediction task, and the third loss value for the compression damage prediction task, respectively. Specifically, calculate the first loss value L_noise for the noise intensity prediction task, the second loss value blur_size for the sharpness score prediction task, and the third loss value L_qp for the compression damage prediction task. Each loss function can use mean squared error (MSE) or smoothed L1 loss.

[0065] S362: Assign learnable dynamic weights to the first, second, and third loss values, respectively, where the dynamic weight for each task is negatively correlated with the uncertainty of the prediction for that task. Specifically, assign learnable dynamic weights w1, w2, and w3 to the first, second, and third loss values, respectively. A task uncertainty weighting method is used here. A learnable uncertainty parameter σ is introduced for each task. i This parameter reflects the uncertainty of the model's prediction for the task. The dynamic weights are negatively correlated with the uncertainty of the task; specifically, they can be set to w. i =1 / (2*σ i ^2), where i=1,2,3 correspond to noise, fuzziness, and compression tasks, respectively.

[0066] S363: Sum the weighted first loss value, second loss value, and third loss value to obtain the multi-task joint loss L_total. The formula is as follows: L_total=w1*L_noise+w2*blur_size+w3*L_qp+log(σ1*σ2*σ3) The addition of the log(σ1*σ2*σ3) term is to prevent uncertainty in the parameter σ. i It becomes too large. During training, the model not only optimizes the backbone and branch parameters of the network to minimize the prediction error, but also optimizes these uncertainty parameters σ. i For tasks with high prediction uncertainty (such as σ), i (Large), its loss weight w i It will automatically reduce the focus on the difficult task during training, thereby achieving automatic balancing and collaborative optimization among the three tasks.

[0067] Optionally, in the construction of the dynamic correlation model, the correlation between noise intensity σ_noise and fuzziness range parameter blur_size can also be modeled as other nonlinear relationships.

[0068] For example, blur_size = a * exp(b * σ_noise) + c can be used to simulate the phenomenon that blur may increase sharply under extremely high noise. The coefficients a, b, and c can be obtained by regression fitting through analysis of real degraded video datasets.

[0069] Optionally, in the training of the task-deep joint prediction network, the backbone network sharing the feature extraction layer is not limited to ResNet, but can also be other advanced convolutional neural network architectures such as DenseNet and EfficientNet, or a hybrid architecture that includes a Transformer module, as long as it has strong multi-scale feature extraction capabilities.

[0070] Optionally, in obtaining the sharpness score, in addition to using dilated convolution, a spatial attention mechanism can be introduced to make the network pay more attention to areas that may be blurred in space (such as the edges of moving objects), thereby further enhancing its ability to capture blurred features.

[0071] The video quality analysis method and the model trained by this invention can not only output three independent degradation parameters, but also, based on these three parameters and a predefined quality mapping table, comprehensively evaluate video frames or sequences into "high quality," "medium quality," and "low quality" levels, providing users with more intuitive quality assessment results. An example is shown in Table 1 below: Table 1 Furthermore, the analysis results can be directly used as control signals and input into an adaptive video processing pipeline. For example, the intensity parameters of the denoising algorithm can be dynamically selected based on the predicted noise intensity, and different deblurring strategies can be selected based on the sharpness score to achieve intelligent video enhancement.

[0072] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0073] Secondly, referring to Figure 8 This invention provides a video quality analysis device 100, comprising: Raw data acquisition module 101 is used to acquire undegraded raw video data; The composite degradation processing module 102 is used to perform composite degradation processing on the original video data to generate degraded video data with three-dimensional real labels; The network training module 103 is used to train a multi-task deep joint prediction network based on a dataset containing the degraded video data and its corresponding three-dimensional real labels, so as to obtain a trained network model. The test data acquisition module 104 is used to acquire real-world degraded video data to be analyzed; The data input module 105 is used to input the real-world degraded video data to be analyzed into the network model trained by the network training module; The quality prediction module 106 is used to process the real-world degraded video data to be analyzed input by the data input module through the trained network model, and output the noise intensity prediction value, sharpness score prediction value and compression impairment prediction value corresponding to the real-world degraded video data to be analyzed.

[0074] For specific limitations regarding the video quality analysis device, please refer to the limitations of the video quality analysis method above, which will not be repeated here. Each module in the aforementioned video quality analysis device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independent of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the corresponding operations of each module.

[0075] Thirdly, embodiments of the present invention provide a computer device. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a video quality analysis method.

[0076] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed, can perform the steps provided in the above embodiments.

[0077] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0078] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0079] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0080] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A video quality analysis method, characterized in that, include: Obtain the original, undegraded video data; Perform composite degradation processing on the original video data to generate degraded video data with three-dimensional real-world labels; Based on a dataset containing the degraded video data and its corresponding 3D real labels, a multi-task deep joint prediction network is trained to obtain a trained network model. Acquire real-world degraded video data to be analyzed; The real-world degraded video data to be analyzed is input into the trained network model; The trained network model processes the input real-world degraded video data to be analyzed, and outputs the predicted noise intensity, sharpness score, and compression impairment values ​​corresponding to the real-world degraded video data to be analyzed.

2. The video quality analysis method according to claim 1, characterized in that, The process of performing composite degradation processing on the original video data to generate degraded video data with 3D real-world labels includes: Gaussian-Poisson mixed noise is superimposed on the video frames in the original video data to obtain noisy video frames, and the noise intensity parameters of the current noise injection process are recorded. A non-uniform blur kernel is constructed based on randomly generated motion trajectories. The noisy video frame is then convolved using the non-uniform blur kernel to obtain a motion-blurred simulated video frame. The blur range parameters of the current motion-blurred simulation are recorded. Based on the noise intensity parameter and the fuzzy range parameter, the corresponding compression quantization parameter is calculated through a dynamic correlation model; The motion-blurred simulated video frame is encoded and compressed based on the compression quantization parameters to obtain the final degraded video frame. The noise intensity parameters, blur range parameters, and compression quantization parameters are then combined to form the 3D real label corresponding to the current video frame.

3. The video quality analysis method according to claim 2, characterized in that, The multi-task deep joint prediction network is trained on a dataset containing the degraded video data and its corresponding 3D ground truth labels to obtain a trained network model, including: The degraded video data is input into the shared feature extraction layer of the multi-task deep joint prediction network to extract a multi-scale fused shared feature map. The shared feature maps are input into the noise intensity prediction branch, the sharpness score prediction branch, and the compression damage prediction branch respectively for parallel processing. In the noise intensity prediction branch, the noise intensity value is predicted based on the shared feature map; In the sharpness score prediction branch, based on the shared feature map, a sharpness score is predicted through gradient-sensitive feature selection and dilated convolution processing. In the compression damage prediction branch, based on the shared feature map, the high-frequency distortion region is focused through a channel attention mechanism to predict the compression damage value. Based on the predicted noise intensity, sharpness score, and compression impairment, a multi-task joint loss is calculated with the corresponding three-dimensional ground truth labels, and the network parameters of the network model are updated based on the multi-task joint loss.

4. The video quality analysis method according to claim 3, characterized in that, In the sharpness score prediction branch, based on the shared feature map, a sharpness score is predicted through gradient-sensitive feature selection and dilated convolution processing, including: Gradient calculation is performed on the shared feature map to extract high-frequency edge features; Based on the high-frequency edge features, features related to the blurred and non-blurred regions in the motion-blurred simulated video frame are selected. Use dilated convolutional layers with different dilation rates to capture multi-scale contextual information from the filtered features; The sharpness score is obtained by regression based on the captured multi-scale contextual information.

5. The video quality analysis method according to claim 3, characterized in that, In the compression damage prediction branch, based on the shared feature map, a channel attention mechanism is used to focus on high-frequency distortion regions to predict the compression damage value, including: Channel attention weights are calculated for the shared feature map; The feature maps weighted by channel attention weights are then fused. Locate the spatial region where the block effect or ringing effect is located from the fused features; The compressive damage value is obtained by regression based on the characteristic intensity of the located area.

6. The video quality analysis method according to claim 3, characterized in that, The step of calculating the multi-task joint loss based on the predicted noise intensity, predicted sharpness score, and predicted compression impairment, along with the corresponding 3D ground truth labels, and updating the network parameters based on the multi-task joint loss, includes: Calculate the first loss value for the noise intensity prediction task, the second loss value for the sharpness score prediction task, and the third loss value for the compression damage prediction task, respectively. Learnable dynamic weights are assigned to the first loss value, the second loss value, and the third loss value, respectively, wherein the dynamic weight of each task is negatively correlated with the uncertainty of the prediction of that task; The weighted first loss value, second loss value, and third loss value are summed to obtain the multi-task joint loss.

7. The video quality analysis method according to claim 1, characterized in that, Obtain the undegraded raw video data, including: Read video files from a pre-defined video dataset; The read video file is decoded to obtain an uncompressed raw video frame sequence; The original video frame sequence is subjected to format unification processing, and the processed original video frame sequence is output as the undegraded original video data; wherein, the format unification processing includes adjusting the resolution to a preset size and converting the color space to a preset format.

8. A video quality analysis device, characterized in that, include: The raw data acquisition module is used to acquire undegraded raw video data. The composite degradation processing module is used to perform composite degradation processing on the original video data to generate degraded video data with three-dimensional real labels; The network training module is used to train a multi-task deep joint prediction network based on a dataset containing the degraded video data and its corresponding three-dimensional real labels, so as to obtain a trained network model. The test data acquisition module is used to acquire real-world degraded video data to be analyzed; The data input module is used to input the real-world degraded video data to be analyzed into the network model trained by the network training module; The quality prediction module is used to process the real-world degraded video data to be analyzed input by the data input module through the trained network model, and output the noise intensity prediction value, sharpness score prediction value and compression impairment prediction value corresponding to the real-world degraded video data to be analyzed.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a video quality analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a video quality analysis method as described in any one of claims 1 to 7.

Images