Video quality step-by-step evaluation method and device, system, and storage medium

Abstract

This invention discloses a method, apparatus, system, and storage medium for progressively evaluating video quality, comprising: editing an original video into non-overlapping video segments; extracting segment-level subtle motion implicit features from the video segments; extracting frame-level spatial explicit features from each video frame, fitting the frame-level spatial explicit features to obtain frame-stream-level spatial explicit features; fusing the segment-level subtle motion implicit features and frame-stream-level spatial explicit features within each video segment, performing temporal modeling through time hysteresis pooling, and performing quality regression to obtain a first-level video quality; statistically analyzing the feature maps of sampled video frames across all video segments at different stages, performing differential processing on the explicit features of feature maps of consecutive video frames at the same stage to simulate inter-frame variation differences, and performing quality regression to obtain a second-level video quality; and fusing the first-level and second-level video quality to obtain the final video quality.

Description

Technical Field

[0001] This invention belongs to the field of video quality evaluation, and particularly relates to a method, apparatus, system, and storage medium for step-by-step video quality evaluation. Background Technology

[0002] With the rapid development of self-media technology and the widespread adoption of smart devices, massive amounts of personally created videos are uploaded to streaming platforms every moment. However, limited by shooting equipment and creative experience, the perceived quality of these videos varies greatly, and many fail to meet the quality requirements of streaming platforms. Furthermore, video quality degrades during transmission, processing, and storage. Traditional methods relying on manual inspection are no longer sufficient to meet the timeliness requirements of assessing the quality of massive amounts of video. There is a need to explore and establish automated and efficient video quality assessment methods that achieve perceived quality consistent with human perception.

[0003] Zhejiang University of Technology, in its patent "A Video Action Quality Evaluation Method Based on Spatiotemporal Domain Awareness" (patent application number: CN 202310656613.X), proposes to perform spatiotemporal segmentation processing on the input video, dividing it into a set of video segments p and a set of keyframe segments x; extracting multi-rate motion features from the video segment set p and multi-scale spatial features from the keyframe segment set x respectively; finally, fusing the aligned spatiotemporal motion features and spatial features, and obtaining the video action quality evaluation score through a quality regression module. This method only samples one keyframe within each video segment, assigning segment-level motion attributes to static regions in the image, leading to motion artifacts and blurring distortion. Furthermore, human visual perception relies on close collaboration between the eye and brain, as well as fine coordination between different processing stages within the brain; this method has limitations in simulating the human visual perception mechanism. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, apparatus, system, and storage medium for progressively evaluating video quality.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for progressively evaluating video quality includes:

[0007] Step 1: Edit the original video into non-overlapping video clips;

[0008] Step 2: Perform patch sampling on the video clips to extract segment-level subtle motion implicit features;

[0009] Step 3: Sample several video frames continuously within each video segment, extract the frame-level spatial explicit features of each video frame, and fit the frame-level spatial explicit features to obtain the frame stream-level spatial explicit features.

[0010] Step 4: In each video segment, fuse segment-level micro-variation motion implicit features and frame stream-level spatial explicit features, and perform temporal modeling through time hysteresis pooling to obtain the first level of video quality through quality regression.

[0011] Step 5: Statistically analyze the feature maps of different stages of the sampled video frames within all video segments. Perform differential processing on the explicit features of the feature maps of consecutive video frames in the same stage. Use the Transformer model to simulate the differences in inter-frame changes and perform quality regression to obtain the second level of video quality.

[0012] Step 6: Combine the first-level video quality and the second-level video quality to obtain the final video quality.

[0013] Preferably, step 2 includes:

[0014] In the spatial dimension, video clips are processed into a grid, and patch sampling is performed within each video clip grid using a 3D sliding window to obtain video patch clips.

[0015] The Video Swin Transformer model was used to extract latent features of fragment-level infinitesimal motion.

[0016] Preferably, step 3 includes:

[0017] Frame-level multi-scale mean and standard deviation features are obtained by pre-training a ConvNext V2 model on ImageNet. A global variance-guided attention mechanism is introduced to quantify the importance of the temporal standard deviation features of each channel at different time points, which are then mapped to channel attention weights. The channel attention weights are multiplied element-wise with the multi-scale standard deviation features to obtain the initial frame quality. The continuously sampled initial frame quality is then combined with two one-dimensional convolutions and the GeLU activation function along the time axis to obtain temporal attention weights. The temporal attention weights are then assigned to the initial frame quality to obtain the explicit spatial features of the frame.

[0018] Arrange the frame-level spatial explicit features along the time dimension to obtain the initial spatial explicit features of the frame stream; then perform adaptive pooling of the initial spatial explicit features of the frame stream along the time dimension to obtain the frame stream spatial explicit features of each video segment.

[0019] Preferably, learnable parameters are introduced into the memory retention elements within each timestamp in the time hysteresis pool, and nonlinear multi-scale fitting is performed using an exponential function.

[0020] The present invention also provides a video quality step-by-step evaluation device, comprising:

[0021] The editing module is used to cut the original video into non-overlapping video clips;

[0022] The extraction module is used to perform patch sampling on video segments and extract segment-level micro-variation motion implicit features;

[0023] The fitting module is used to continuously sample several video frames within each video segment, extract the frame-level spatial explicit features of each video frame, and fit the frame-level spatial explicit features to obtain the frame stream-level spatial explicit features.

[0024] The first processing module is used to fuse segment-level micro-variation motion implicit features and frame stream-level spatial explicit features within each video segment, and to perform temporal modeling through time hysteresis pooling and quality regression to obtain the first level of video quality.

[0025] The second processing module is used to statistically analyze the feature maps of different stages of the sampled video frames within all video segments, perform differential processing on the explicit features of the feature maps of consecutive video frames in the same stage, simulate the differences in inter-frame changes using the Transformer model, and perform quality regression to obtain the second level of video quality.

[0026] The fusion module is used to merge the first-level video quality and the second-level video quality to obtain the final video quality.

[0027] Preferably, the extraction module includes:

[0028] The sampling unit is used to mesh video clips in the spatial dimension. It performs patch sampling within each video clip through a stereo sliding window to obtain video patch clips.

[0029] Extraction unit, used to extract latent features of fragment-level infinitesimal motion using the Video Swin Transformer model.

[0030] Preferably, the fitting module includes:

[0031] The first processing unit obtains frame-level multi-scale mean and standard deviation features using a ConvNext V2 model pre-trained on ImageNet. It introduces a global variance-guided attention mechanism to quantify the importance of the temporal standard deviation features of each channel at different time points, mapping them to channel attention weights. The channel attention weights are multiplied element-wise with the multi-scale standard deviation features to obtain the frame-level initial quality. The continuously sampled frame-level initial quality is then processed along the time axis using a combination of two one-dimensional convolutions and the GeLU activation function to obtain temporal attention weights. These temporal attention weights are then assigned to the frame-level initial quality to obtain the frame-level spatial explicit features.

[0032] The second processing unit is used to arrange the frame-level spatial explicit features along the time dimension to obtain the initial spatial explicit features of the frame stream level; and to perform adaptive pooling of the initial spatial explicit features of the frame stream level along the time dimension to obtain the frame stream level spatial explicit features of each video segment.

[0033] Preferably, the fusion module introduces learnable parameters by using the memory retention elements within each timestamp in the time hysteresis pool, and performs nonlinear multi-scale fitting through an exponential function.

[0034] The present invention also provides a video quality step-by-step evaluation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program performs a video quality step-by-step evaluation method when executed by the processor.

[0035] The present invention also provides a storage medium storing a computer program that executes a video quality step-by-step evaluation method when running.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] This invention fully simulates the progressive feedback mechanism of the human brain to extract spatiotemporal features of videos step by step, and overcomes the problems of insufficient expression of the human brain's visual mechanism and large differences between video evaluation quality and human subjective perception in existing technologies by integrating the first-level and second-level video quality through weight allocation. It also overcomes the problem of insufficient motion information extraction caused by the keyframe selection method in existing technologies by fitting the spatial explicit features of the sampled frame stream through an explicit feature aggregation pool. In the spatiotemporal domain modeling, it fully considers the human eye hysteresis effect and proposes an improved time hysteresis pool to overcome the problem of monotonous fitting scale in the modeling of motion visual stimulus perception in existing technologies. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0039] Figure 1 A schematic diagram of a spatiotemporally progressive feedback video quality assessment method provided by the present invention;

[0040] Figure 2 This is a schematic diagram of video segment patch sampling provided in an embodiment of the present invention;

[0041] Figure 3This is a scatter plot of the video quality prediction values ​​on the CVD2014 dataset, according to an embodiment of the present invention.

[0042] Figure 4 This is a scatter plot of the video quality prediction values ​​in the LIVE-Qualcomm dataset, according to an embodiment of the present invention.

[0043] Figure 5 This is a scatter plot of the video quality prediction values ​​in the KoNViD-1k dataset according to an embodiment of the present invention.

[0044] Figure 6 This is a scatter plot of the video quality prediction values ​​in the LIVE-VQC dataset according to an embodiment of the present invention.

[0045] Figure 7 This is a scatter plot of the predicted video quality values ​​in the YouTube-UGC dataset, as shown in this embodiment of the invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] Example 1:

[0049] like Figure 1 As shown, this embodiment of the invention provides a method for progressively evaluating video quality, including:

[0050] Step 1: Cut the original video into non-overlapping video clips, each containing an equal number of video frames;

[0051] Suppose that each video segment contains τ video frames. If the last video segment has fewer than τ frames, then the last τ video frames of the original video are extracted to form a complete video segment.

[0052] The original video can be represented as in, Let i be a video segment, and N be the index of the video segment. s This represents the number of video clips after editing.

[0053] Step 2: Perform patch sampling on the video clips to extract segment-level subtle motion implicit features;

[0054] Step 2.1 The video clips are processed into a grid in the spatial dimension. Patch sampling is performed within each video clip grid using a 3D sliding window to obtain video patch clips.

[0055] like Figure 2 As shown, firstly, all video frames within each video segment are divided into a 7×7 grid, with the grid size adapting to the video input resolution. Then, a fixed m×m×τ cube patch window is used, and sampling is performed randomly along the time dimension within each grid within each video segment, sampling once per grid. The sampled patches are then sequentially stitched together, maintaining the consistency of local sampling information temporal alignment without compromising the compositional position information of the video frames. This ensures that within each video segment... We obtain a (m×7)×(m×7)×τ video patch fragment, denoted as In this embodiment, m = 7 and τ = 32.

[0056] Step 2.2 Use the Video Swin Transformer model to extract implicit features of fragment-level infinitesimal motion.

[0057] The subtle motion latent features of video patch segments are extracted using a pre-trained Video Swin Transformer model, denoted as Motion.

[0058] Given a video patch fragment Using a Video Swin Transformer model pre-trained on the Something-Something V2 dataset, we extract the implicit motion features of the final stage of the output.

[0059]

[0060] Subsequently, Global Standard Deviation Pooling (GP) was applied. mean ) and Global Average Pooling (GP) std Implicit features of motion The pooling operation yields feature vectors denoted as follows:

[0061]

[0062]

[0063] Finally, along the channel dimension and Sequential splicing yields implicit features of fragment-level infinitesimal motion.

[0064]

[0065] Step 3: Sample several video frames consecutively within each video segment, extract the frame-level spatial explicit features of each video frame, and fit the frame-level spatial explicit features to obtain the frame stream-level spatial explicit features.

[0066] Step 3.1 Learn the feature variance distribution on the channel through a global variance-guided attention mechanism;

[0067] The ConvNext V2 model, pre-trained on ImageNet, is used as the frame-level spatial explicit feature extractor, denoted as Spatio. Specifically, in each video segment with frame number τ... The video is sampled continuously for τ / 2 frames, and the t-th video frame is denoted as I. t , t∈{1,2,…,τ / 2}. Let I... t As input to the ConvNext V2 model, the feature map of stage g, g∈{1,2,3,4} is obtained.

[0068]

[0069] Global average pooling (GP) is performed on the feature maps of different stages of video frame It. mean ) and Global Standard Deviation Pooling (GP) std ), to obtain the global mean and global variance

[0070]

[0071]

[0072] For global mean and global variance By splicing the data according to the channels, the multi-scale mean feature G of the t-th video frame is obtained. t and multi-scale standard deviation characteristics S t :

[0073]

[0074]

[0075] Calculate the global mean Temporal mean characteristics in the time dimension

[0076]

[0077] The time-domain standard deviation characteristics were calculated. The length of the eigenvector of the time-domain standard deviation is 2880.

[0078]

[0079] By introducing a nonlinear mapping to quantify the importance of each channel at different time points and mapping the temporal standard deviation features to attention weights, the model can learn the temporal standard deviation features more flexibly. The channel weights are obtained from the differential distribution across the channels.

[0080]

[0081] Channel weight With multi-scale standard deviation characteristics S t Multiplication yields the initial quality representation at the video frame level.

[0082]

[0083] Step 3.2 Fit the frame-level spatial explicit features to obtain the frame stream-level spatial explicit features.

[0084] To simulate the human eye's perception of continuous image details, two one-dimensional convolutions, denoted as Conv1 and Conv2, were used. These were combined with the GeLU activation function to obtain the temporal attention weight tensor. Then, the attention weights are applied using the hyperbolic tangent activation function. Map to the range (-1,1), then use the temporal attention weight tensor With frame-level initial quality Multiplication yields frame-level spatial explicit features.

[0085]

[0086]

[0087] Next, a video clip Explicit spatial features at the video frame level after internal sampling The initial spatial explicit features at the frame stream level are arranged sequentially along the time dimension. To effectively fuse temporal information at different scales, the initial spatial explicit features at the frame stream level are iteratively subjected to m adaptive pooling operations along the channel dimension of the time axis, with 2^m adaptive pooling layers. n-1 , n∈{1,2,…,m}. Finally, arranged sequentially along the time dimension, the frame-stream level spatial explicit features of each video segment are obtained. In this embodiment, m = 4.

[0088] Step 4: Fuse segment-level micro-variation motion implicit features and frame stream-level spatial explicit features within each video segment, perform temporal modeling through time hysteresis pooling, and perform quality regression to obtain the first level of video quality;

[0089] Step 4.1 Fusion of segment-level micro-variation motion implicit features and frame stream-level spatial explicit features;

[0090] Based on the channel dimension, the implicit features of fragment-level micro-variation motion are... and frame stream level spatial explicit features To splice:

[0091]

[0092] F i To obtain the fused segment-level spatiotemporal quality perception features, all segment-level spatiotemporal quality perception features are accumulated along the time dimension to obtain the video-level spatiotemporal quality perception features, denoted as... Subsequently, the video-level spatiotemporal quality perception features F are then processed through a fully connected layer FC3. o The feature dimension is mapped to 1, so that each frame after fusion has a primary quality score.

[0093]

[0094] Step 4.2 Improved time lag pool modeling;

[0095] To enrich the nonlinear multi-scale fitting to simulate the hysteresis effect of the human eye, a learnable parameter β, initially set to 1, and an exponential function mathematical model are introduced to increase the nonlinear fitting scale. Specifically, for 2δ+1 frames of images within a timestamp t∈(t-δ,t+δ), to update the quality score of the t-th frame, a memory retention element k is defined within t∈(t-δ,t-1). t Define a quality influence element p in t∈(t,t+δ). t In this embodiment, δ = 12.

[0096]

[0097] And t∈V pre

[0098] Among them, V pre ={max(1,t-δ),…,t-2,t-1}, It is defined as the minimum value in t∈(t-δ,t-1).

[0099]

[0100]

[0101] in, Depend on The learnable parameter β is obtained through continuous learning using the Sigmoid function. and The memory retention element k is obtained by nonlinear fitting using an exponential function. t .

[0102] Within the segment t∈(t,t+δ), the quality influence element p is defined by assigning a larger weight to frames with lower quality using the Softmin function. t :

[0103]

[0104]

[0105] in, w k This represents the weight of the k-th frame within the timestamp t∈(t,t+δ), and finally aggregates the quality influence elements p using an exponential function. t .

[0106] Step 4.3 First-level video quality regression;

[0107] After obtaining the memory retention element k t And the quality of element p t Then, the hyperparameter ρ is introduced to balance k. t and p t The quality score of the element for frame t Due to the influence of ρ, in this embodiment ρ = 0.5:

[0108]

[0109] renew The quality score of each video frame is calculated using average pooling. The quality scores of all video frames in the dataset are used to obtain the first-level video quality, Q1.

[0110]

[0111] Step 5: Statistically analyze the feature maps of different stages of the sampled video frames within all video segments, perform differential processing on the explicit features of the feature maps of consecutive video frames in the same stage, use the Transformer model to simulate the differences in inter-frame changes, and perform quality regression to obtain the second level of video quality.

[0112] Step 5.1 Perform differential processing on the explicit features of the feature maps of consecutive video frames.

[0113] Calculate the feature map of the t-th video frame in the g-th stage of the video segment. Multi-scale global mean features and global standard deviation features

[0114] The feature domain difference between the t-th frame and the (t-1)-th frame in the g-th stage was calculated.

[0115]

[0116] The average value of the feature domain difference between the t-th frame and the (t-1)-th frame in the g-th stage is calculated.

[0117]

[0118] The variance of the feature domain difference between the t-th frame and the (t-1)-th frame in the g-th stage was calculated.

[0119]

[0120] These are the covariance values ​​of the explicit features at different times. These are the correlation coefficients of explicit features at different timestamps. Due to the strong correlation between consecutive frames, the default... therefore, It can be simplified to:

[0121]

[0122]

[0123] The average difference of feature domains in the same stage along the feature dimension variance of the difference between the feature domains Sequentially concatenating the feature domains of frame t and frame t-1 at the same stage yields the difference.

[0124]

[0125] Similarly, the four stages Perform a stitching operation to obtain the difference explicit feature D between frame t and frame (t-1). t t∈{2,3,,…,T}

[0126]

[0127] By using a fully connected layer FC4, the difference between frame t and frame (t-1) becomes explicit. t The feature dimension is reduced to C2, which in this embodiment is C2 = 256, resulting in the explicit difference feature E between the t-th frame and the (t-1)-th frame after dimensionality reduction. t :

[0128] E t =FC4(D t )

[0129] Step 5.2 Global Modeling of Long Sequences Based on Transformer Model

[0130] Sine position coding is introduced to impart temporal information to the feature differences between consecutive video frames, where PE t Indicates position code, The explicit difference between frame t and frame (t-1) after position coding is represented as follows:

[0131]

[0132] Differential explicit features after position encoding The input is fed into a Transformer encoder with a feedforward network, specifically including: a layer normalization layer (LN), a multi-head self-attention module (MHA) with a masking mechanism, and a multilayer perceptron (MLP). This process is described in detail below:

[0133]

[0134]

[0135] Where d represents the encoder layer index, and the encoder has a total of L layers. These represent the intermediate tokens of the differential explicit features after position encoding at layer d. This represents the output embedding of layer d. The process is repeated for all L layers of the encoder to obtain the final output embedding. In this embodiment, L=5, the hidden dimension in MHA is set to 128, and the number of heads is set to 6.

[0136] The final output embedding is achieved through a multilayer perceptron (MLP). With a dimension of 1, the frame difference level quality prediction score is obtained.

[0137]

[0138] Step 5.3 Second-level video quality regression

[0139] Calculate the quality prediction score for all frame differences over time. The average value is used to obtain the second-level video quality Q2:

[0140]

[0141] Step 6: Use a weighted coefficient to combine the contributions of the first-level video quality and the second-level video quality to obtain the final video quality.

[0142] Q pred =λQ1+(1-λ)Q2

[0143] Where Q1 represents the first level of video quality, Q2 represents the second level of video quality, and Q... pred This is for the final video quality prediction. In this embodiment of the invention, λ = 0.5.

[0144] like Figures 3-7 As shown, the video quality stepwise evaluation method of this invention was validated on five public datasets: CVD2014, LIVE-Qualcomm, KoNViD-1k, LIVE-VQC, and YouTube-UGC. The test and training sets were randomly divided into 80% and 20% sets. The mini-batch size for training was 8, the initial learning rate was set to 0.001, the decay factor every two epochs was set to 0.4, the Adam optimizer was used to optimize the parameters, and the total number of epochs was set to 30. Spearman rank correlation coefficient (SRCC) and Pearson linear correlation coefficient (PLCC) were selected as evaluation metrics.

[0145] Example 2:

[0146] This invention also provides a video quality step-by-step evaluation device, comprising:

[0147] The editing module is used to cut the original video into non-overlapping video clips;

[0148] The extraction module is used to perform patch sampling on video segments and extract segment-level micro-variation motion implicit features;

[0149] The fitting module is used to continuously sample several video frames within each video segment, extract the frame-level spatial explicit features of each video frame, and fit the frame-level spatial explicit features to obtain the frame stream-level spatial explicit features.

[0150] The first processing module is used to fuse segment-level micro-variation motion implicit features and frame stream-level spatial explicit features within each video segment, and to perform temporal modeling through time hysteresis pooling and quality regression to obtain the first level of video quality.

[0151] The second processing module is used to statistically analyze the feature maps of different stages of the sampled video frames within all video segments, perform differential processing on the explicit features of the feature maps of consecutive video frames in the same stage, simulate the differences in inter-frame changes using the Transformer model, and perform quality regression to obtain the second level of video quality.

[0152] The fusion module is used to merge the first-level video quality and the second-level video quality to obtain the final video quality.

[0153] As one embodiment of the present invention, the extraction module includes:

[0154] The sampling unit is used to mesh video clips in the spatial dimension. It performs patch sampling within each video clip through a stereo sliding window to obtain video patch clips.

[0155] Extraction unit, used to extract latent features of fragment-level infinitesimal motion using the Video Swin Transformer model.

[0156] As one embodiment of the present invention, the fitting module includes:

[0157] The first processing unit obtains frame-level multi-scale mean and standard deviation features using a ConvNext V2 model pre-trained on ImageNet. It introduces a global variance-guided attention mechanism to quantify the importance of the temporal standard deviation features of each channel at different time points, mapping them to channel attention weights. The channel attention weights are multiplied element-wise with the multi-scale standard deviation features to obtain the frame-level initial quality. The continuously sampled frame-level initial quality is then processed along the time axis using a combination of two one-dimensional convolutions and the GeLU activation function to obtain temporal attention weights. These temporal attention weights are then assigned to the frame-level initial quality to obtain the frame-level spatial explicit features.

[0158] The second processing unit is used to arrange the frame-level spatial explicit features along the time dimension to obtain the initial spatial explicit features of the frame stream level; and to perform adaptive pooling of the initial spatial explicit features of the frame stream level along the time dimension to obtain the frame stream level spatial explicit features of each video segment.

[0159] As one embodiment of the present invention, the fusion module introduces learnable parameters into the memory retention elements within each timestamp in the time hysteresis pool, and performs nonlinear multi-scale fitting through an exponential function.

[0160] Example 3:

[0161] This invention also provides a video quality step-by-step evaluation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program performs a video quality step-by-step evaluation method when executed by the processor.

[0162] Example 4:

[0163] This invention also provides a storage medium storing a computer program that executes a video quality step-by-step evaluation method during runtime.

[0164] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for progressively evaluating video quality, characterized in that, include: Step 1: Edit the original video into non-overlapping video clips; Step 2: Perform patch sampling on the video clips to extract segment-level subtle motion implicit features; Step 3: Sample several video frames continuously within each video segment, extract the frame-level spatial explicit features of each video frame, and fit the frame-level spatial explicit features to obtain the frame stream-level spatial explicit features. Step 4: In each video segment, fuse segment-level micro-variation motion implicit features and frame stream-level spatial explicit features, and perform temporal modeling through time hysteresis pooling to obtain the first level of video quality through quality regression. Step 5: Statistically analyze the feature maps of different stages of the sampled video frames within all video segments. Perform differential processing on the explicit features of the feature maps of consecutive video frames in the same stage. Use the Transformer model to simulate the differences in inter-frame changes and perform quality regression to obtain the second level of video quality. Step 6: Combine the first-level video quality and the second-level video quality to obtain the final video quality; Step 2 includes: In the spatial dimension, video clips are processed into a grid, and patch sampling is performed within each video clip grid using a 3D sliding window to obtain video patch clips. The Video Swin Transformer model was used to extract implicit features of fragment-level infinitesimal motion. Step 3 includes: Frame-level multi-scale mean and standard deviation features are obtained by pre-training a ConvNext V2 model on ImageNet. A global variance-guided attention mechanism is introduced to quantify the importance of the temporal standard deviation features of each channel at different time points, which are then mapped to channel attention weights. The channel attention weights are multiplied element-wise with the multi-scale standard deviation features to obtain the initial frame quality. The continuously sampled initial frame quality is then combined with two one-dimensional convolutions and the GeLU activation function along the time axis to obtain temporal attention weights. The temporal attention weights are then assigned to the initial frame quality to obtain the explicit spatial features of the frame. Arrange the frame-level spatial explicit features along the time dimension to obtain the initial spatial explicit features of the frame stream; perform adaptive pooling of the initial spatial explicit features of the frame stream along the time dimension to obtain the frame stream spatial explicit features of each video segment. In step 4, learnable parameters are introduced into the memory retention elements within each timestamp in the time hysteresis pool, and nonlinear multi-scale fitting is performed through an exponential function. In step 3, the ConvNext V2 model pre-trained on ImageNet is used as a frame-level spatial explicit feature extractor, denoted as Spatio, for each video segment with frame number τ. The video is sampled continuously for τ / 2 frames, and the t-th video frame is denoted as I. t , t∈{1,2,…,τ / 2}; let I t As input to the ConvNext V2 model, the feature map of stage g is obtained. Where g∈{1,2,3,4}: For video frame I respectively t Global average pooling (GP) is performed on feature maps at different stages. mean and global standard deviation pooling GP std To obtain the global mean and global variance For global mean and global variance By splicing the data according to the channels, the multi-scale mean feature G of the t-th video frame is obtained. t and multi-scale standard deviation characteristics S t : Calculate the global mean Temporal mean characteristics in the time dimension The time-domain standard deviation characteristics were calculated. The length of the eigenvector of the time-domain standard deviation is 2880. By introducing a nonlinear mapping to quantify the importance of each channel at different time points and mapping the temporal standard deviation features to attention weights, the model can learn the temporal standard deviation features more flexibly. The channel weights are obtained from the differential distribution across the channels. Channel weight With multi-scale standard deviation characteristics S t Multiplication yields the initial quality representation at the video frame level. In step 4, for a total of 2δ+1 frames of images within a timestamp t∈(t-δ,t+δ), a memory retention element k is defined within t∈(t-δ,t-1). t Define a quality influence element p in t∈(t,t+δ). t , Among them, V pre ={max(1,t-δ),…,t-2,t-1}, in, Depend on The learnable parameter β is obtained through continuous learning using the Sigmoid function. and The memory retention element k is obtained by nonlinear fitting using an exponential function. t ; This is the initial quality score; Within the segment t∈(t,t+δ), the quality influence element p is defined by assigning a larger weight to frames with lower quality using the Softmin function. t : in, w k This represents the weight of the k-th frame within the timestamp t∈(t,t+δ), and finally aggregates the quality influence elements p using an exponential function. t N s This represents the number of video clips.

2. A video quality step-by-step evaluation apparatus for implementing the video quality step-by-step evaluation method of claim 1, characterized in that, include: The editing module is used to cut the original video into non-overlapping video clips; The extraction module is used to perform patch sampling on video segments and extract segment-level micro-variation motion implicit features; The fitting module is used to continuously sample several video frames within each video segment, extract the frame-level spatial explicit features of each video frame, and fit the frame-level spatial explicit features to obtain the frame stream-level spatial explicit features. The first processing module is used to fuse segment-level micro-variation motion implicit features and frame stream-level spatial explicit features within each video segment, and to perform temporal modeling through time hysteresis pooling and quality regression to obtain the first level of video quality. The second processing module is used to statistically analyze the feature maps of different stages of the sampled video frames within all video segments, perform differential processing on the explicit features of the feature maps of consecutive video frames in the same stage, simulate the differences in inter-frame changes using the Transformer model, and perform quality regression to obtain the second level of video quality. The fusion module is used to merge the first-level video quality and the second-level video quality to obtain the final video quality.

3. A video quality hierarchical evaluation system, characterized in that, include: A memory and a processor, wherein the memory stores a computer program executed by the processor, the computer program performing the video quality step-by-step evaluation method as described in claim 1 when executed by the processor.

4. A storage medium, characterized in that, The storage medium stores a computer program that, when executed, performs the video quality step-by-step evaluation method as described in claim 1.

Images