Video image quality evaluation method and device

Abstract

The invention relates to the technical field of video processing, and provides a video image quality evaluation method and device, and the method comprises the steps: carrying out the pyramid zooming of each video frame in a to-be-evaluated video frame sequence, and obtaining a plurality of pyramid images with different resolutions corresponding to each video frame; for each layer of image in the pyramid image corresponding to each video frame, uniformly sampling Gh * Gw image blocks with the resolution of fh * fw, and combining the image blocks to obtain a fragment image with the resolution of (Gh * fh) * (Gw * fw); performing space masking and time masking on the fragment images to obtain a target image sequence; and inputting the target image sequence into a video image quality evaluation model to obtain a score output by the video image quality evaluation model, the video image quality evaluation model being obtained by training based on a target image sequence sample and a score tag corresponding to the target image sequence sample. According to the method and the device, the ultrahigh-resolution video frame sequence to be tested is accurately evaluated, and the evaluation result conforms to subjective perception.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a method and apparatus for evaluating video quality. Background Technology

[0002] The Visual Networking System (VBS) is a new type of information infrastructure based on video communication technology, enabling the visual interconnection of everything. VBS provides a new type of information service across the entire video chain, encompassing video acquisition, transmission, storage, analysis, and presentation, for scenarios such as video surveillance, video conferencing, video communication, and digital twins. Video quality assessment is one of the key technologies for this end-to-end perception. Existing end-to-end quality monitoring solutions for VBS mainly involve collecting alarm information from cloud platforms or client players and monitoring the online / offline status of devices for end-to-end experience evaluation and fault analysis, but they do not provide evaluation of video image quality details.

[0003] As ultra-high-definition (UHD) technology has expanded from the digital cinema industry to high-definition video conferencing and video communication, 2K or 4K UHD cameras currently account for approximately 50% of mainstream surveillance cameras, with resolutions of 2560×1440 and 3840×2160, respectively. Existing AI models for video quality evaluation, due to their small input size, are primarily used for lower-resolution videos. When applied to evaluating the quality of ultra-high-resolution videos, their results differ significantly from subjectively perceived quality. Therefore, accurately and reliably evaluating the video quality of high-resolution or ultra-high-resolution videos is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a video quality evaluation method and apparatus to solve the aforementioned technical problems existing in the video evaluation methods of the prior art.

[0005] This application provides a method for evaluating video quality, including the following steps: Pyramid scaling is performed on each video frame in the video frame sequence to be evaluated, resulting in multiple pyramid images of different resolutions for each video frame. For each layer of the pyramid image corresponding to each video frame, uniform sampling is performed. G h × G w Each resolution is f h × f w The image blocks are merged to obtain a resolution of ( ). G h · f h )×( G w · fw (fragment image); Based on spatial and temporal masking, multiple segment images corresponding to each video frame in the video frame sequence to be evaluated are gradually masked from the highest resolution to the lowest resolution to obtain the target image sequence. The target image sequence is input into the video quality evaluation model to obtain the score output by the video quality evaluation model. The video quality evaluation model is trained based on the target image sequence samples and the score labels corresponding to the target image sequence samples.

[0006] According to the video quality evaluation method provided in this application, multiple segment images corresponding to each video frame in the video frame sequence to be evaluated are gradually masked from the highest resolution to the lowest resolution based on spatial masking and temporal masking to obtain a target image sequence, including: Based on binary masking, a masking operation is performed on the first segment image corresponding to the original resolution and at least one of the second segment images corresponding to the downsampled resolution in multiple segment images corresponding to each video frame to obtain at least one target image to form a target image sequence. The downsampled resolution corresponding to the second segment image of the video frame that is earlier in time is not less than the downsampled resolution corresponding to the second segment image of the video frame that is later in time. The downsampled resolution corresponding to the second segment image of the earliest video frame is the largest, and the downsampled resolution corresponding to the second segment image of the last video frame is the smallest.

[0007] According to a video quality evaluation method provided in this application, a masking operation is performed on a first segment image corresponding to the original resolution and at least one second segment image corresponding to a downsampled resolution in multiple segment images corresponding to each video frame, based on a binary mask, to obtain at least one target image, thereby forming a target image sequence, including: Based on binary masks, the first segment image corresponding to the original resolution and the two second segment images corresponding to the downsampled resolutions in the multiple segment images corresponding to each video frame are masked to obtain the two target images.

[0008] According to the video quality evaluation method provided in this application, after inputting the target image sequence into the video quality evaluation model and obtaining the score output by the video quality evaluation model, the method further includes: An alarm is triggered if the scores for a consecutive preset number of frames are all below the score threshold.

[0009] According to the video quality evaluation method provided in this application, after inputting the target image sequence into the video quality evaluation model and obtaining the score output by the video quality evaluation model, the method further includes: Step S1: When the scores corresponding to a consecutive preset number of frames are all lower than the score threshold, traverse the parameter tuning strategy library to obtain multiple historical adjustment strategies, historical network state data before the adjustment of the video network parameters, and the confidence level of each historical adjustment strategy. The historical adjustment strategy represents the correspondence between the adjustment amount of any parameter of the video network and the increase in score after adjustment. Step S2: Determine the network state similarity based on the historical network state data before parameter adjustment and the current network state data; Step S3: Based on the network state similarity, the confidence of historical adjustment strategies, and the parameter adjustment amount and score increase amount in any historical adjustment strategy, determine the quality relevance corresponding to any historical adjustment strategy; Step S4: Select the historical adjustment strategy corresponding to the maximum quality correlation as the current adjustment strategy; Step S5: Adjust the visual network according to the current adjustment strategy.

[0010] According to the video quality evaluation method provided in this application, after adjusting the video network with the current adjustment strategy, it further includes: If the adjusted real-time score increases, the parameter adjustment amount of the current adjustment strategy, the increase in the adjusted real-time score, the network status data before the current adjustment, and the confidence level of the current adjustment strategy are written into the parameter tuning strategy library.

[0011] According to the video quality evaluation method provided in this application, after adjusting the video network with the current adjustment strategy, it further includes: If the adjusted real-time score does not increase, update the confidence level of the historical adjustment strategy corresponding to the maximum quality correlation, and repeat steps S1 to S5 until the adjusted real-time score increases.

[0012] According to the video quality evaluation method provided in this application, the video quality evaluation model includes: The feature extraction module is used to extract the feature sequence of the target image sequence; The quality regression module is used to map the feature sequence to the score and output it.

[0013] According to the video quality evaluation method provided in this application, before performing pyramid scaling on each video frame in the video frame sequence to be evaluated, the method further includes: Receive the raw video stream and convert it into a video stream in a predetermined protocol format; Decode the converted video stream and extract multiple keyframes; The key frames are combined in chronological order to form the video frame sequence to be evaluated.

[0014] According to the video quality evaluation method provided in this application, before combining multiple key frames into the video frame sequence to be evaluated, the method further includes: A Laplacian sharpening filter is applied to each keyframe to obtain the filtered keyframe.

[0015] This application also provides a video quality evaluation device, including the following units: The pyramid scaling unit is used to perform pyramid scaling on each video frame in the video frame sequence to be evaluated, so as to obtain multiple pyramid images of different resolutions for each video frame. The segment image sampling unit is used to uniformly sample each layer of the pyramid image corresponding to each video frame. G h × G w Each resolution is f h × f w The image blocks are merged to obtain a resolution of ( ). G h · f h )×( G w · f w (fragment image); The spatiotemporal masking unit is used to perform masking processing on multiple segment images corresponding to each video frame in the video frame sequence to be evaluated from the highest resolution to the lowest resolution based on spatial masking and temporal masking, so as to obtain the target image sequence. The model execution unit is used to input the target image sequence into the video quality evaluation model and obtain the score output by the video quality evaluation model. The video quality evaluation model is trained based on the target image sequence samples and the score labels corresponding to the target image sequence samples.

[0016] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the video quality evaluation method as described above.

[0017] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video quality evaluation method as described above.

[0018] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the video quality evaluation method as described above.

[0019] The video quality evaluation method and apparatus provided in this application construct a multi-scale pyramid image by scaling each video frame in the video frame sequence to be tested. Then, a segment sampling method is used to extract the local content of the pyramid image at each scale to form a segment image. Then, based on spatial and temporal masks, multiple segment images corresponding to each video frame in the video frame sequence to be evaluated are gradually masked from the highest resolution to the lowest resolution to obtain the target image sequence for input video quality evaluation model. Finally, the video quality evaluation model is evaluated using this target image sequence. This target image sequence not only adapts to the input size of the subsequent video quality evaluation model, but also preserves the local details and global semantic information in the image, thereby achieving accurate evaluation of the ultra-high resolution video frame sequence to be tested, and the evaluation result conforms to subjective perception. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is one of the flowcharts illustrating the video quality evaluation method provided in this application.

[0022] Figure 2 This is a schematic diagram illustrating the principle of the video quality evaluation method provided in this application.

[0023] Figure 3 yes Figure 2 Enlarged schematic diagram of (b) in the middle.

[0024] Figure 4 This is the second flowchart of the video quality evaluation method provided in this application.

[0025] Figure 5 This is a schematic diagram of the video quality evaluation device provided in this application.

[0026] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The video quality evaluation method in this application embodiment, such as Figure 1 As shown, it includes steps S110 to S140.

[0029] Step S110: Perform pyramid scaling on each video frame in the video frame sequence to be evaluated to obtain multiple pyramid images of different resolutions corresponding to each video frame.

[0030] For ultra-high-definition video streams, the resolution of each video frame in the corresponding video frame sequence to be evaluated is high, such as 2K or 4K. The resolution is much larger than the resolution of the input image of the video quality evaluation model, making it difficult for the input size of the model's pre-trained weights to cover the entire screen. Usually, it is necessary to crop or scale the ultra-high resolution image, but it is difficult to take into account both the local details and global semantic information in the video frame.

[0031] For video frames x The pyramid, after being scaled down, forms a shape like... Figure 2 Image (a) represents the pyramid image. p x , p x ={ x 0, x 1,…, x n}, where the first layer image x 0 resolution and video frames x Same, image x i (i takes values ​​from 1 to n) is obtained through bilinear interpolation. x The image is downsampled, but the aspect ratio is still maintained. p x middle x i The resolution from x 0 to x n Linear reduction. In this step, multi-scale pyramid images are sampled to preserve more data information from local to global perspectives. High-resolution images preserve more local details, while lower-resolution images preserve global semantic information.

[0032] Step S120: For each layer of the pyramid image corresponding to each video frame, uniformly sample... G h × G w Each resolution is f h × f w The image blocks are merged to obtain a resolution of ( ).G h · f h )×( G w · f w () fragment images.

[0033] like Figure 2 (b) and Figure 3 As shown, the pyramid image is divided into layers. G h × G w The grid of the lower image is larger than that of the upper image. For each grid in each image layer, the random sampling resolution is... f h × f w Image blocks ( Figure 3 The yellow image patch in the middle), merging the image patches to obtain a resolution of ( G h · f h )×( G w · f w (Image fragments, such as) Figure 2 As shown in (c), each layer of the pyramid image corresponds to a segment image. That is, based on the multi-scale pyramid image, segment sampling is used to extract content, and pyramid images of different resolutions are converted to a fixed resolution size. G h · f h )×( G w · f w The representation of ) is used to construct a fragment pyramid image. : .

[0034] From x i The sampled image segments can be understood as follows: although each They have the same resolution, that is ( G h · f h )×( G w · f w ), but different xi The segments are sampled from different scales in a multi-scale pyramid image, therefore the segments It can also be viewed as a pyramid.

[0035] in, G h , G w , f h and f w The resolution of the input image can be determined based on the resolution of the subsequent video quality evaluation model. For example, with an input size of 224×224 resolution, the grid can be a 7×7 grid, and each image block is a 32×32 image block.

[0036] In this step, images of different resolutions in the pyramid image are generated by segment sampling to obtain images of different resolutions ( ). G h · f h )×( G w · f w The fragmented images not only adapt to the input size of subsequent video quality evaluation models, but also preserve local details and global semantic information in the images. Furthermore, due to grid partitioning, the spatial content is uniformly distributed. In addition, the Transformer-based backbone architecture can be used to eliminate discontinuities between adjacent blocks when input into the subsequent video quality evaluation model.

[0037] It should be noted that the width and height of the input image for video quality evaluation models are usually equal; therefore, the final segment image... The resolution is m×m, m=min( G h · f h , G w · f w ).

[0038] Step S130: Based on spatial and temporal masks, the multiple segment images corresponding to each video frame in the video frame sequence to be evaluated are progressively masked from the highest resolution to the lowest resolution to obtain the target image sequence. By masking and rearranging the multi-scale data into a regular shape through spatial and temporal masks, local details and global semantics are preserved in the regular data shape, so that the subsequent model processing can take into account both local distortion and semantic information, and obtain more accurate evaluation results.

[0039] Step S140: Input the target image sequence into the video quality evaluation model to obtain the score output by the video quality evaluation model. The video quality evaluation model is trained based on the target image sequence samples and the corresponding score labels. The score labels can be the scores given to the target image sequence samples by experts in the video field.

[0040] In the video quality evaluation method of this embodiment, a multi-scale pyramid image is constructed by scaling each video frame in the video frame sequence to be tested. Then, a segment sampling method is used to extract the local content of the pyramid image at each scale to form a segment image. Then, based on spatial masking and temporal masking, multiple segment images corresponding to each video frame in the video frame sequence to be evaluated are gradually masked from the highest resolution to the lowest resolution to obtain a target image sequence, which is then used as the target image sequence for input to the video quality evaluation model. Finally, the video quality evaluation model is evaluated using this target image sequence. This target image sequence not only adapts to the input size of the subsequent video quality evaluation model, but also preserves the local details and global semantic information in the image, thereby achieving accurate evaluation of the ultra-high resolution video frame sequence to be tested, and the evaluation result conforms to subjective perception.

[0041] In some embodiments, step S130 specifically includes: performing a masking operation on the first segment image corresponding to the original resolution and at least one of the second segment images corresponding to the downsampled resolution of the multiple segment images corresponding to each video frame based on a binary mask, to obtain at least one target image to form a target image sequence, wherein the downsampled resolution corresponding to the second segment image of the earlier video frame is not less than the downsampled resolution corresponding to the second segment image of the later video frame, the downsampled resolution corresponding to the second segment image of the earliest video frame is the largest, and the downsampled resolution corresponding to the second segment image of the last video frame is the smallest.

[0042] Since the fragment pyramid image increases the amount of data input to the model from S video frames in the test video frame sequence to (n+1)×S, directly feeding the fragment pyramid image into the video quality evaluation model would result in a significant increase in complexity, which does not meet the real-time video quality evaluation requirements of video network services. Therefore, a masking strategy is introduced to reduce the number of fragment pyramid images to a size close to or equal to the conventional input size of S.

[0043] In this embodiment, firstly, based on a binary mask, the first segment image (i.e., the original resolution of the multiple segment images corresponding to each video frame) is processed. ) and at least one of the downsampled resolutions corresponding to the second segment image (i.e. Perform a masking operation to obtain at least one target image. Each video frame corresponds to one target image, thus obtaining a target image sequence.

[0044] The mask operation formula is as follows: M It is a spatial binary mask. M A value of 1 indicates that the first segment of the image corresponding to the original resolution is taken. , M A value of 0 indicates that the second segment of the image corresponding to the sampling resolution is used. . Specifically, M Black / white areas are used to represent different scales, which are interspersed. See Figure 2 The chessboard mask, where the black positions correspond to the grid from... The sampled fragment, while the white indicator is from Sampling fragments.

[0045] Figure 2 (d) gives the image at its original resolution. and 1 / 8 downsampling resolution Spatial masking results. From Sampling fragments focus more on local details (such as blockiness and texture blurring) but struggle to infer global semantics. The sampled fragments contain more global information (such as scene structure and motion coherence) but lack local distortion information. In this step, a spatial masking strategy is used to mask and rearrange the multi-scale data into a regular shape, preserving local details and global semantics within the regular data shape.

[0046] Secondly, in the time dimension, video frames not only retain spatial correlation but also rich temporal redundancy. Therefore, a temporal mask needs to be introduced. Specifically, the downsampling resolution of the second segment image of the earlier video frame is no less than that of the second segment image of the later video frame. The downsampling resolution of the second segment image of the earliest video frame is the largest, and the downsampling resolution of the second segment image of the latest video frame is the smallest. That is, the masking process is performed step by step along the time dimension from the highest resolution to the lowest resolution.

[0047] For example: For the first video frame in the video frame sequence to be tested, select the first image segment. Second fragment image Perform a masking operation; for the last video frame, select the first segment image. Second fragment image Perform a masking operation, selecting the first segment of the image from the middle video frames. Second fragment image Perform a masking operation, ensuring that the downsampling resolution of the second segment image of the earlier video frame is not less than the downsampling resolution of the second segment image of the later video frame.

[0048] like Figure 2 As shown in (e), by using the above spatial and temporal masking processes, the number of fragment pyramid images is reduced to the normal input size of S, which reduces the computational complexity of the subsequent video quality evaluation model. Furthermore, the generated target image sequence retains the local details and global semantic information of the video frame sequence to be evaluated, enabling more accurate scoring of the video frame sequence to be evaluated, and making the scoring results highly consistent with subjective perception.

[0049] In some embodiments, a masking operation is performed on a first segment image corresponding to the original resolution and at least one second segment image corresponding to a downsampled resolution in a plurality of segment images corresponding to each video frame based on a binary mask to obtain at least one target image to form a target image sequence, including: performing a masking operation on a first segment image corresponding to the original resolution and two second segment images corresponding to downsampled resolutions in a plurality of segment images corresponding to each video frame based on a binary mask to obtain two target images.

[0050] For example: For the first video frame in the video frame sequence to be tested, select the first image segment. Compared with the second segment image respectively and Perform a masking operation to obtain two target images. For the last video frame, select the first segment image. Compared with the second segment image respectively and Perform a masking operation to obtain two target images. For the middle video frame, select the first segment image. Compared with the second segment image respectively and A masking operation is performed to obtain two target images, where the downsampling resolution of the second segment image from the earlier video frame is not less than the downsampling resolution of the second segment image from the later video frame. The final target image sequence is as follows: .

[0051] and This represents the two target images corresponding to the first video frame. and This represents the two target images corresponding to the last video frame. There are a total of 2S target images in the target image sequence.

[0052] In this embodiment, two target images are generated for each video frame in the video frame sequence to be tested, containing richer local details and global semantic information. This allows for a more accurate scoring of the video frame sequence to be tested while ensuring that the video quality evaluation model's computing power can handle the task.

[0053] In some embodiments, after inputting the target image sequence into the video quality evaluation model in step S140 and obtaining the score output by the video quality evaluation model, the method further includes: triggering an alarm if the scores corresponding to a consecutive preset number of frames are all lower than the score threshold.

[0054] Specifically, the number of consecutive preset frames and the scoring threshold can be set according to the actual situation. For example, if the score is 3.5 for 5 consecutive frames, the video quality is considered to be low, thus triggering an alarm. After receiving the alarm, the staff will conduct corresponding troubleshooting and optimization measures on the video network to improve the video quality.

[0055] In some embodiments, after step S140, where the target image sequence is input into the video quality evaluation model to obtain the score output by the video quality evaluation model, the method further includes: Step S141: When the scores corresponding to a preset number of consecutive frames are all lower than the score threshold, traverse the parameter tuning strategy library to obtain multiple historical adjustment strategies, historical network state data before the adjustment of the video network parameters, and the confidence level of each historical adjustment strategy. The historical adjustment strategy represents the correspondence between the adjustment amount of any parameter of the video network and the increase in score after adjustment.

[0056] The preset number of consecutive frames and the scoring threshold can be set according to actual conditions. For example, with 5 consecutive frames and a scoring threshold of 3.5, the video quality is considered low if the score is below 3.5 for 5 consecutive video frames. Then, the parameter tuning strategy library is traversed to obtain multiple historical adjustment strategies, historical network state data before the video network parameter adjustment, and the confidence level of each historical adjustment strategy. The historical adjustment strategy represents the correspondence between any parameter adjustment amount of the video network and the increase in score after the adjustment. For example, a 2Mbps increase in bitrate can improve the score MOS by 0.3 points; therefore, the bitrate adjustment amount of 2 Mbps and the score MOS increase of 0.3 are in one-to-one correspondence, and adjusting the bitrate is the adjustment strategy. The parameter tuning strategy library stores multiple historical adjustment strategies, historical network state data before the video network parameter adjustment, and the confidence level of each historical adjustment strategy.

[0057] Step S142: Determine the network state similarity based on the historical network state data before parameter adjustment and the current network state data. S networkIn this step, for each historical adjustment strategy, it is necessary to determine the similarity between the current network state and the corresponding historical network state to obtain the network state similarity. S network For example, network state data of the visual network, taking packet loss rate and latency as examples, includes network state similarity. S network The calculation formula is as follows: (1).

[0058] in, PLR This indicates the current packet loss rate. PLR ref This represents the packet loss rate before parameter adjustment for any historical adjustment strategy. Latency Indicates the current delay. Latency ref The parameter before adjustment is represented by any historical adjustment strategy. In the above formula (1), the denominator represents the historical extreme value, and represents the maximum possible difference in network state (packet loss rate and delay), which is used to scale the actual difference to the [0,1] interval.

[0059] Calculate the similarity between the current network state and each historical network state in the parameter tuning strategy library according to the above formula (1) to obtain the similarity of multiple network states. S network .

[0060] Step S143: Based on the network state similarity, the confidence level of historical adjustment strategies, and the parameter adjustment amount and score increase amount in any historical adjustment strategy, determine the quality relevance corresponding to any historical adjustment strategy. For example, the quality relevance... R The calculation formula is as follows: (2).

[0061] In formula (2), the first term is the MOS efficiency, and the weighting coefficient is... α The default value is 0.5; the second item is network state similarity, with a weight coefficient of [value missing]. β The default value is 0.3; the third item is the confidence level of the historical adjustment strategy, with a weighting coefficient of 0.3. γ The default value is 0.2.

[0062] The first option prioritizes strategies that yield the greatest MOS improvement through unit parameter adjustments. Δ MOS Δ represents the increase in MOS (Mean Orientation) of any historical tuning strategy recorded in the parameter tuning strategy library (e.g., MOS + 0.3). P Δ represents the parameter adjustment magnitude (e.g., bitrate +2Mbps) corresponding to the increase in the score MOS in any historical adjustment strategy recorded in the parameter tuning strategy library. MOS / ΔP This indicates the efficiency of the effect of adjusting the video network parameters on the scoring MOS.

[0063] The second option indicates that historical adjustment strategies that are close to the current network state should be prioritized.

[0064] The third item indicates the preference for using validated, high-confidence historical adjustment strategies. For example, the confidence level of the historical adjustment strategy... C historical The calculation formula is as follows: .

[0065] in, N success This indicates that the historical adjustment strategy has successfully increased the number of times the score MOS has been improved. N total This indicates the total number of times the historical adjustment strategy has been invoked. N success and N total All of these can be obtained through statistics. The larger the value, the more the historical adjustment strategy has been applied.

[0066] Step S144: Select the historical adjustment strategy corresponding to the maximum quality correlation as the current adjustment strategy. Substitute the historical adjustment strategy of each record in the parameter tuning strategy library, the historical network state data before the video network parameter adjustment, the confidence level of each historical adjustment strategy, and the current network state data into the above formula (2) to obtain the quality correlation corresponding to each record. R , R The larger the value, the better the video quality of the video network under the current network conditions is improved by adopting the corresponding historical adjustment strategy.

[0067] Step S145: Adjust the visual network according to the current adjustment strategy, for example: R The larger the value, the higher the historical adjustment strategy is to increase the bitrate by 2Mbps, and the current adjustment strategy is also to increase the bitrate by 2Mbps.

[0068] In this embodiment, when the scores corresponding to a preset number of consecutive frames are all lower than the score threshold, the above steps S141 to S145 are used to implement a decision-making scheme for low video quality in the video network, that is, to realize the function of automatically adjusting the video network parameters to improve video quality.

[0069] It is understandable that for different parameters of the video network (e.g., bitrate, frame rate), multiple corresponding parameters can be calculated according to the above formula (2). R Value, select the largest. RThe historical adjustment strategies corresponding to the values ​​are used to form a combined parameter adjustment strategy, such as adjusting the bitrate and frame rate at the same time to improve the video quality score.

[0070] In some embodiments, after adjusting the visual network with the current adjustment strategy, the method further includes: if the real-time score increases after adjustment, writing the parameter adjustment amount of the current adjustment strategy, the increase in the real-time score after adjustment, the network status data before the current adjustment, and the confidence level of the current adjustment strategy into the parameter tuning strategy library.

[0071] Since the current network state of the visual network in formula (2) is not exactly the same as the historical network state, and even if the corresponding parameters of the visual network are adjusted according to the same historical adjustment strategy, the increase in score may not be exactly the same as the increase in score in the historical adjustment strategy. For example, after the bitrate increases by 2 Mbps, the increase in real-time score is 0.25 or 0.4. Therefore, in this embodiment, when the real-time score increases after adjustment, the parameter adjustment amount of the current adjustment strategy, the increase in real-time score after adjustment, the network state data before the current adjustment, and the confidence level of the current adjustment strategy are written into the parameter tuning strategy library to enrich the parameter tuning strategy library and facilitate the matching of more accurate adjustment strategies in the future.

[0072] In some embodiments, after adjusting the visual network with the current adjustment strategy, the method further includes: if the adjusted real-time score does not increase, updating the confidence level of the historical adjustment strategy corresponding to the maximum quality correlation (i.e., reducing the confidence level), and re-executing steps S141 to S145 until the adjusted real-time score increases.

[0073] Since the current network state of the visual network in the above formula (2) is not exactly the same as the historical network state, and even if the corresponding parameters of the visual network are adjusted according to the same historical adjustment strategy, the real-time score may remain unchanged or decrease. Obviously, the historical adjustment strategy selected this time has not successfully improved the score. Update the confidence of the historical adjustment strategy and re-execute steps S141 to S145 until the adjusted real-time score increases.

[0074] In some embodiments, the video quality evaluation model includes: The feature extraction module is used to extract feature sequences from the target image sequence. For example, the feature extraction module can be a backbone network pre-trained with a large-scale video quality assessment dataset such as Kinetics-400 or LSVQ, or other pre-trained weight models such as ResNet-50, as the feature extractor. Compared with traditional classification models, because its input undergoes special spatiotemporal masking processing, the model simultaneously takes into account local distortion and semantic information.

[0075] The quality regression module is used to map the feature sequence to the score and output it. For example, two multilayer perceptrons (MLPs) are designed to compress the features and the output MOS is mapped to the range [0,1] through sigmoid, that is, the multi-dimensional feature sequence output by Backbone is mapped to the standardized quality score.

[0076] Furthermore, the output can be adjusted using linear or nonlinear transformations (such as Support Vector Machine Regression, SVR) to obtain the decoded video quality score. MOS final The range is [0-5], to conform to the range standard of manual scoring. An example linear transformation function formula is as follows: MOS final =5× MOS + bias .

[0077] The bias can be adjusted based on practical experience.

[0078] It should be noted that since each video frame in the video frame sequence to be evaluated has undergone pyramid scaling and segment sampling, the final target image meets the input requirements of the feature extraction module. Therefore, when training the video quality evaluation model, the model parameters of the feature extraction module can be fixed, and only the model parameters of the quality regression module can be updated to improve the model training efficiency.

[0079] In some embodiments, prior to pyramid scaling of each video frame in the sequence of video frames to be evaluated, the method further includes: Step 1: Receive the raw video stream and convert it into a video stream of a predetermined protocol format. Since different video acquisition devices in a video network have different video stream protocol formats, parallel access of the video network's proprietary protocol and standard streaming media protocols (RTSP / RTP / ONVIF) is supported. The video network's proprietary interface can implement low-level signal parsing of the video stream through hardware logic chips; the standard network video interface supports RTP / RTSP over the IP protocol stack.

[0080] For example, the implementation of converting the raw video stream into a video stream of a predetermined protocol format (such as a standard streaming media protocol) is as follows: By integrating a video network private protocol parser, this parser extracts the timestamp field (8 bytes long) and stream identifier (4 bytes long) from the protocol header, changes the packet header, and reassembles the private protocol data packets into a standard RTP packet structure. This can unify the data processing pipeline, so that the subsequent video decoding process does not need to adapt to multiple protocols and can directly use standard hardware decoding, simplifying the processing flow.

[0081] Step 2: Decode the converted video stream and extract multiple keyframes. For example, an FPGA chip can be used to implement hardware decoding of the H.265 / AV1 video stream, supporting 10-bit color depth and 4K@120fps real-time decoding. Furthermore, during hardware decoding, spatiotemporal synchronization and packet loss compensation are performed simultaneously to ensure the integrity and timing consistency of the input video stream.

[0082] Alternatively, a pre-trained end-to-end encoding / decoding method based on AI neural networks can be used to decode the converted video stream.

[0083] By real-time parsing of the NAL unit header information of the video bitstream (such as slice_type==I, first_mb_in_slice==0, etc. in H.265), the I-frame position is quickly located. From the entire set of I-frames in the video, multiple I-frames (e.g., 8 frames) are extracted evenly distributed over time. Specifically, based on the original timestamp of the video, frame capture is triggered every time window ΔT = acquisition duration / 8 (minimum precision 0.1ms), and 8 keyframes are extracted.

[0084] Step 3: Combine the multiple key frames in chronological order to form the video frame sequence to be evaluated.

[0085] In this embodiment, by converting the original video stream into a predetermined protocol format, and decoding and extracting frames from the converted video stream, online video quality scoring of ultra-high-definition video streams in multiple protocol formats is achieved.

[0086] In some embodiments, before combining multiple key frames into the video frame sequence to be evaluated, the method further includes: inputting a Laplacian sharpening filter into each key frame to obtain filtered key frames. In this embodiment, Laplacian sharpening filtering is applied to the extracted multiple key frames. The Laplacian sharpening filter enhances the edges and compensates for blur in each key frame, improving the quality of the extracted key frames to ensure a more accurate and objective score in the subsequent process.

[0087] like Figure 4 The diagram shown illustrates the process of applying the video quality evaluation method of this application to a video network, realizing a closed-loop management process from multi-source video access to intelligent quality control.

[0088] For example, in traffic monitoring scenarios, the system needs to simultaneously process mixed video streams from proprietary video network protocols and standard streaming media protocols. Multiple intersections receive 4K high-definition camera video inputs (3840×2160) using different protocols. After protocol conversion, a unified RTP stream is output to the preprocessing layer. The FPGA decoding chip then processes the two video streams in parallel, extracting 8 keyframes from each stream at equal time intervals to create a sequence of video frames to be evaluated.

[0089] The following steps are performed during the quality assessment phase: (a) Spatial dimension adopts fragment sampling: construct a 4-level resolution pyramid (original resolution → 1 / 2 downsampling → 1 / 4 downsampling → 1 / 8 downsampling), and sample 32×32 fragments through a 7×7 grid, that is, perform the above steps S110 to S120.

[0090] (b) Spatial masking: Based on binary masking, the first segment image corresponding to the original resolution and the second segment image corresponding to two downsampled resolutions in the multiple segment images corresponding to each video frame are masked to interleave local details and global semantics; the temporal dimension adopts progressive temporal masking: the downsampled resolution of the second segment image of the video frame that is earlier in time is not less than the downsampled resolution of the second segment image of the video frame that is later in time, the downsampled resolution of the second segment image of the video frame that is the earliest in time is the largest, and the downsampled resolution of the second segment image of the video frame that is the last in time is the smallest, so as to form a target image with 16 temporal scales in 32 (4×8, 4 resolutions, 8 keyframes) frame inputs.

[0091] (c) Pre-trained weighted feature inference: The pre-trained Swin-Transformer model (e.g., Video Swin-Tiny) receives the target image sequence after spatiotemporal masking and outputs the initial multidimensional feature representation.

[0092] (d) Quality regression: The results are mapped to a score range of 0-5 MOS by two layers of MLP and Sigmoid normalization, and then by linear transformation.

[0093] The final results show that on the LSVQ dataset, the Pearson linear correlation coefficient (PLCC) and Spearman order correlation coefficient (SRCC) of this method are 0.8144 and 0.8293, respectively. Compared with the PLCC and SRCC obtained by directly resizing or cropping the original image, which are 0.7910 and 0.8025, the accuracy and monotonicity are improved.

[0094] The following steps are performed during the alarm output phase: (a) A continuous decrease in MOS was detected by a sliding window of 5 frames, triggering an alarm.

[0095] (b) Retrieve matching strategies from the parameter tuning strategy library: A 2Mbps increase in bitrate results in a 0.3 MOS increase, a network state similarity of 0.85, and a confidence level of 0.92 for historical adjustment strategies; A 5fps decrease in frame rate results in a 0.1 MOS decrease, a network state similarity of 0.78, and a confidence level of 0.88 for historical adjustment strategies.

[0096] (c) Calculate the optimal solution for these matching strategies according to the quality correlation formula. Taking the data in the example as an example: R=0.5×(0.3 / 2)+0.3×0.85+0.2×0.92=0.619.

[0097] (d) Generate a hybrid strategy: increase the bit rate by 2Mbps and issue execution instructions, such as adjusting encoder parameters in real time (adjusting the bit rate) or triggering transport layer packet loss detection.

[0098] (e) Perform model inference on the video quality after the operation to obtain the MOS score result after the operation.

[0099] (f) If MOS recovers, add the strategy to the database and update the confidence level.

[0100] (g) If the MOS does not change or decreases, update the confidence of the matched historical adjustment strategy in the parameter tuning strategy library, and jump to step (b) to rematch the historical adjustment strategy until the score recovers.

[0101] The video quality evaluation apparatus provided in this application is described below. The video quality evaluation apparatus described below can be referred to in correspondence with the video quality evaluation method described above.

[0102] The video quality evaluation device of this application, such as Figure 5 As shown, it includes: The pyramid scaling unit 510 is used to perform pyramid scaling on each video frame in the video frame sequence to be evaluated, so as to obtain multiple pyramid images of different resolutions corresponding to each video frame.

[0103] The segment image sampling unit 520 is used to uniformly sample each layer of the pyramid image corresponding to each video frame. G h × G w Each resolution is f h × f w The image blocks are merged to obtain a resolution of ( ). Gh · f h )×( G w · f w () fragment images.

[0104] The spatiotemporal masking unit 530 is used to perform masking processing on multiple segment images corresponding to each video frame in the video frame sequence to be evaluated from the highest resolution to the lowest resolution based on spatial masking and temporal masking, so as to obtain the target image sequence.

[0105] The model execution unit 540 is used to input the target image sequence into the video quality evaluation model to obtain the score output by the video quality evaluation model. The video quality evaluation model is trained based on the target image sequence samples and the score labels corresponding to the target image sequence samples.

[0106] The video quality evaluation device in this embodiment constructs a multi-scale pyramid image by scaling each video frame in the video frame sequence to be tested. Then, it uses a segment sampling method to extract local content of the pyramid image at each scale to form a segment image. Next, based on spatial and temporal masks, it performs masking processing on multiple segment images corresponding to each video frame in the video frame sequence to be evaluated, from the highest resolution to the lowest resolution, to obtain a target image sequence. This target image sequence is then used to input the video quality evaluation model. Finally, the video quality evaluation model evaluates the model using this target image sequence. This target image sequence not only adapts to the input size of the subsequent video quality evaluation model but also preserves local details and global semantic information in the image, thereby achieving accurate evaluation of the ultra-high resolution video frame sequence to be tested, and the evaluation result conforms to subjective perception.

[0107] In some embodiments, the spatiotemporal masking unit 530 is specifically used to perform a masking operation on the first segment image corresponding to the original resolution and at least one of the second segment images corresponding to the downsampled resolution of the multiple segment images corresponding to each video frame based on a binary mask, so as to obtain at least one target image to form a target image sequence, and the downsampled resolution corresponding to the second segment image of the video frame that is earlier in time is not less than the downsampled resolution corresponding to the second segment image of the video frame that is later in time, the downsampled resolution corresponding to the second segment image of the video frame that is earliest in time is the largest, and the downsampled resolution corresponding to the second segment image of the video frame that is latest in time is the smallest.

[0108] In some embodiments, the spatiotemporal masking unit 530 is specifically used to perform masking operations on the first segment image corresponding to the original resolution and the two second segment images corresponding to the downsampled resolution in the multiple segment images corresponding to each video frame based on a binary mask, so as to obtain two target images.

[0109] In some embodiments, the video quality evaluation device further includes an alarm unit, configured to trigger an alarm when the scores corresponding to a consecutive preset number of frames are all lower than a score threshold after the target image sequence is input into the video quality evaluation model and the score output by the video quality evaluation model is obtained.

[0110] In some embodiments, the video quality evaluation device further includes: a video network parameter adjustment unit, configured to, after inputting the target image sequence into a video quality evaluation model and obtaining the score output by the video quality evaluation model, if the scores corresponding to a consecutive preset number of frames are all lower than a score threshold, traverse a parameter tuning strategy library to obtain multiple historical adjustment strategies, historical network state data before the video network parameter adjustment, and the confidence level of each historical adjustment strategy, wherein the historical adjustment strategy characterizes the correspondence between any parameter adjustment amount of the video network and the score increase after adjustment; determine the network state similarity based on the historical network state data before parameter adjustment and the current network state data; determine the quality correlation corresponding to any historical adjustment strategy based on the network state similarity, the confidence level of the historical adjustment strategy, and the parameter adjustment amount and score increase in any historical adjustment strategy; select the historical adjustment strategy corresponding to the maximum quality correlation as the current adjustment strategy; and adjust the video network with the current adjustment strategy.

[0111] In some embodiments, the video quality evaluation device further includes: an adjustment strategy storage unit, configured to, after adjusting the video network with the current adjustment strategy, write the parameter adjustment amount of the current adjustment strategy, the increase in the real-time score after adjustment, the network status data before the current adjustment, and the confidence level of the current adjustment strategy into the parameter tuning strategy library when the real-time score after adjustment increases.

[0112] In some embodiments, the video quality evaluation apparatus further includes a library update unit, configured to, after adjusting the video network with the current adjustment strategy, update the confidence level of the historical adjustment strategy corresponding to the maximum quality correlation value if the adjusted real-time score does not increase, and re-execute steps S1 to S5 until the adjusted real-time score increases.

[0113] In some embodiments, the video quality evaluation model includes: The feature extraction module is used to extract the feature sequence of the target image sequence.

[0114] The quality regression module is used to map the feature sequence to the score and output it.

[0115] In some embodiments, the video quality evaluation apparatus further includes: a preprocessing unit, configured to receive an original video stream before performing pyramid scaling on each video frame in the video frame sequence to be evaluated, convert the original video stream into a video stream of a predetermined protocol format; decode the converted video stream and extract multiple key frames; and combine the multiple key frames in chronological order to form the video frame sequence to be evaluated.

[0116] In some embodiments, the video quality evaluation apparatus further includes a filtering unit, configured to input a Laplacian sharpening filter into each of the key frames before combining multiple key frames into the video frame sequence to be evaluated, to obtain filtered key frames.

[0117] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a video quality evaluation method, which includes: Pyramid scaling is performed on each video frame in the video frame sequence to be evaluated, resulting in multiple pyramid images of different resolutions for each video frame.

[0118] For each layer of the pyramid image corresponding to each video frame, uniform sampling is performed. G h × G w Each resolution is f h × f w The image blocks are merged to obtain a resolution of ( ). G h · f h )×( G w · f w () fragment images.

[0119] Based on spatial and temporal masking, multiple segment images corresponding to each video frame in the video frame sequence to be evaluated are gradually masked from the highest resolution to the lowest resolution to obtain the target image sequence.

[0120] The target image sequence is input into the video quality evaluation model to obtain the score output by the video quality evaluation model. The video quality evaluation model is trained based on the target image sequence samples and the score labels corresponding to the target image sequence samples.

[0121] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the video quality evaluation method provided by the above methods, the method including: Pyramid scaling is performed on each video frame in the video frame sequence to be evaluated, resulting in multiple pyramid images of different resolutions for each video frame.

[0123] For each layer of the pyramid image corresponding to each video frame, uniform sampling is performed. G h × G w Each resolution is f h × f w The image blocks are merged to obtain a resolution of ( ). G h · f h )×( G w · f w () fragment images.

[0124] Based on spatial and temporal masking, multiple segment images corresponding to each video frame in the video frame sequence to be evaluated are gradually masked from the highest resolution to the lowest resolution to obtain the target image sequence.

[0125] The target image sequence is input into the video quality evaluation model to obtain the score output by the video quality evaluation model. The video quality evaluation model is trained based on the target image sequence samples and the score labels corresponding to the target image sequence samples.

[0126] Furthermore, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the video quality evaluation methods provided by the methods described above, the method comprising: Pyramid scaling is performed on each video frame in the video frame sequence to be evaluated, resulting in multiple pyramid images of different resolutions for each video frame.

[0127] For each layer of the pyramid image corresponding to each video frame, uniform sampling is performed. G h × G w Each resolution is f h × f w The image blocks are merged to obtain a resolution of ( ). G h · f h )×( G w · f w () fragment images.

[0128] Based on spatial and temporal masking, multiple segment images corresponding to each video frame in the video frame sequence to be evaluated are gradually masked from the highest resolution to the lowest resolution to obtain the target image sequence.

[0129] The target image sequence is input into the video quality evaluation model to obtain the score output by the video quality evaluation model. The video quality evaluation model is trained based on the target image sequence samples and the score labels corresponding to the target image sequence samples.

[0130] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A video quality evaluation method characterized by comprising: The method comprises the following steps: pyramid scaling is performed on each video frame in the video frame sequence to be evaluated to obtain a plurality of pyramid images of different resolutions corresponding to each video frame; For each layer image in the pyramid image corresponding to each of the video frames, uniformly sampling G h × G w image blocks with a resolution of f h × f w , and merging the image blocks to obtain a segment image with a resolution of G h · f h × G w · f w ​ mask processing is performed on the plurality of segment images corresponding to each video frame in the video frame sequence to be evaluated from the highest resolution to the lowest resolution based on the spatial mask and the temporal mask to obtain a target image sequence; the target image sequence is input into a video quality evaluation model to obtain a score output by the video quality evaluation model, and the video quality evaluation model is trained based on target image sequence samples and score labels corresponding to the target image sequence samples.

2. The video quality evaluation method according to claim 1, characterized by, mask processing is performed on the plurality of segment images corresponding to each video frame in the video frame sequence to be evaluated from the highest resolution to the lowest resolution based on the spatial mask and the temporal mask to obtain a target image sequence, comprising: masking operations are performed on a first segment image corresponding to an original resolution and at least one second segment image corresponding to a down-sampling resolution in the plurality of segment images corresponding to each video frame based on a binary mask to obtain at least one target image to form a target image sequence, and the down-sampling resolution corresponding to the second segment image of a video frame earlier in time is not less than the down-sampling resolution corresponding to the second segment image of a video frame later in time, the down-sampling resolution corresponding to the second segment image of the earliest video frame in time is the largest, and the down-sampling resolution corresponding to the second segment image of the latest video frame in time is the smallest.

3. The video quality evaluation method according to claim 2, characterized by, masking operations are performed on a first segment image corresponding to an original resolution and at least one second segment image corresponding to a down-sampling resolution in the plurality of segment images corresponding to each video frame based on a binary mask to obtain at least one target image to form a target image sequence, comprising: masking operations are performed on a first segment image corresponding to an original resolution and at least one second segment image corresponding to a down-sampling resolution in the plurality of segment images corresponding to each video frame based on a binary mask to obtain at least one target image to form a target image sequence, comprising:

4. The video quality evaluation method according to claim 1, characterized by, after the target image sequence is input into a video quality evaluation model to obtain a score output by the video quality evaluation model, the method further comprises: in a case where the scores corresponding to a continuous preset number of frames are all lower than a score threshold, an alarm is triggered.

5. The video quality evaluation method according to claim 1, characterized by, after the target image sequence is input into a video quality evaluation model to obtain a score output by the video quality evaluation model, the method further comprises: Step S1: in a case where the scores corresponding to a continuous preset number of frames are all lower than a score threshold, a parameter tuning strategy library is traversed to obtain a plurality of historical adjustment strategies, historical network state data before parameter adjustment of a video live streaming network, and a confidence degree of each historical adjustment strategy, the historical adjustment strategy representing a corresponding relationship between a parameter adjustment amount of the video live streaming network and a score increase amount after adjustment; Step S2: based on the historical network state data before parameter adjustment and current network state data, a network state similarity is determined; Step S3: based on the network state similarity, the confidence degree of the historical adjustment strategy, and the parameter adjustment amount and the score increase amount in any historical adjustment strategy, a quality correlation corresponding to any historical adjustment strategy is determined; Step S4: selecting a historical adjustment strategy corresponding to the maximum quality correlation as a current adjustment strategy; Step S5: adjusting the video network in the current adjustment strategy.

6. The video quality evaluation method according to claim 5, characterized by, After adjusting the video network in the current adjustment strategy, further comprising: In the case of an increase in the adjusted real-time score, the parameter adjustment amount of the current adjustment strategy, the increase in the adjusted real-time score, the network state data before the current adjustment, and the confidence of the current adjustment strategy are written into the parameter optimization strategy library.

7. The video quality evaluation method according to claim 5, characterized by, After adjusting the video network in the current adjustment strategy, further comprising: In the case of no increase in the adjusted real-time score, updating the confidence of the historical adjustment strategy corresponding to the maximum quality correlation, and re-executing steps S1 to S5 until the adjusted real-time score increases.

8. The video quality evaluation method according to claim 1, characterized by, The video quality evaluation model comprises: a feature extraction module for extracting a feature sequence of the target image sequence; a quality regression module for mapping the feature sequence to the score and outputting.

9. The video quality assessment method according to any one of claims 1 to 8, characterized by, Before pyramid scaling each video frame in the video frame sequence to be evaluated, further comprising: receiving an original video stream, converting the original video stream into a video stream in a predetermined protocol format; decoding and extracting multiple key frames from the converted video stream; combining the multiple key frames into the video frame sequence to be evaluated in chronological order.

10. The video quality evaluation method according to claim 9, characterized by, Before combining the multiple key frames into the video frame sequence to be evaluated, further comprising: inputting each key frame into a Laplacian sharpening filter to obtain a filtered key frame.

11. A video quality evaluation apparatus characterized by comprising: comprising: a pyramid scaling unit for pyramid scaling each video frame in the video frame sequence to be evaluated to obtain multiple pyramid images of different resolutions corresponding to each video frame; a segment image sampling unit, for uniformly sampling each image in the pyramid image corresponding to each of the video frames G h × G w image blocks with a resolution of f h × f w , and merging the image blocks to obtain a segment image with a resolution of G h · f h ) × ( G w · f w ) a space-time mask unit for gradually performing mask processing on multiple segment images corresponding to each video frame in the video frame sequence to be evaluated from the highest resolution to the lowest resolution based on a spatial mask and a temporal mask to obtain a target image sequence; a model execution unit for inputting the target image sequence into a video quality evaluation model to obtain a score output by the video quality evaluation model, the video quality evaluation model being trained based on a target image sequence sample and a score label corresponding to the target image sequence sample.

12. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to realize the video quality evaluation method of any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the video quality evaluation method of any one of claims 1 to 10.

14. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the video quality evaluation method of any one of claims 1 to 10.

Images