Video quality evaluation method for video enhancement

Abstract

The application discloses a video enhancement quality evaluation method, comprising the following steps: carrying out pre-processing parameter setting, different values can be set for the pre-processing parameters of each stage; setting a video pre-processing flow engine, including a pre-defined flow and a self-defined flow, in the pre-defined flow, three flow templates of super-resolution, old film restoration and low-code high-definition are provided; when the effect of self-defined optimization does not reach the expectation, the self-defined pre-processing flow sequence is entered in a dragging mode; after the pre-processing flow engine module is configured, the effect comparison and preview before and after optimization are carried out under the same page; after previewing, the self-defined preview start time is set, the frame picture of preview is a plurality of frames adjacent to the first frame from the start time point, the picture comparison before and after optimization is carried out by sliding the comparison line in the middle of the left and right pictures, if the picture of preview reaches the expected effect, the whole file enhancement output is carried out; the index evaluation is carried out in the same page of the video pre-processing flow engine module.

Description

Technical Field

[0001] This invention belongs to the field of video processing technology, and specifically relates to a method for evaluating the image quality of video enhancement. Background Technology

[0002] With the widespread use of video content across various industries, the requirements for video quality are becoming increasingly stringent. However, traditional enhancement transcoding systems are lacking in video enhancement processing logic and the visualization of the effects before and after enhancement.

[0003] First, traditional video enhancement and transcoding systems typically include multiple optimization algorithms, such as super-resolution, noise reduction, deblocking, and color enhancement. However, the parameter processing logic of these algorithms is relatively simple, and each preprocessing parameter can only be set to one value in each processing step, making flexible combinations impossible. This can lead to some videos not performing well with the default processing logic, or some complex videos requiring multi-step image enhancement processing. Users need to try and compare the effects of different optimization schemes according to the needs of their actual scenarios, which often requires a significant amount of time and effort.

[0004] Secondly, traditional video enhancement and transcoding systems typically lack a visual parameter adjustment interface and real-time preview function. Users can only rely on experience to adjust parameters. If they encounter complex optimization scenarios, they need to try different optimization schemes and repeatedly debug to achieve satisfactory results. However, during the entire optimization process, it is not possible to intuitively view the effects before and after enhancement. The final effect can only be viewed after the entire file has been converted, making it impossible to quickly and intuitively understand the enhanced effect.

[0005] The entire video enhancement process often requires a significant investment of time and effort, is inefficient, and makes it difficult for users to quickly find the best enhancement solution. Summary of the Invention

[0006] In view of the above-mentioned problems, the present invention provides a video enhancement quality evaluation method, which features flexible parameter settings, customizable preprocessing workflow, rich quality evaluation indicators, and optimized user interaction experience, thereby greatly improving work efficiency and final video quality.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] A method for evaluating the image quality of video enhancement includes the following steps:

[0009] Preprocessing parameters can be set, and different values ​​can be set for each stage of preprocessing. Preprocessing parameters include resolution, bitrate, bitrate control mode, AI super-resolution parameters, scratch removal, deep learning deblocking, intelligent frame interpolation, noise reduction, and original image enhancement.

[0010] The video preprocessing workflow engine is set up, including predefined workflows and custom workflows. In the predefined workflows, workflow templates for three scenarios are provided: super-resolution, old film restoration, and low-bitrate high-definition. When the effect of custom optimization does not meet expectations, users can enter the custom preprocessing workflow order by dragging and dropping. The custom processing workflow is the preprocessing order of the current function's default logic, and the preprocessing parameters that can be adjusted are displayed for users to adjust.

[0011] Once the preprocessing engine module is configured, preview the before and after optimization effects on the same page. After clicking preview, set a custom preview start time. The preview frames are the first frame and several adjacent frames after it. Slide the comparison line in the middle of the screen left and right to compare the before and after optimization. If the preview achieves the expected effect, output the entire file with enhancement. Otherwise, continue adjusting the preprocessing parameters until the expected effect is achieved.

[0012] On the same page as the video preprocessing workflow engine module, metrics evaluation is performed, including objective metrics, parameter-free metrics, and color bias correlation.

[0013] In one possible implementation, the resolution can be set to four resolution parameters: resolution 1, resolution 2, resolution 3, and resolution 4. The settable values ​​for each resolution parameter include 720*576, 1280*720, 1920*1080, 3840*2160, and follow the source, with the default value being follow the source.

[0014] In one possible implementation, the bitrate can be set to a maximum of four bitrate parameters, including bitrate 1, bitrate 2, bitrate 3 and bitrate 4, wherein the value range of each bitrate parameter is 1 to 10,000,000, and the default value is 1,000.

[0015] In one possible implementation, the rate control mode can be set with up to four rate control parameters, including rate control 1, rate control 2, rate control 3 and rate control 4, wherein each rate control parameter can be set to VBR, CBR and VBR+, with the default value being VBR.

[0016] In one possible implementation, the AI ​​super-resolution can be set with two super-resolution parameters, including super-resolution 1 and super-resolution 2. Each super-resolution parameter can be set to a speed mode, quality mode, CG mode, or off, with the default value being off. The AI ​​super-resolution parameters require that the width and height of the output resolution be greater than the width and height of the input resolution; otherwise, transcoding will fail.

[0017] In one possible implementation, the scratch removal can be set with two scratch removal parameters, including scratch removal 1 and scratch removal 2, wherein each scratch removal parameter can be set to a speed mode, quality mode V1, quality mode V2, and off, with the default value being off.

[0018] In one possible implementation, the original image enhancement can set two original image enhancement parameters, Original Image Enhancement 1 and Original Image Enhancement 2. Each parameter can be set to either on or off, with the default value being off. The image enhancement parameters require that the width and height of the output resolution be equal to the width and height of the input resolution; otherwise, transcoding will fail.

[0019] In one possible implementation, the super-resolution scene mode settings in the predefined process include: setting the resolution and setting the super-resolution mode selection.

[0020] In one possible implementation, the predefined process for setting the old film restoration scene mode includes: setting the resolution, AI deblocking, and noise reduction.

[0021] In one possible implementation, the low-bitrate high-definition scene settings in the predefined process include: setting the resolution + bitrate control mode + bitrate size.

[0022] The present invention has the following beneficial effects:

[0023] (1) Preprocessing parameters can be set individually. Each algorithm / function is independent and can be added or deleted at will, making it flexible to modify.

[0024] (2) The preprocessing order can be flexibly adjusted, and multiple parameters can be set according to requirements.

[0025] (3) It can support seek preview / specify position and compare the video effects before and after enhancement. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the steps of a video enhancement quality evaluation method according to an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] See Figure 1The above is a flowchart of a video enhancement quality evaluation method according to an embodiment of the present invention, including the following steps:

[0029] S10 involves setting preprocessing parameters. Different values ​​can be set for each stage of the preprocessing parameters, including resolution, bitrate, bitrate control mode, AI super-resolution parameters, scratch removal, deep learning deblocking, intelligent frame interpolation, noise reduction, and original image enhancement. This allows for flexible selection of parameter values ​​for each step, significantly improving the efficiency and flexibility of video enhancement. Furthermore, each preprocessing parameter is modular, allowing for easier addition, deletion, and modification of preprocessing parameter modules later.

[0030] S20 sets up the video preprocessing workflow engine, including predefined workflows and custom workflows. The predefined workflows provide workflow templates for three scenarios: super-resolution, old film restoration, and low-bitrate high-definition. When the custom optimization effect does not meet expectations, users can drag and drop to enter the custom preprocessing workflow order. The custom processing workflow uses the default preprocessing logic for the current function and displays adjustable preprocessing parameters for user adjustment. When the custom optimization effect does not meet expectations, the preprocessing workflow order can be customized by dragging and dropping.

[0031] The ability to display which algorithms can be adjusted in the custom module depends on which parameters are enabled in the video preprocessing parameters module. User-created custom preprocessing workflows can be saved as templates and reused in future video processing. This improves efficiency and reduces repetitive work. Furthermore, there are corresponding interface guidelines between the preprocessing parameters, and the parameter module itself has its own limitations. If these limitations are not met, subsequent previewing or transcoding operations cannot be performed.

[0032] S30: After the preprocessing engine module is configured, preview the before and after optimization effects on the same page. After clicking preview, set a custom preview start time. The preview frames are the first frame and several adjacent frames after it. Slide the comparison line in the middle of the screen left and right to compare the before and after optimization. If the preview achieves the expected effect, the entire file is enhanced and output. Otherwise, continue to adjust the preprocessing parameters until the expected effect is achieved.

[0033] S40, on the same page as the video preprocessing workflow engine module, performs indicator evaluation, which includes objective indicators, parameter-free indicators, and color deviation correlation.

[0034] Specifically, in S10, the resolution can be set with four parameters: Resolution 1, Resolution 2, Resolution 3, and Resolution 4. Each resolution parameter can be set to values ​​including, but not limited to, 720*576, 1280*720, 1920*1080, 3840*2160, and Follow Source, with the default value being Follow Source. The bitrate can also be set with four resolution parameters: Bitrate 1, Bitrate 2, Bitrate 3, and Bitrate 4. Each bitrate parameter can be set to a value ranging from 1 to 10,000,000, with a default value of 1000. The bitrate control mode can be set with four bitrate control parameters: Bitrate Control 1, Bitrate Control 2, Bitrate Control 3, and Bitrate Control, with each bitrate control parameter set to values ​​of VBR, CBR, and VBR+, with the default value being VBR. The AI ​​super-resolution can be set with two super-resolution parameters: Super-resolution 1 and Super-resolution 2. Each super-resolution parameter can be set to values ​​of Speed ​​Mode, Quality Mode, CG Mode, and Off, with the default value being Off. AI super-resolution parameters require that the width and height of the output resolution be greater than the width and height of the input resolution; otherwise, transcoding will fail. Scratch removal can be configured with two parameters, Scratch Removal 1 and Scratch Removal 2. Each parameter can be set to Speed ​​Mode, Quality Mode V1, Quality Mode V2, or Off. The default value is Off. Deep Learning Deblocking can be configured with two parameters, Deblocking 1 and Deblocking 2. Each parameter can be set to On or Off. The default value is Off. Smart Frame Interpolation can be configured with two parameters, Interpolation 1 and Interpolation 2. Each parameter can be set to Smart Mode, Stabilized Mode, or Off. The default value is Off. Denoising can be configured with two parameters, Denoising 1 and Denoising 2. Each parameter can be set to On or Off. The default value is Off. Original Image Enhancement can be configured with two parameters, Original Image Enhancement 1 and Original Image Enhancement 2. Each enhancement parameter can be set to On or Off. The default value is Off. The original image enhancement parameters require that the width and height of the output resolution be equal to the width and height of the input resolution; otherwise, the transcoding will fail.

[0035] Specifically, in S20, the predefined workflow's super-resolution scene mode settings include: setting the resolution and selecting the super-resolution mode. For example, setting it to super-resolution: Resolution 1 (3840*2160) + Super-resolution 1 (Quality Mode) + Bitrate 1 (20000). The predefined workflow's old film restoration scene mode settings include: setting the resolution, AI deblocking, and denoising. For example, setting it to old film restoration: Resolution 1 (1920*1080) + Deblocking 1 (On) + Denoising (Level 3) + Bitrate 1 (10000). The predefined workflow's low-bitrate high-definition scene settings include: setting the resolution, bitrate control mode, and bitrate size. For example, setting it to low-bitrate high-definition: Resolution 1 (1920*1080) + Bitrate Control 1 (VBR+) + Bitrate 1 (3000).

[0036] Specifically, in S40, objective metrics for evaluation are typically used when comparing the quality of the original video image with that of the compressed or transmitted video image. These metrics calculate the difference between the two to measure video quality. Objective metrics can include Peak Signal-to-Noise Ratio (PSNR) and Visual Multimethod Assessment Fusion (VMAF). A higher PSNR value indicates less image distortion and better quality. In video compression, the typical PSNR range is 30dB to 50dB. Above 30dB, the difference between the compressed and original images is difficult for the human eye to perceive. Especially when the PSNR is close to 50dB, it means that the compressed image has only a very small error, and the image quality can be considered very good. VMAF combines the results of objective evaluation and subjective human evaluation, providing a more accurate and comprehensive video quality assessment. A higher VMAF value indicates better quality. Generally speaking, a score of 95 or above means that the difference is extremely difficult to distinguish with the naked eye. A score of 93-95 indicates that a slight difference can be perceived but is completely acceptable. A score below 91 usually indicates a more obvious difference.

[0037] No-reference metrics are methods for quality assessment that do not require an original reference video. These metrics rely primarily on the content and characteristics of the video itself to evaluate quality, rather than on an original high-quality version. They assess video quality by analyzing content and characteristics such as blurring, blocking, and naturalness. Blurring is mainly used to evaluate the degree of blurring in an image, especially in denoising and image enhancement scenarios, directly measuring the sharpness of the image. A lower score generally indicates better image quality because it suggests a sharper image. Blocking is mainly used to evaluate the blocking effect in an image, especially in deblocking and video compression scenarios, directly measuring the smoothness of the image. A lower score generally indicates better video quality and a smoother visual effect. NIQE, particularly used in AI super-resolution scenarios, evaluates image quality by calculating the difference in statistical characteristics between an image and a natural image. Therefore, a lower score means that the image is closer to the statistical characteristics of a natural scene, and is thus considered to be of higher quality.

[0038] Color cast can be represented using vector graphics, and vector oscilloscopes are used to display color information in color video signals. They help us understand the color saturation and hue in a video, but do not include brightness information. In a vector graphic, color information is converted into a vector, whose direction and length represent the color information. The direction of the vector represents the hue; while the length of the vector represents the color saturation. The vector extends outward from the center; the longer the vector, the more saturated the color. If the color information of two images of the same frame in the original video and the transcoded video is consistent, the resulting vector graphics will also be consistent.

[0039] Furthermore, S40 can provide visualized statistical charts, giving corresponding reference value ranges for each indicator, and can also export the results of these evaluation indicators to help users quickly assess whether the current solution has achieved the expected results. Users can combine subjective previews with these objective indicators to flexibly adjust enhancement parameters and optimize the final video quality.

[0040] The video enhancement quality evaluation method described above features flexible parameter settings, customizable preprocessing workflows, rich quality assessment metrics, and an optimized user experience, significantly improving work efficiency and final video quality. It predefines preprocessing templates for common scenarios while supporting user-defined workflows and parameters to meet individual needs. Furthermore, combining quality assessment metrics with subjective preview allows for a comprehensive analysis of the video enhancement effect. In addition, features such as preview comparisons and intuitive data statistical analysis further optimize the user experience.

[0041] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method for evaluating the image quality of video enhancement, characterized in that, Includes the following steps: Preprocessing parameters can be set, and different values ​​can be set for each stage of preprocessing. Preprocessing parameters include resolution, bitrate, bitrate control mode, AI super-resolution parameters, scratch removal, deep learning deblocking, intelligent frame interpolation, noise reduction, and original image enhancement. Set up a video preprocessing workflow engine, including predefined workflows and custom workflows. In the predefined workflows, provide workflow templates for three scenarios: super-resolution, old film restoration, and low-bitrate high-definition. When the effect of custom optimization does not meet expectations, users can drag and drop to enter a custom preprocessing flow order. The custom processing flow is the preprocessing order of the current function's default logic, and displays adjustable preprocessing parameters for users to adjust. Once the preprocessing engine module is configured, preview the before and after optimization effects on the same page. After clicking preview, set a custom preview start time. The preview frames are the first frame and several adjacent frames after it. Slide the comparison line in the middle of the screen left and right to compare the before and after optimization. If the preview achieves the expected effect, output the entire file with enhancement. Otherwise, continue adjusting the preprocessing parameters until the expected effect is achieved. On the same page as the video preprocessing workflow engine module, metrics evaluation is performed, including objective metrics, parameter-free metrics, and color bias correlation.

2. The video enhancement image quality evaluation method as described in claim 1, characterized in that, The resolution can be set to four resolution parameters: resolution 1, resolution 2, resolution 3, and resolution 4. The settable values ​​for each resolution parameter include 720*576, 1280*720, 1920*1080, 3840*2160, and follow the source, with the default value being follow the source.

3. The video enhancement image quality evaluation method as described in claim 1, characterized in that, The bitrate can be set to a maximum of four bitrate parameters, including bitrate 1, bitrate 2, bitrate 3 and bitrate 4. The value range of each bitrate parameter is 1 to 10,000,000, and the default value is 1,000.

4. The video enhancement image quality evaluation method as described in claim 1, characterized in that, The bitrate control mode can set up to four bitrate control parameters, including bitrate control 1, bitrate control 2, bitrate control 3 and bitrate control 4. The values ​​of each bitrate control parameter can be set to VBR, CBR and VBR+, with the default value being VBR.

5. The video enhancement image quality evaluation method as described in claim 1, characterized in that, The AI ​​super-resolution can be set with two super-resolution parameters, including super-resolution 1 and super-resolution 2. Each super-resolution parameter can be set to speed mode, quality mode, CG mode, or off. The default value is off. The AI ​​super-resolution parameters require that the width and height of the output resolution be greater than the width and height of the input resolution; otherwise, the transcoding will fail.

6. The video enhancement image quality evaluation method as described in claim 1, characterized in that, The scratch removal can be set with two scratch removal parameters, scratch removal 1 and scratch removal 2. Each scratch removal parameter can be set to a speed mode, quality mode V1, quality mode V2, or off, with the default value being off.

7. The video enhancement image quality evaluation method as described in claim 1, characterized in that, The original image enhancement can set two original image enhancement parameters, Original Image Enhancement 1 and Original Image Enhancement 2. Each parameter can be set to on or off, with the default value being off. The image enhancement parameters require that the width and height of the output resolution be equal to the width and height of the input resolution; otherwise, the transcoding will fail.

8. The video enhancement image quality evaluation method as described in claim 1, characterized in that, The super-resolution scene mode settings in the predefined process include: setting the resolution and selecting the super-resolution mode.

9. The video enhancement image quality evaluation method as described in claim 1, characterized in that, The predefined workflow for old film restoration scene mode settings includes: setting resolution, AI deblocking, and noise reduction.

10. The video enhancement image quality evaluation method as described in claim 1, characterized in that, The low-bitrate high-definition scene settings in the predefined process include: setting the resolution, bitrate control mode, and bitrate size.

Images