Image quality evaluation method for video enhancement
By flexibly setting preprocessing parameters for video enhancement and providing real-time preview comparison, the system solves the problems of limited parameter combinations and insufficient visualization in traditional video enhancement and transcoding systems, achieving efficient video quality improvement.
Patent Information
- Application Number
- CN202411137464.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-19
AI Technical Summary
Traditional video enhancement and transcoding systems lack flexible parameter combinations and visual adjustments, resulting in low efficiency in video enhancement processing and making it difficult for users to quickly find the best solution.
This paper presents a method for evaluating the image quality of video enhancement, which allows each preprocessing parameter to be set independently, supports predefined and custom processes, and combines objective and subjective evaluation indicators to achieve real-time preview and comparison of video effects.
It improves the flexibility and efficiency of video enhancement, allowing users to quickly find the best processing solutions, thereby improving video quality and work efficiency.
Smart Images

Figure CN119011906A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of video processing, and in particular relates to a method for evaluating the image quality of video enhancement. Background Art
[0002] With the widespread application of video content in various industries, the requirements for video quality are getting higher and higher. However, the traditional enhanced transcoding system lacks video enhancement processing logic and visual presentation of the effects before and after enhancement.
[0003] First, traditional video enhancement transcoding systems usually include a variety of optimization algorithms, such as super-resolution, denoising, deblocking, color enhancement, etc., but the parameter processing logic of these algorithms is relatively simple, and a pre-processing parameter can only be set to one value in each processing step, and cannot be flexibly combined, which will result in some videos not being very effective using the default processing logic, or some complex videos requiring multi-step image quality enhancement processing. Users need to try and compare the effects of different optimization solutions according to the needs of the actual scene, which often requires a lot of time and effort.
[0004] Secondly, traditional video enhancement transcoding systems usually lack a visual parameter adjustment interface and real-time preview function. Users can only adjust parameters based on experience. If they encounter complex optimization scenarios, they need to try different optimization solutions and debug repeatedly to achieve satisfactory results. However, during the entire optimization solution adjustment process, they cannot visually view the effects before and after the enhancement. They need to wait until the entire file is converted before they can view the final effect, and cannot quickly and intuitively understand the effect after the enhancement.
[0005] The entire video enhancement process often requires a lot of time and energy investment, is inefficient, and it is difficult for users to quickly find the best enhancement solution. Summary of the invention
[0006] In view of the above problems, the present invention provides a video enhanced picture quality evaluation method with the characteristics of flexible parameter setting, customizable preprocessing process, rich quality assessment indicators, optimized user interaction experience, etc., which greatly improves work efficiency and final video quality.
[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0008] A method for evaluating the image quality of video enhancement comprises the following steps:
[0009] Set preprocessing parameters. Different values can be set for preprocessing parameters at each stage. Preprocessing parameters include resolution, bit rate, bit rate control mode, AI super-resolution parameters, scratch removal, deep learning deblocking, intelligent interpolation, denoising, and original picture enhancement.
[0010] Set up the video preprocessing process engine, including predefined processes and custom processes. In the predefined process, process templates for three scenarios are provided: super-resolution, old film restoration, and low-code high-definition. When the effect of custom optimization does not meet expectations, enter the custom preprocessing process sequence by dragging and dropping. When customizing the process, the preprocessing sequence of the current function default logic is used, and the preprocessing parameters that can be adjusted are displayed for users to adjust the parameters.
[0011] After the pre-processing flow engine module is configured, the effect comparison before and after optimization is performed on the same page to view the preview; after clicking Preview, set the custom preview start time, and the previewed frames are the frames adjacent to the first frame at the start time point. Slide the comparison line in the middle of the picture left and right to compare the pictures before and after optimization. If the previewed picture reaches the expected effect, the whole file is enhanced and output; otherwise, continue to adjust the pre-processing parameters until the expected effect is achieved;
[0012] The video preprocessing process engine module is set to perform index evaluation on the same page, and the index evaluation includes objective index, non-parameter index and color deviation correlation.
[0013] In a possible implementation, the resolution can be set to include four resolution parameters: resolution 1, resolution 2, resolution 3, and resolution 4, where each resolution parameter can be set to values including 720*576, 1280*720, 1920*1080, 3840*2160, and follow source, etc., and the default value is follow source.
[0014] In a possible implementation manner, the bit rate can be set to include up to four bit rate parameters including bit rate 1, bit rate 2, bit rate 3 and bit rate 4, wherein the settable value range of each bit rate parameter is 1 to 10000000, and the default value is 1000.
[0015] In a possible implementation manner, the rate control mode can be set to include up to four rate control parameters, including rate control 1, rate control 2, rate control 3 and rate control 4, where each rate control parameter can be set to a value of VBR, CBR and VBR+, and the default value is VBR.
[0016] In one possible implementation, the AI super-resolution can be set to two super-resolution parameters including super-resolution 1 and super-resolution 2, wherein the settable value of each super-resolution parameter is speed mode, quality mode, CG mode and off, and the default value is off. The AI super-resolution parameters require that the width and height of the output resolution be greater than the width and height of the input resolution, otherwise transcoding will fail.
[0017] In a possible implementation manner, the scratch removal may be set to include two scratch removal parameters, scratch removal 1 and scratch removal 2, wherein each scratch removal parameter may have a settable value of speed mode, quality mode V1, quality mode V2 and off, and the default value is off.
[0018] In a possible implementation, the original picture enhancement can set two original picture enhancement parameters, original picture enhancement 1 and original picture enhancement 2, wherein the settable value of each setting parameter is on or off, and the default value is off. The picture quality enhancement parameter requires that the width and height of the output resolution be equal to the width and height of the input resolution, otherwise transcoding will fail.
[0019] In a possible implementation manner, the super-resolution scene mode setting in the predefined process includes: setting the resolution and setting the super-resolution mode selection.
[0020] In a possible implementation manner, the old film restoration scene mode setting in the predefined process includes: setting resolution, AI deblocking and denoising.
[0021] In a possible implementation manner, the low-code high-definition scene setting in the predefined process includes: setting resolution+bit rate control mode+bit rate size.
[0022] The present invention has the following beneficial effects:
[0023] (1) Preprocessing parameters can be set individually. Each algorithm / function is independent and algorithms can be added or deleted at will, making modification flexible.
[0024] (2) The preprocessing order can be flexibly adjusted, and the same parameter can be set multiple times as needed.
[0025] (3) It can support seek preview / specify position and compare the video effects before and after enhancement. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 The present invention is a flowchart of the steps of a method for evaluating the image quality of video enhancement according to an embodiment of the present invention. DETAILED DESCRIPTION
[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] See also Figure 1, which is a flowchart of a method for evaluating the quality of video enhancement according to an embodiment of the present invention, comprising the following steps:
[0029] S10, set the preprocessing parameters. Different values can be set for the preprocessing parameters at each stage. The preprocessing parameters include resolution, bit rate, bit rate control mode, AI super-resolution parameters, scratch removal, deep learning deblocking, intelligent interpolation, denoising and original picture enhancement. This allows flexible selection of parameter values for each step, greatly improving the efficiency and flexibility of video enhancement. Moreover, each preprocessing parameter is modular, so it is more convenient to add or delete preprocessing parameter modules in the later stage.
[0030] S20, set the video preprocessing flow engine, including predefined flows and custom flows. In the predefined flow, flow templates for three scenarios are provided: super-resolution, old film restoration, and low-code high-definition. When the effect of custom optimization does not meet expectations, enter the custom preprocessing flow sequence by dragging and dropping. When customizing the processing flow, the preprocessing sequence of the current function default logic is used, and the preprocessing parameters that can be adjusted are displayed for users to adjust the parameters. When the effect of custom optimization does not meet expectations, the preprocessing flow sequence can be customized by dragging and dropping.
[0031] The premise of displaying which algorithms can be adjusted in the custom module is that which parameters are set and enabled in the video preprocessing parameter module. At the same time, the custom preprocessing process created by the user can be saved as a template and reused in future video processing. This can improve work efficiency and reduce duplication of work. At the same time, there are corresponding interface guidelines between each preprocessing parameter, and the parameter module itself has its own limitations. If the relevant conditions and restrictions are not met, subsequent preview or transcoding operations cannot be performed.
[0032] S30, when the pre-processing flow engine module is configured, the effect comparison before and after optimization is performed on the same page for preview; after clicking preview, a custom preview start time is set, and the previewed frames are the frames adjacent to the first frame at the start time point, and the comparison line in the middle of the picture is slid left and right to compare the pictures before and after optimization. If the previewed picture reaches the expected effect, the whole file is enhanced and output; otherwise, the pre-processing parameters are continuously adjusted until the expected effect is achieved;
[0033] S40, setting the video preprocessing flow engine module to perform index evaluation on the same page, the index evaluation includes objective index, non-parameter index and color deviation correlation.
[0034] Specifically, in S10, the resolution can be set to include 4 resolution parameters including resolution 1, resolution 2, resolution 3 and resolution 4, where the values that can be set for each resolution parameter include but are not limited to 720*576, 1280*720, 1920*1080, 3840*2160, follow source, etc., and the default value is follow source. The bit rate can be set to include 4 resolution parameters including bit rate 1, bit rate 2, bit rate 3 and bit rate 4, where the value range that can be set for each resolution parameter is 1 to 10000000, and the default value is 1000. The bit rate control mode can be set to include bit rate control 1, bit rate control 2, bit rate control 3 and 4 bit rate control parameters, where the value that can be set for each bit control parameter is VBR, CBR, VBR+, and the default value is VBR. AI super-resolution can be set to include 2 super-resolution parameters including super-resolution 1 and super-resolution 2, where the value that can be set for each super-resolution parameter is speed mode, quality mode, CG mode, and off, and the default value is off. AI super-resolution parameters require that the width and height of the output resolution be greater than the width and height of the input resolution, otherwise transcoding will fail. Scratch removal can set two scratch removal parameters, including scratch removal 1 and scratch removal 2. The values that can be set for each scratch removal parameter are speed mode, quality mode V1, quality mode V2, and off, and the default value is off. Deep learning deblocking can set two deblocking parameters, including deblocking 1 and deblocking 2. The values that can be set for each setting parameter are on and off, and the default value is off. Smart interpolation can set two interpolation parameters, including interpolation 1 and interpolation 2. The values that can be set for each smart interpolation parameter are smart mode, stable mode, and off, and the default value is off. Denoising can set two denoising parameters, including denoising 1 and denoising 2. The values that can be set for each denoising parameter are on and off, and the default value is off. Original picture enhancement can set two original picture enhancement parameters, including original picture enhancement 1 and original picture enhancement 2. The values that can be set for each original picture enhancement setting parameter are on and off, and the default value is off. The original image enhancement parameters require that the width and height of the output resolution be equal to the width and height of the input resolution, otherwise the transcoding will fail.
[0035] Specifically, in S20, the super-resolution scene mode setting in the predefined process includes: setting the resolution and the super-resolution mode selection. If set to super-resolution: resolution 1 (3840*2160) + super-resolution 1 (quality mode) + bit rate 1 (20000). The old film restoration scene mode setting in the predefined process includes: setting the resolution, AI deblocking and denoising. If set to old film restoration: resolution 1 (1920*1080) + deblocking 1 (on) + denoising (level 3) + bit rate 1 (10000). The low-code high-definition scene setting in the predefined process includes: setting the resolution + bit rate control mode + bit rate size. If set to low-code high-definition: resolution 1 (1920*1080) + code control 1 (VBR+) + bit rate 1 (3000).
[0036] Specifically, in S40, the objective indicators of the index evaluation are usually used to compare the quality of the original video image with the quality of the compressed or transmitted video image, and calculate the difference between them to measure the quality of the video. Objective indicators may include Peak Signal to Noise Ratio (PSNR) and Visual Multimethod Assessment Fusion (VMAF). The higher the PSNR value, the smaller the image distortion and the better the quality. In video compression, the typical value range of PSNR is 30dB to 50dB. When it is higher than 30dB, it is difficult for the human eye to perceive the difference between the compressed and original images. In particular, when the PSNR is close to 50dB, it means that the compressed image has only a very small error, and the image quality can be considered to be very good. VMAF combines the results of objective evaluation and subjective evaluation of the human eye to provide a more accurate and comprehensive video quality evaluation. The higher the VMAF value, the better the quality. Generally speaking, a score above 95 means that it is extremely difficult to tell the difference with the naked eye, a score of 93-95 means that slight differences can be perceived but are completely acceptable, and a score below 91 usually means that the differences are more obvious.
[0037] No-reference metrics are a method that does not require the original reference video for quality assessment. This type of metric relies mainly on the content and characteristics of the video itself to assess quality, rather than the original high-quality version. They assess video quality by analyzing the content and characteristics of the video, such as blurring, blocking, naturalness, etc. Blurring is mainly used to assess the blurriness of an image, especially in denoising and image enhancement scenarios, directly measuring the clarity of the image. A lower score generally means better image quality, because this indicates that the image is clearer. Blocking is mainly used to assess the blocking effect of an image, especially in deblocking and video compression scenarios, directly measuring the smoothness of the image. A lower score generally means better video quality and smoother visual effects. NIQE is especially used in AI super-resolution scenarios. NIQE evaluates image quality by calculating the difference in statistical characteristics between an image and a natural image. Therefore, a lower score means that the image is closer to the statistical characteristics of a natural scene and is therefore considered to be of higher quality.
[0038] Color deviation can be represented by a vectorscope, which is used to display color information in color video signals. It helps us understand the color saturation and hue in the video, but does not include brightness information. In a vectorscope, color information is converted into a vector whose direction and length represent color information. The direction of the vector represents the hue; while the length of the vector represents the saturation of the color. The longer the vector extends from the center, the more saturated the color. If the color information of the two images in the same frame of the original video and the transcoded video is consistent, the presented vector graphics will also be consistent.
[0039] Furthermore, in S40, a visual statistical chart can be provided, and each indicator can give a corresponding reference value range. These evaluation indicator results can also be exported to help users quickly evaluate whether the current solution has achieved the expected results. Users can combine subjective previews with these objective indicators to flexibly adjust enhancement parameters and optimize the final video quality.
[0040] The above-mentioned video enhancement image quality evaluation method has the characteristics of flexible parameter settings, customizable preprocessing processes, rich quality evaluation indicators, and optimized user interaction experience, which can greatly improve work efficiency and final video quality. Preprocessing templates for common scenarios are predefined, and users are supported to customize processes and parameters to meet personalized needs. At the same time, the quality evaluation indicators are combined with subjective previews to comprehensively analyze the enhancement effect of the video. In addition, preview comparisons and intuitive data statistical analysis are set to further optimize the user experience.
[0041] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.
Claims
1. A video enhancement quality evaluation method, characterized in that: The following steps are involved: Set preprocessing parameters. Different values can be set for preprocessing parameters at each stage. Preprocessing parameters include resolution, bit rate, bit rate control mode, AI super-resolution parameters, scratch removal, deep learning deblocking, intelligent interpolation, denoising, and original picture enhancement. Set up the video preprocessing process engine, including predefined processes and custom processes. In the predefined processes, process templates for three scenarios are provided: super-resolution, old film restoration, and low-code high-definition; When the effect of customized optimization does not meet expectations, you can enter the customized preprocessing process sequence by dragging and dropping; when customizing the process, the preprocessing sequence of the current function default logic is set, and the adjustable preprocessing parameters are displayed for users to adjust the parameters; After the pre-processing flow engine module is configured, the effect comparison before and after optimization is performed on the same page to view the preview; after clicking Preview, set the custom preview start time, and the previewed frames are the frames adjacent to the first frame at the start time point. Slide the comparison line in the middle of the picture left and right to compare the pictures before and after optimization. If the previewed picture reaches the expected effect, the whole file is enhanced and output; otherwise, continue to adjust the pre-processing parameters until the expected effect is achieved; The video preprocessing process engine module is set to perform index evaluation on the same page, and the index evaluation includes objective index, non-parameter index and color deviation correlation.
2. The video enhancement quality evaluation method according to claim 1, characterized in that: The resolution can be set to include four resolution parameters: resolution 1, resolution 2, resolution 3 and resolution 4, where each resolution parameter can be set to values including 720*576, 1280*720, 1920*1080, 3840*2160 and follow source, etc. The default value is follow source.
3. The video enhancement quality evaluation method according to claim 1, characterized in that: The bit rate can be set to include up to 4 bit rate parameters including bit rate 1, bit rate 2, bit rate 3 and bit rate 4, wherein the value range of each bit rate parameter that can be set is 1 to 10000000, and the default value is 1000.
4. The video enhancement quality evaluation method according to claim 1, characterized in that: The rate control mode can be set to a maximum of four rate control parameters including rate control 1, rate control 2, rate control 3 and rate control 4, wherein each rate control parameter can be set to a value of VBR, CBR and VBR+, and the default value is VBR.
5. The video enhancement quality evaluation method according to claim 1, characterized in that: The AI super-resolution can set two super-resolution parameters including super-resolution 1 and super-resolution 2, wherein the settable value of each super-resolution parameter is speed mode, quality mode, CG mode and off, and the default value is off. The AI super-resolution parameters require that the width and height of the output resolution must be larger than the width and height of the input resolution, otherwise transcoding will fail.
6. The video enhancement quality evaluation method according to claim 1, characterized in that: The scratch removal can be set to include two scratch removal parameters, scratch removal 1 and scratch removal 2, wherein each scratch removal parameter can be set to a speed mode, a quality mode V1, a quality mode V2 and off, and the default value is off.
7. The video enhancement quality evaluation method according to claim 1, characterized in that: The original picture enhancement can set two original picture enhancement parameters, original picture enhancement 1 and original picture enhancement 2, where the settable value of each setting parameter is on or off, and the default value is off. The picture quality enhancement parameter requires that the width and height of the output resolution must be equal to the width and height of the input resolution, otherwise the transcoding will fail.
8. The video enhancement quality evaluation method according to claim 1, characterized in that: The super-resolution scene mode setting in the predefined process includes: setting the resolution and setting the super-resolution mode selection.
9. The video enhancement quality evaluation method according to claim 1, characterized in that: The old film restoration scene mode settings in the predefined process include: resolution, AI deblocking and denoising settings.
10. The video enhancement quality evaluation method according to claim 1, characterized in that: The low-code high-definition scene setting in the predefined process includes: setting the resolution + bit rate control mode + bit rate size.
Citation Information
Patent Citations
Video shooting method and electronic equipment
CN115191110A
Intelligent program directing method and system for panoramic video
CN117499696A
Video image quality prediction method and device, equipment, storage medium and program product
CN118354119A
Image Content Removal Method and Related Apparatus
US20230217097A1
