Region-of-interest-based VVC adaptive video coding method and system

Abstract

The invention discloses a VVC adaptive video coding method and system based on a region of interest, and belongs to the technical field of video coding. According to the technical scheme, the method comprises the following steps: S1, carrying out mixed region-of-interest detection on an input video frame; s2, generating a quantization parameter mapping table according to the ROI mask, and executing three-level QP distribution; s3, the QP mapping table is input into an encoder for block-level quantization control, meanwhile, a visual interface is generated to display an ROI detection result and QP space distribution in real time, and the interface supports adjustment of a significance threshold value and motion detection sensitivity through a sliding block; the system comprises a video input processing module, an ROI detection module, a QP mapping generation module, a coding control module and a visual interface module. The method has the beneficial effects that the dynamic QP mapping table is generated through motion-saliency mixed detection, and the coding efficiency is improved while the subjective quality is ensured in combination with a visual debugging interface.

Description

Technical Field

[0001] The present invention relates to the technical field of video coding, and particularly to a VVC adaptive video coding method and system based on regions of interest. Background Art

[0002] With the rapid development of video technology, the applications of high-definition and ultra-high-definition videos are becoming increasingly widespread, and video coding technology faces higher requirements for compression efficiency and better subjective quality. Traditional video coding standards (such as H.264 / AVC, H.265 / HEVC) encode by evenly distributing quantization parameters (QP). Although certain compression effects can be achieved, in complex scenes or high-dynamic motion scenes, this uniform distribution method often fails to balance coding efficiency and visual quality. Specifically, it wastes bitrate in the background area, while in the core areas sensitive to the human eye (such as moving objects or significant regions), details are lost due to insufficient bitrate allocation, affecting the subjective experience.

[0003] In recent years, adaptive video coding methods based on regions of interest (ROI) have gradually become a research hotspot. These methods detect the ROI regions in video frames and assign lower QP values to them to retain more details, while assigning higher QP values to non-ROI regions to save bitrate. However, existing ROI detection methods are mostly based on a single feature (such as motion information or visual saliency), and it is difficult to fully cover the requirements of different scenes. For example, methods relying solely on motion detection may miss significant targets in static complex scenes, while methods relying solely on saliency detection may not accurately capture moving targets in high-dynamic scenes. In addition, existing methods lack a dynamic smooth transition mechanism during QP allocation, resulting in obvious quality jumps at the ROI boundaries and affecting the visual experience.

[0004] In terms of coding control, although the existing VVC (Versatile Video Coding) standard supports block-level QP adjustment, it lacks an efficient ROI detection and QP mapping generation mechanism, making it difficult to achieve adaptive coding in dynamic scenes. At the same time, existing technologies usually lack visualization debugging tools, making parameter adjustment and effect verification rely on experience and increasing the difficulty of development and optimization.

[0005] Therefore, there is an urgent need for an adaptive video coding method that can combine motion and saliency hybrid detection, dynamically generate a QP mapping table, and support visualization debugging to significantly improve coding efficiency while ensuring subjective quality. Summary of the Invention

[0006] The object of the present invention is to provide a VVC adaptive video coding method and system based on regions of interest, which generates a dynamic QP mapping table through a hybrid motion and saliency detection, combines a three-level QP allocation and a visualization interface, and significantly improves the coding efficiency and subjective quality.

[0007] The inventive concept of the present invention is as follows: The present invention provides a VVC adaptive video coding method and system based on regions of interest. First, a hybrid region of interest detection is performed on the input video frame, and an ROI mask is generated based on the weighted fusion of a motion energy map and a visual saliency map. Then, a quantization parameter mapping table is generated according to the ROI mask, and a three-level QP allocation is performed, including quantization parameters for the core region, the transition region, and the background region, where the QP value of the transition region is calculated according to an exponential decay model. Finally, the QP mapping table is input into the encoder for block-level quantization control, and at the same time, a visualization interface is generated to display the ROI detection result and the QP spatial distribution in real time, and the interface supports adjusting the saliency threshold and the motion detection sensitivity through sliders.

[0008] In order to achieve the above object of the invention, the technical solution adopted by the present invention is specifically as follows: A VVC adaptive video coding method based on regions of interest, comprising the following steps:

[0009] 1.1. Perform a hybrid region of interest (ROI) detection on the input video frame, and generate an ROI mask based on the weighted fusion of a motion energy map and a visual saliency map;

[0010] 1.2. Generate a quantization parameter (QP) mapping table according to the ROI mask, and perform a three-level QP allocation, including quantization parameters for the core region, the transition region, and the background region, where the QP value of the transition region is calculated according to an exponential decay model;

[0011] 1.3. Input the QP mapping table into the encoder for block-level quantization control, and at the same time, generate a visualization interface to display the ROI detection result and the QP spatial distribution in real time, and the interface supports adjusting the saliency threshold and the motion detection sensitivity through sliders.

[0012] Further, the step 1.1 includes the following steps:

[0013] 2.1. Read the input YUV, extract the Y components of the current frame and the previous frame, and perform a two-dimensional convolution on the frame difference result between the current frame and the previous frame using a Gaussian filter with a kernel size of 5×5 to generate a motion energy map. The formula for frame difference is D t (x,y) = |Y t (x,y) - Y t-1 (x,y)|, where Y t represents the luminance component of the t-th frame, D t is the difference result, and the formula for Gaussian filter is where the size of the standard deviation σ is 0.5;

[0014] 2.2. Read the input YUV, extract the Y component of the current frame, perform preprocessing of normalization and Hanning window on the current frame, and perform a fast Fourier transform on the preprocessed result f(x, y). Extract the corresponding phase φ(u, v) = ∠F(u, v), then construct an intermediate frequency band-pass filter M(u, v), retain 30%-70% of the intermediate frequency components in the frequency components, and then perform phase spectrum reconstruction through inverse Fourier transform. Finally, perform post-processing of Gaussian smoothing and normalization to generate a visual saliency map;

[0015] 2.3. Perform weighted fusion of the motion energy map and the visual saliency map according to the weight coefficient α, 0.3 ≤ α ≤ 0.7, where the selection of α depends on the scene type. If the processed video is a static complex scene, α is taken as 0.3. If the processed video is a general scene, α is taken as 0.5. If the processed video is a high-dynamic motion scene, α is taken as 0.7. Then, eliminate detection holes through morphological closing operation based on an elliptical kernel, and finally output the ROI mask.

[0016] Further, the specific steps of step 1.2 include the following steps:

[0017] 3.1. QP of the ROI core area c Allocate the first QP value (22 ± 2), and its boundary is determined by connected component analysis;

[0018] 3.2. QP of the ROI transition area t Allocate the second QP value (28 ± 3), the width of this area is 5-15 pixels and the QP value is smoothed according to the formula where d is the normalized distance of the pixel to the ROI boundary, and β is the smoothing factor (2.0 ≤ β ≤ 3.0). When it is detected that the proportion of the ROI area is less than 10%, β is automatically increased by 20%-50% to enhance the smoothing effect;

[0019] 3.3. QP of the ROI background area b Allocate the third QP value as 38 ± 5. When the target bitrate is lower than the preset threshold, according to the formula QP′ b = min(QP b + 3, 45) to increase the QP value of the background area.

[0020] Further, the step 1.3 includes the following steps:

[0021] 4.1. Input the QP mapping table into the VVC encoder to achieve block-level quantization control;

[0022] 4.2. Generate a visualization interface to display the ROI area and QP distribution in real time, and support parameter interactive adjustment.

[0023] The VVC adaptive video coding method based on the region of interest of the present invention has the following significant advantages compared with the prior art, which can effectively improve the efficiency and subjective quality of video coding:

[0024] (1) The present invention creatively performs weighted fusion of the motion energy map and the visual saliency map. The motion energy map is generated by inter-frame difference and Gaussian filtering, which can effectively capture dynamic targets. The visual saliency map uses Fourier transform and intermediate frequency band-pass filtering technology, which is particularly suitable for detecting salient regions in static complex scenes. This dual-modal detection method overcomes the limitations of single-feature detection and can adapt to diverse video scenes;

[0025] (2) The present invention can automatically adjust the weight coefficient α, where 0.3 ≤ α ≤ 0.7, and adopts optimized weight values of 0.3, 0.5, and 0.7 in static complex scenes, general scenes, and high-dynamic motion scenes respectively to ensure the robustness of ROI detection in different scenes. In addition, the present invention effectively eliminates the possible hole problems in the detection process through morphological closing operation based on elliptical kernels, further improving the integrity and accuracy of the ROI mask;

[0026] (3) The present invention proposes an innovative three-level QP allocation strategy, which divides the video frame into three regions: the core region, the transition region, and the background region, designs different quantization parameters for each region, and realizes the smooth transition of QP values through a unique exponential decay model, effectively avoiding the boundary quality jump problem common in traditional methods;

[0027] (4) The present invention realizes more flexible coding control and more intuitive debugging methods. By directly inputting the generated QP mapping table into the VVC encoder, refined block-level quantization control is achieved, significantly improving the coding rate utilization. In addition, the innovative visual debugging interface can display the original frame, ROI detection results, and QP spatial distribution in real time, and adjust the saliency threshold and motion detection sensitivity through intuitive slider interaction, greatly simplifying the parameter optimization process and shortening the debugging cycle;

[0028] (5) The present invention demonstrates wide applicability and compatibility. Its innovative scene adaptation mechanism enables it to perfectly handle various video types such as static complex scenes, general scenes, and high-dynamic motion scenes, effectively solving the performance limitations of traditional methods in a single scene. At the same time, the system is completely designed based on the VVC coding standard and can be directly integrated into the existing video coding pipeline without additional hardware support, having good industrial implementation feasibility. Description of the Drawings

[0029] The accompanying drawings are used to provide a further understanding of the present invention and form a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention.

[0030] Figure 1 It is a schematic flowchart of the method of the present invention;

[0031] Figure 2 It is an exemplary diagram of the motion energy generated by the present invention;

[0032] Figure 3 It is an exemplary diagram of the visual saliency generated by the present invention;

[0033] Figure 4 It is an exemplary diagram of the weighted fusion result based on the motion energy map and the visual saliency map;

[0034] Figure 5 It is an exemplary diagram of the visualization interface generated by the present invention;

[0035] Figure 6 It is a schematic diagram of the system structure of the present invention. Detailed Embodiments

[0036] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0037] Embodiment 1

[0038] Refer to Figures 1 to 5 , the technical solution provided in this embodiment is a VVC adaptive video coding method based on regions of interest, including the following steps:

[0039] 1. Perform hybrid region of interest (ROI) detection on the input video frame, and generate an ROI mask based on the weighted fusion of the motion energy map and the visual saliency map;

[0040] 2. Generate a quantization parameter (QP) mapping table according to the ROI mask, and perform three-level QP allocation, including the quantization parameters of the core region, the transition region, and the background region, where the QP value of the transition region is calculated according to the exponential decay model;

[0041] 3. Input the QP mapping table into the encoder for block-level quantization control, and at the same time generate a visualization interface to display the ROI detection result and the QP spatial distribution in real time. The interface supports adjusting the saliency threshold and the motion detection sensitivity through sliders.

[0042] Specifically, in step 1, referring to Figures 2 - 4, perform hybrid Region of Interest (ROI) detection on the input video frame, generate an ROI mask based on the weighted fusion of the Motion Energy Map and the Visual Salience Map, including the following steps:

[0043] 1) Read the input YUV, extract the Y components of the current frame and the previous frame, perform two-dimensional convolution on the inter-frame difference result of the current frame and the previous frame using a Gaussian filter with a kernel size of 5×5 to generate a Motion Energy Map. The formula for inter-frame difference is D t (x,y) = |Y t (x,y) - Y t-1 (x,y)|, where Y t represents the luminance component of the t-th frame, and D t is the difference result. The formula for Gaussian filter is where the standard deviation σ is 0.5;

[0044] 2) Read the input YUV, extract the Y component of the current frame, perform preprocessing of normalization and Hanning window on the current frame, perform a Fast Fourier Transform on the preprocessed result f(x,y) extract the corresponding phase φ(u,v) = ∠F(u,v), then construct an intermediate frequency band-pass filter M(u,v), retain 30%-70% of the intermediate frequency components in the frequency components, and then perform phase spectrum reconstruction through inverse Fourier transform finally perform post-processing of Gaussian smoothing and normalization to generate a Visual Salience Map;

[0045] 3) Perform weighted fusion on the Motion Energy Map and the Visual Salience Map with a weight coefficient α, 0.3 ≤ α ≤ 0.7. The selection of α depends on the scene type. If the processed video is a static complex scene, α is taken as 0.3; if the processed video is a general scene, α is taken as 0.5; if the processed video is a high-dynamic motion scene, α is taken as 0.7. Then eliminate detection holes through morphological closing operation based on an elliptical kernel, and finally output the ROI mask.

[0046] Specifically, in step 2, generate a quantization parameter QP mapping table according to the ROI mask, perform three-level QP allocation, including quantization parameters for the core area, transition area, and background area. Among them, the QP value in the transition area is calculated according to an exponential decay model, including the following steps:

[0047] 1) ROI core area QP c Allocate the first QP value of 22 ± 2, and its boundary is determined through connected component analysis;

[0048] 2) ROI transition area QP t Allocate the second QP value of 28 ± 3. The width of this area is 5 - 15 pixels and the QP value is calculated according to the formula QP t = QP c + (QP b - QPc )×(1 - e -dβ ) for smooth transition, where d is the normalized distance from the pixel to the ROI boundary, β is the smoothing factor, 2.0 ≤ β ≤ 3.0. When the detected proportion of the ROI area is less than 10%, β is automatically increased by 20% - 50% to enhance the smoothing effect;

[0049] 3) QP for the ROI background area b Allocate the third QP value of 38 ± 5. When the target bitrate is lower than the preset threshold, according to the formula QP′ b = min(QP b + 3, 45) to increase the QP value of the background area.

[0050] Specifically, in step 3, referring to Figure 5 , input the QP mapping table into the encoder for block - level quantization control, and at the same time generate a visual interface to display the ROI detection results and the QP spatial distribution in real - time. The interface supports adjusting the saliency threshold and motion detection sensitivity through sliders.

[0051] The experiments in this embodiment use JCT - VC standard test sequences, covering three typical scenarios: for static complex scenarios, "BQSquare" (416×240) and "SlideEditing" (1280×720) are selected to focus on the texture detail processing ability; for general mixed scenarios, "ParkScene" (1920×1080) and "BasketballDrill" (832×480) are selected to test the general performance; for high - dynamic motion scenarios, "Traffic" (2560×1600) and "Runners" (1280×720) are selected to verify the motion adaptability.

[0052] In the QP allocation stage, the region division takes the first 10% of the pixels in the fusion graph as the core area, and expands an outer ring of 12 pixels as the transition area. The basic QP settings are 22 for the core area, 28 for the transition area (β = 2.5), and 38 for the background area, and a bitrate adaptive mechanism is introduced: when the target bitrate is lower than 1 Mbps, it is dynamically adjusted according to ΔQP = 2×log(target bitrate / current bitrate), and at the same time, the QP of the background area is restricted not to exceed 45.

[0053] In the ROI detection stage, for motion detection, an improved 3 - frame difference method is used, combined with a 5×5 Gaussian filter, σ = 0.5, to generate an energy map. The BORDER_REFLECT mode is used for boundary processing, and the motion threshold is adaptively determined by the Otsu algorithm. For visual saliency detection, first apply a Hann window to the input frame and normalize it, then extract the intermediate - frequency components after Fourier transform accelerated by FFTW3, and finally enhance the saliency map through 7×7 Gaussian smoothing and gamma correction with γ = 0.6.

[0054] The comparative test with the "Traffic" sequence as an example shows that: in terms of coding efficiency, the BD-rate(Y) index increases by 15.2%; in terms of quality evaluation, the PSNR increases by 1.5dB and the SSIM increases by 2%; in terms of time complexity, the encoding time increases by 16.7% while the decoding time remains stable. Especially in the "BQSquare" sequence, the ROI detection accuracy reaches 92.3%. The reasonable allocation of QP = 23 for the text region and QP = 39 for the background significantly improves the text readability while saving 13.7% of the bitrate.

[0055] The test data shows that under the premise of maintaining a reasonable computational complexity, this embodiment achieves an average BD-rate saving of 12% - 18%, and the block effect in the motion area is reduced by 76%. Through the innovative three-level QP allocation and dynamic smoothing mechanism, it performs excellently in texture detail preservation and motion processing, and is particularly suitable for video application scenarios with high requirements for real-time performance and compression ratio.

[0056] Embodiment 2

[0057] As Figure 6 shown, to achieve the above object, this embodiment discloses a VVC adaptive video coding system based on regions of interest, including:

[0058] 1. A video input processing module 11 for reading and parsing YUV videos and managing the frame buffer area;

[0059] 2. An ROI detection module 12 for performing motion detection, saliency detection, and ROI region determination based on thresholds;

[0060] 3. A QP mapping generation module 13 for generating a QP value mapping according to the ROI result and smoothing the QP value;

[0061] 4. An encoding control module 14 for passing the QP mapping table to the encoder and then controlling the encoding parameters;

[0062] 5. A visualization interface module 15 for displaying the original frame, ROI region, and QP distribution on the visualization interface.

[0063] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An adaptive video coding method for VVC based on regions of interest, characterized in that, Including the following steps: S1. Perform hybrid Region of Interest (ROI) detection on the input video frames, and generate an ROI mask based on the weighted fusion of the motion energy map and the visual saliency map; S2. Generate a quantization parameter QP mapping table according to the ROI mask, and perform three-level QP allocation, including quantization parameters for the core region, the transition region, and the background region, where the QP value of the transition region is calculated according to the exponential decay model; S3. Input the QP mapping table into the encoder for block-level quantization control, and generate a visualization interface to display the ROI detection results and the QP spatial distribution in real time. The interface supports adjusting the saliency threshold and the motion detection sensitivity through sliders.

2. The VVC adaptive video coding method based on region of interest according to claim 1, wherein The step S1 includes the following steps: S21. Read the input YUV, extract the Y components of the current frame and the previous frame, and perform 2D convolution on the inter-frame difference result of the current frame and the previous frame using a Gaussian filter with a kernel size of 5×5 to generate a motion energy map. The formula for inter-frame difference is D t (x,y) = |Y t (x,y) - Y t-1 (x,y)|, where Y t represents the luminance component of the t-th frame, D t is the difference result, and the formula for Gaussian filter is where the standard deviation σ is 0.5; S22. Read the input YUV, extract the Y component of the current frame, perform preprocessing of normalization and Hanning window on the current frame, and perform a fast Fourier transform on the preprocessed result f(x, y). Extract the corresponding phase φ(u, v) = ∠F(u, v), then construct an intermediate frequency band-pass filter M(u, v), retain 30%-70% of the intermediate frequency components in the frequency components, and then perform phase spectrum reconstruction through inverse Fourier transform. Finally, perform post-processing of Gaussian smoothing and normalization to generate a visual saliency map. S23. Perform weighted fusion of the motion energy map and the visual saliency map with a weight coefficient α, where 0.3 ≤ α ≤ 0.

7. The selection of α depends on the scene type. Eliminate detection holes through morphological closing operation based on an elliptical kernel, and output the ROI mask.

3. The VVC adaptive video coding method based on region of interest according to claim 1, wherein The step S2 includes the following steps: S31, QP of the ROI core area c Allocate the first QP value to be 22 ± 2, and its boundary is determined by connected component analysis; S32, QP of ROI Transition Region t Allocate the second QP value to be 28 ± 3, the width of this region is 5 - 15 pixels and the QP value is calculated according to the formula Smooth transition, where d is the normalized distance of the pixel to the ROI boundary, β is the smoothing factor 2.0 ≤ β ≤ 3.

0. When it is detected that the proportion of the ROI area is less than 10%, automatically increase β by 20% - 50% to enhance the smoothing effect; S33, QP of ROI background area b Allocate the third QP value to 38 ± 5. When the target bitrate is lower than the preset threshold, according to the formula QP b ' = min(QP b + 3, 45) to increase the QP value of the background area.

4. A VVC adaptive video coding system for implementing the method according to claims 1-3, characterized in that, Including: S41. Video input processing module, which reads and parses the YUV video and manages the frame buffer area; S42. ROI detection module, which includes a motion detection algorithm, a saliency detection algorithm, and a threshold-based ROI region determination; S43. QP mapping generation module, which generates a QP value mapping according to the ROI result and smooths the QP value; S44. Encoding control module, which transfers the QP mapping table to the encoder and then performs encoding parameter control; S45. Visualization interface module, which displays the original frame, the ROI region, and the QP distribution on the visualization interface.

Images