Adaptive quantization parameter adjustment method of joint coding pre-filtering
By introducing adaptive quantization parameter adjustment technology into GBTF, using matching error and similarity calculations to optimize quantization parameters, the compression efficiency loss problem caused by quantization loss during the encoding process of GBTF is solved, and the HEVC and VVC standards are realized. Improve compression efficiency.
Patent Information
- Application Number
- CN202410911011.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-09
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The latest pre-coding filtering technology GBTF has lost compression efficiency due to quantization loss during the encoding process, and no optimization algorithm is found from the perspective of adjusting adaptive quantization parameters.
Introduce adaptive quantization parameter adjustment technology, using the matching errors generated during the GBTF time domain block matching process, combining intra/out-of-frame similarity calculation and encoding unit-level parameter quantization, adjusting the quantization parameters to optimize encoder efficiency, and multiplexing GBTF calculations As a result, additional algorithm complexity is reduced.
The compression efficiency of the encoder has been significantly improved, with the increased encoding time of only 0.4%. At the same time, the PSNR and SSIM indicators on the HEVC and VVC standards have been increased by 3.5%, 7.6%, or 2.0% and 4.0%, respectively, and the GBTF algorithm effect has been optimized. .
Smart Images

Figure CN119071487A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of digital high-definition video encoding and decoding, and in particular relates to an adaptive quantization parameter adjustment method for joint pre-encoding filtering. Background Art
[0002] In order to further improve the compression efficiency of video codecs, technologies such as pre-coding filtering and adaptive quantization parameter (QP) adjustment have been proposed. The above methods do not require the modification of the decoder and can be applied to encoders of various standards because they do not require the modification of the decoder. Pre-coding filtering technology aims to perform operations such as background blurring, edge sharpening, and noise removal on the video to be encoded to improve the objective or subjective quality of the video after encoding. The adaptive quantization technology aims to set different QPs for different frames in the video being encoded and different coding units (CUs) in a frame to improve compression efficiency.
[0003] The most novel pre-coding filtering technology is called picture group-based temporal filtering (GBTF), which has been applied to the reference software of the high-efficiency video coding (HEVC) standard and the versatile codec (VVC) standard. GBTF performs time-domain block matching on the video to be encoded in units of picture groups (GOP) before encoding, and performs bilateral filtering based on the matching results, which facilitates the encoder to perform inter-frame prediction on the filtered video. However, since the quantization process of the encoder will introduce quantization loss, there will be a difference between the original pixels after GBTF filtering and the reconstructed pixels after encoding, and this difference will cause the algorithm effect of GBTF to suffer a loss. Except for the present invention, there is currently no report on the algorithm for optimizing GBTF from the perspective of adjusting the adaptive quantization parameters. Summary of the invention
[0004] The object of the present invention is to provide a method for adjusting the adaptive quantization parameter of the joint pre-coding filter to optimize the GBTF algorithm and improve the efficiency of the encoder.
[0005] In order to solve the problem of GBTF compression efficiency loss caused by quantization loss in the encoding process, the present invention introduces an adaptive quantization parameter adjustment technology. While significantly improving the compression efficiency of the encoder, the calculation results of GBTF are reused to achieve adaptive quantization parameter adjustment with extremely low additional algorithm complexity.
[0006] The method for adjusting the adaptive quantization parameter of the joint pre-coding filter provided by the present invention is based on a software video encoding and decoding platform including GBTF technology. The top-level block diagram of the encoder system involved in the present invention is as follows: Figure 1As shown. It includes a time domain filtering module based on image groups, an adaptive quantization module, and an encoder; wherein: the original pixels of the video to be encoded are input to the time domain filtering module based on image groups, processed by the time domain filtering module, and the filtered pixels are output and input to the encoder; at the same time, the original pixels of the video to be encoded are input to the adaptive quantization module, and the time domain filtering module outputs the matching errors generated in the time domain block matching process and inputs them to the adaptive quantization module; the adaptive quantization module processes the input original pixels and the matching errors, outputs the adjusted quantization parameters, and inputs them to the encoder; the encoder encodes the input filtered pixels and the quantization parameters, and outputs the encoded bit stream.
[0007] The processing contents of the time domain filtering module include: time domain block matching calculation based on motion estimation, filter weight calculation, and bilateral filter calculation; the adaptive quantization module includes: intra-frame / extra-frame similarity calculation, inter-frame prediction type judgment, and coding unit level parameter quantization calculation. Figure 2 shown.
[0008] The present invention does not modify the temporal filtering part based on the image group (i.e., uses the existing technology) [1], but only reuses the matching error generated in the temporal block matching process for the adaptive quantization parameter adjustment to reduce the computational complexity required by the present invention. Figure 2 As shown, the calculation steps in the adaptive quantization module are as follows:
[0009] Step 1: Intra-frame / extra-frame similarity calculation, that is, the input is the original pixel, and the output is the intra-frame similarity. The specific process is as follows:
[0010] The luminance channel of the input original image is traversed in units of the largest coding unit (LCU), and the size of the LCU is the same as the configuration of the encoder;
[0011] For each LCU, divide it into blocks of size 8x8 and calculate the gradient of each 8x8 block:
[0012]
[0013] Among them, g b represents the gradient of 8x8 block level, o x,y Represents the original pixel with coordinates x,y within the block.
[0014] Use the average gradient of all 8x8 blocks under an LCU as the gradient of the current LCU, denoted as g c ;
[0015] Use the gradient of an LCU to calculate its intra-frame similarity:
[0016]
[0017] Among them, Cintra is the intra-frame similarity; γ is a hyperparameter, for the HEVC standard encoder, γ is 4.5, for the VVC standard encoder, γ is 5.1; μ is a hyperparameter, for the HEVC standard, μ is 1.5, for the VVC standard, μ is 1.0.
[0018] When calculating intra-frame similarity, the input is 8x8 block-level matching error, and the output is inter-frame similarity:
[0019] GBTF performs block matching in units of 8x8 blocks, and the error of block matching is measured by the mean square error (MSE) between the original pixels and the matched pixels, that is, the present invention extracts the MSE of each 8x8 block in a frame from GBTF.
[0020] The average value of the MSE of all 8x8 blocks under an LCU is used as the MSE of the current LCU.
[0021] The PSNR (peak signal-to-noise ratio) is calculated using MSE, and the PSNR of each LCU is used as its inter-frame similarity C inter .
[0022] Step 2: Determine the inter-frame prediction type. The specific process is as follows:
[0023] When judging the inter-frame prediction type, the input is the inter-frame similarity and intra-frame similarity of the LCU, and the output is the LCU-level prediction type.
[0024] If the intra-frame similarity of the current LCU is higher than the inter-frame similarity, it is considered that intra-frame prediction should be adopted, otherwise it is considered that inter-frame prediction should be adopted.
[0025] Step 3: Calculate the coding unit level quantization parameter. The specific process is as follows:
[0026] If the current LCU is considered to use inter-frame prediction in step 2, its QP offset is calculated using the following formula:
[0027]
[0028] α is a hyperparameter. For the HEVC standard encoder, α is 0.8; for the VVC standard encoder, α is 1.2.
[0029] The final QP of the current LCU is calculated according to the following formula:
[0030] QP=QP frame +ΔQP, (4)
[0031] Among them, QP frameThe frame-level quantization parameter used by the current frame to be encoded is determined by the configuration file used by the encoder, and the LCU-level quantization parameter QP is equal to the sum of the frame-level quantization parameter used by the frame to which it belongs and the LCU-level quantization parameter offset ΔQP calculated by the present invention.
[0032] Experiments have shown that the present invention only increases the encoding time by 0.4% on the reference software HM18.0 of the HEVC standard, and improves the compression efficiency by 7.6% (measured by SSIM) or 3.5% (measured by PSNR). On the VTM19.0 software platform, the present invention can improve the compression efficiency by 4.0% (measured by SSIM) or 2.0% (measured by PSNR). BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is a framework diagram of the adaptive quantization parameter adjustment method for joint pre-coding filtering according to the present invention.
[0034] Figure 2 The present invention is a flowchart illustrating a method for adjusting an adaptive quantization parameter of a joint pre-coding filter. DETAILED DESCRIPTION
[0035] The present invention is further described below by way of embodiments in conjunction with the accompanying drawings.
[0036] The method for adjusting the adaptive quantization parameter of the joint pre-coding filter proposed in the present invention is specifically divided into the following steps:
[0037] (1) In the pre-coding filtering files (TEncTemporalFilter.h and TEncTemporalFilter.cpp) of the HEVC standard official reference code HM18.0 or the VVC standard official reference code VTM19.0, the adaptive quantization parameter adjustment algorithm of the present invention is implemented. The specific process is as follows: Figure 1 .
[0038] (2) Set the gradient g used in step 2 c Calculate the intra-frame similarity C intra The hyperparameter γ in Equation 2 (4.5 in HM18.0 and 5.1 in VTM19.0), and the hyperparameter μ (1.5 in HM18.0 and 1.0 in VTM19.0).
[0039] (3) Set the intra-frame similarity C used in step 3 intra Similarity between frames C inter The hyperparameter α in Formula 3 (0.8 in HM18.0 and 1.2 in VTM19.0) for calculating the LCU-level quantization parameter bias ΔQP.
[0040] (4) Before the rate-distortion optimization (RDO) process of HM18.0 and VTM19.0, the final quantization parameter QP is calculated for each LCU according to Formula 4.
[0041] (5) Compile the updated software code and test the compression performance of the algorithm proposed in the present invention according to the test requirements of JCTVC-L1100[2] (for the encoder of the HEVC standard) and JVET-T2010[3] (for the encoder of the VVC standard). The test conditions of the present invention are as follows: the encoded video sequence is the official test sequence of the HEVC standard specified in JCTVC-L1100[2], the number of encoded frames is 65 frames, the encoding structure is random access, the encoding mode is the fixed quantization parameter mode, and 4 quantization parameters (22, 27, 32, 37) are used for encoding, and the remaining configuration parameters are the default values specified in JCTVC-L1100[2] and JVET-T2010[3].
[0042] (6) The test results of the algorithm proposed in the present invention in HM18.0 and VTM19.0 are shown in Table 1 and Table 2 respectively. Among them, BIM[4] is a similar method of the present invention. As shown in Table 1, the present invention is applied in HM18.0, and with the peak signal-to-noise ratio (PSNR) as the measurement index, it can improve the compression performance by an average of 3.5%, and with the structural similarity (SSIM) as the measurement index, it can improve the compression performance by an average of 7.6%, both of which are higher than the similar method BIM[4]. As shown in Table 2, the present invention is applied in VTM19.0, and with the peak signal-to-noise ratio (PSNR) as the measurement index, it can improve the compression performance by an average of 2.0%, and with the structural similarity (SSIM) as the measurement index, it can improve the compression performance by an average of 4.0, both of which are higher than the similar method BIM[4].
[0043] Table 1, test results of the compression performance of the method of the present invention on the HM18.0 platform
[0044]
[0045] Table 2, test results of the compression performance of the method of the present invention on the VTM19.0 platform
[0046]
[0047] References
[0048] [1]P.Wennersten,“Encoder-only gop-based temporal filter,”ITU-T SG16WP 3 and ISO / IECJTC 1 / SC 29 / WG 11,Document:JCTVC-AI0023-v2,2019.
[0049] [2]Frank Bossen et al.,“Common test conditions and software referenceconfigurations,”JCTVC-L1100,vol.12,no.7,2013.
[0050] [3]F Bossen,J Boyce,X Li,V Seregin,and K S uhring,“VTM common testconditions and software reference configurations for SDR video(JVET-T2010),”Joint Video Experts Team,2020.
[0051] [4]Jack Enhorn,Christopher Hollmann,Rickard Sj oberg,Jacob Str om,andPer Wennersten,“Block Importance Mapping for Video Encoding,”in 2022 IEEEInternational Conference on Visual Communications and Image Processing(VCIP).IEEE,2022,pp.1–4。
Claims
1. A method for adjusting an adaptive quantization parameter of a joint pre-coding filter, characterized in that: The invention is based on a software video encoding and decoding platform including GBTF technology, and includes a time domain filtering module based on image groups, an adaptive quantization module, and an encoder; wherein: the original pixels of the video to be encoded are input into the time domain filtering module based on image groups, processed by the time domain filtering module, and the filtered pixels are output and input into the encoder; at the same time, the original pixels of the video to be encoded are input into the adaptive quantization module, and the time domain filtering module outputs the matching errors generated in the process of time domain block matching, and inputs into the adaptive quantization module; the adaptive quantization module processes the input original pixels and the matching errors, outputs the adjusted quantization parameters, and inputs into the encoder; the encoder encodes the input filtered pixels and the quantization parameters, and outputs the encoded bit stream.
2. The method for adjusting the adaptive quantization parameter of joint pre-coding filtering according to claim 1, characterized in that: The processing contents of the time domain filtering module include: time domain fast matching calculation based on motion estimation, filter weight calculation, and bilateral filtering calculation; the processing contents of the adaptive quantization module include: intra-frame / extra-frame similarity calculation, inter-frame prediction type judgment, and coding unit level parameter quantization calculation; the part of the time domain filtering based on the image group is the existing technology, and here only the matching error generated in the time domain block matching process is reused for adaptive quantization parameter adjustment; the calculation steps in the adaptive quantization module are as follows: Step 1: Intra-frame / extra-frame similarity calculation, the input is the original pixel, and the output is the intra-frame similarity: The specific process is: The luminance channel of the input original image is traversed in units of the largest coding unit (LCU), and the size of the LCU is the same as the configuration of the encoder; For each LCU, divide it into blocks of size 8x8 and calculate the gradient of each 8x8 block: Among them, g b represents the gradient of 8x8 block level, o x,y Represents the original pixel with coordinates x,y in the block; Use the average gradient of all 8x8 blocks under an LCU as the gradient of the current LCU, denoted as g c ; Use the gradient of an LCU to calculate its intra-frame similarity: Among them, C intra is the intra-frame similarity, γ and μ are hyper parameters; When calculating intra-frame similarity, the input is 8x8 block-level matching error, and the output is inter-frame similarity: GBTF performs block matching in units of 8x8 blocks, and the error of block matching is measured as the mean square error (MSE) between the original pixels and the matched pixels, that is, the MSE of each 8x8 block in a frame is extracted from GBTF; Use the average MSE of all 8x8 blocks under an LCU as the MSE of the current LCU; The PSNR is calculated using MSE, and the PSNR of each LCU is used as its inter-frame similarity C inter ; Step 2: Inter-frame prediction type judgment. The input is the inter-frame similarity and intra-frame similarity of LCU, and the output is the LCU-level prediction type. The specific process is as follows: If the intra-frame similarity of the current LCU is higher than the inter-frame similarity, it is considered that intra-frame prediction should be adopted, otherwise it is considered that inter-frame prediction should be adopted; Step 3: Calculate the coding unit level quantization parameter. The specific process is as follows: If the current LCU is considered to use inter-frame prediction in step 2, its QP offset is calculated using the following formula: α is a parameter; The final QP of the current LCU is calculated according to the following formula: QP=QP frame +ΔQP, (4) Among them, QP frame Determined by the profile used by the encoder.
3. The method for adjusting the adaptive quantization parameter of joint pre-coding filtering according to claim 2, characterized in that: The values of parameters γ and μ are as follows: for the HEVC standard encoder, γ is 4.5 and μ is 1.5; for the VVC standard encoder, γ is 5.1 and μ is 1.
0.
4. The method for adaptive quantization parameter adjustment of joint pre-coding filtering according to claim 2, characterized in that: The value of parameter α is as follows: for the HEVC standard encoder, α is 0.8; for the VVC standard encoder, α is 1.2.
Citation Information
Patent Citations
Image decoding device, image encoding device, and method thereof
CN103329536A
Low complexity adaptive quantization for video compression
CN115053524A
Preprocessing method for video coding
CN117834874A
Content adaptive motion compensated temporal filtering for denoising of noisy video for efficient coding
US20180343448A1
