Rate-distortion optimization quantization method, device, equipment and product based on H264

Abstract

The present application relates to a rate-distortion optimization quantization method, apparatus, equipment and product based on H264. Applied to an encoder, the encoder implements the H264 / AVC standard. The method includes: receiving a video frame to be processed; dividing and transforming the video frame to be processed to obtain multiple first transform blocks; using a rate-distortion optimization quantization algorithm to perform parallel quantization processing on the first transform coefficients in the first transform block to obtain pre-quantization coefficients; then fine-tuning the pre-quantization coefficients. If it is an 8x8 transform block, several non-zero coefficients at the end of the scan order are set to zero. Whether the non-zero coefficients are set to zero depends on whether the encoding cost can be minimized. After the above steps, the second transform coefficients after rate-distortion optimization quantization are obtained. This method can process quantization coefficients in parallel, improve the processing speed of hardware, reduce the processing delay of hardware, and can also adapt to the H264 / AVC standard at the same time, share hardware resources as much as possible, and reduce the area of the rate-distortion optimization quantization module.

Description

Technical Field

[0001] The present invention relates to the technical field of video compression, and in particular to a rate-distortion optimization quantization method, device, equipment and product based on H264. Background Art

[0002] Among related technologies, scalar quantization cannot achieve optimal performance due to its fixed dead zone. However, rate-distortion optimized quantization (RDOQ) can maximize compression performance by considering the rate-distortion penalty during the quantization process. RDOQ compares the rate-distortion penalties of multiple quantization schemes and selects the one with the lowest rate-distortion penalty to achieve optimal performance. Quantization schemes can be determined by different quantization parameters (QPs) or quantization offsets. RDOQ technology originated in the H264 / AVC video coding standard.

[0003] In the related art, an early RDOQ method was proposed. This method selects the optimal QP for each coding block to achieve the optimization purpose. In the related art, a further improved method was also proposed, which uses some fast algorithms to remove some unnecessary candidate QPs to reduce the number of traversals. However, these methods are not suitable for applications with rate control enabled, because rate control will further change the QP of each block to achieve the effect of balancing the bit rate. Among them, the key to RDOQ is to calculate the rate-distortion cost of each quantization scheme. The rate-distortion cost can be defined as:

[0004]

[0005] Where C is the rate-distortion cost, D is the distortion between the original and decoded signals, R is the encoding bit rate, and λ is the rate-distortion parameter. D can be calculated by calculating the difference between the original and decoded signals. However, due to the sequential execution nature of the entropy encoder and the high complexity of the context model, an accurate bit rate R is often difficult to obtain. Related art proposes a method for directly inputting quantized coefficients into the entropy encoder to obtain the bit rate R. However, this still requires a large number of operations in the entropy encoder and is therefore not suitable for low-latency scenarios.

[0006] To reduce RDOQ's reliance on the entropy encoder, it employs a number of techniques to quickly estimate distortion D and bitrate R. First, the inverse transform step is skipped to directly calculate distortion D. Then, the bitrate R is estimated by querying a large number of trained entropy coding tables, eliminating the need to rely on the entropy encoder. In related art, a simple and effective RDOQ method for H265 / HEVC has been proposed. This method attempts to change the spatial distribution of non-zero coefficients and reduce the absolute values of non-zero coefficients to minimize the rate-distortion cost. For quantized residual coefficients, the position and absolute value of non-zero coefficients account for a large portion of the encoding bitrate. RDOQ attempts to fine-tune these positions and absolute values so that entropy coding can compress information more efficiently. Generally speaking, RDOQ can reduce the encoding bitrate R, but increase the encoding distortion D. However, by minimizing the rate-distortion cost C, the final encoding quality is improved.

[0007] Related technologies have also proposed a fast algorithm that determines whether RDOQ is needed by checking the product of the quantization coefficient and the QP. If the product is less than a certain threshold, RDOQ is activated. Otherwise, RDOQ is disabled.

[0008] However, the RDOQ method in related art requires that quantized coefficients arrive in a predefined order and be processed sequentially. This is feasible for software implementation, but suboptimal for hardware implementation. Specifically, when the RDOQ module processes a specific quantized coefficient, the processing of the previous quantized coefficient must have completed. This dependency makes parallel processing of multiple quantized coefficients difficult, thus preventing the full utilization of the parallel capabilities of the hardware.

[0009] Current RDOQ methods in H264 / AVC encoders typically optimize the coding cost by adjusting the QP. However, this approach cannot be used in conjunction with rate control, as rate control further modifies the QP. Furthermore, some RDOQ schemes utilize adaptive methods to generate quantization offset parameters to optimize the coding cost, and further propose using lookup tables to estimate the coding bitrate of the quantization coefficients. However, this approach only applies to H265 / HEVC encoders and is not suitable for H264 / AVC. Summary of the Invention

[0010] The purpose of this application is to provide a rate-distortion optimization quantization method, device, equipment and product based on H264, which can process quantization coefficients in parallel, reduce hardware processing delay, and adapt to H264 / AVC at the same time, share hardware resources as much as possible, and reduce the area of the rate-distortion optimization quantization module.

[0011] According to a first aspect of an embodiment of the present application, a rate-distortion optimization quantization method based on H.264 is provided, which is applied to an encoder that implements the H.264 / AVC standard. The method includes:

[0012] Receive video frames to be processed;

[0013] Dividing and transforming the video frame to be processed to obtain a plurality of first transformation blocks;

[0014] performing parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantized coefficients;

[0015] The prequantization coefficients and a zeroing rule corresponding to the transform block size of the first transform block are used to set corresponding coefficients in the first transform block to zero, thereby obtaining second transform coefficients after rate-distortion optimized quantization.

[0016] In one embodiment, performing parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients includes:

[0017] After initializing the context model to predefined values, define the quantization offset parameter and the quantization scaling parameter;

[0018] The pre-quantization coefficient is calculated based on the quantization offset parameter and the quantization scaling parameter; the calculation formula is as follows:

[0019] ;

[0020] Among them, TCoeff[cIdx][x][y] is the transform coefficient, cIdx is the Y, U, and V component indexes, x and y are the horizontal and vertical coordinates of the transform coefficient in the transform block, sign[cIdx][x][y] is the sign of the transform coefficient, and QCoeff[cIdx][x][y] is the pre-quantization result.

[0021] In one embodiment, the first transform block includes: a 4×4 transform block and an 8×8 transform block.

[0022] In one embodiment, the calculation method of the quantization offset parameter includes:

[0023] If the first transform block is a 4×4 transform block, the calculation formula of the quantization offset parameter is:

[0024] ;

[0025] If the first transform block is an 8×8 transform block, the calculation formula of the quantization offset parameter is:

[0026] ;

[0027] In one embodiment, the calculation method of the quantization scaling parameter includes:

[0028] If the first transform block is a 4×4 transform block, the calculation formula of the quantization scaling parameter is:

[0029] ;

[0030] If the first transform block is an 8×8 transform block, the calculation formula of the quantization scaling parameter is:

[0031] ;

[0032] Among them, scalingList4x4 can be obtained by querying the 4x4 quantization matrix table (that is, Table 1), and scalingList8x8 can be obtained by querying the 8x8 quantization matrix table (that is, Table 2).

[0033] In one embodiment, fine-tuning a non-zero pre-quantization coefficient and then setting the fine-tuned quantization coefficient to zero to obtain a second transform coefficient after rate-distortion optimized quantization includes: using the pre-quantization coefficient to calculate the first coding costs corresponding to three fine-tuning methods respectively, and determining a transform coefficient fine-tuning method with a smaller coding cost based on the calculation result of the first coding cost; after fine-tuning the non-zero transform coefficient, calculating the second coding cost corresponding to the fine-tuned non-zero transform coefficient; determining a zeroing rule based on the second coding cost, setting the corresponding transform coefficient in the first transform block to zero, and obtaining the second transform coefficient after rate-distortion optimized quantization.

[0034] In one embodiment, the calculating the first coding costs of the three fine-tuning methods using the prequantization coefficients, and determining the transform coefficient processing method with the lowest coding cost based on the calculation results of the first coding costs, includes:

[0035] After obtaining the prequantization coefficient QCoeff[cIdx][x][y], if the prequantization coefficient is non-zero, then calculating the coding cost after returning the non-zero prequantization coefficient to zero, which is the first fine-tuning method;

[0036] Calculate the coding cost of keeping the sign of the pre-quantized coefficient unchanged and reducing the absolute value by one, which is the second fine-tuning method;

[0037] Calculate the coding cost of keeping the pre-quantization coefficient unchanged, which is the third fine-tuning method;

[0038] Compare the encoding costs of the above three fine-tuning methods, and then choose the processing method with the lowest encoding cost.

[0039] In one embodiment, after obtaining the prequantization coefficient QCoeff[cIdx][x][y], the method further includes:

[0040] Calculate quantization error and encoding bit rate;

[0041] The quantization error is calculated as follows:

[0042] ;

[0043] If the first transform block is a 4×4 transform block, the quantization error restoration process is as follows:

[0044] ;

[0045] If the first transform block is an 8×8 transform block, the quantization error restoration process is as follows:

[0046] ;

[0047] Among them, distortionScaler4x4 is determined based on the 4x4 distortion weight factor table, and distortionScaler8x8 is determined based on the 8x8 distortion weight factor table;

[0048] The encoding bit rate is calculated as follows:

[0049] ;

[0050] Among them, sigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is zero; lastSigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is the last non-zero quantization coefficient in the first transform block; coeffAbsLevelRate[cIdx][x][y] is the rate estimate of the syntax element that encodes the absolute value of the non-zero quantization coefficient.

[0051] In one embodiment, the sigCoeffFlagRate[cIdx][x][y] is calculated as follows:

[0052] ;

[0053] Where sigCoeffFlagTabIdx is determined by x, y and the type of the current coding block, and rateEstTab is the rate estimation table.

[0054] In one embodiment, the lastSigCoeffFlagRate[cIdx][x][y] is calculated as follows:

[0055] ;

[0056] Where lastSigCoeffFlagTabIdx is determined by x, y and the type of the current coding block.

[0057] In one embodiment, the coeffAbsLevelRate[cIdx][x][y] is calculated as follows:

[0058] ;

[0059] in,

[0060] ;

[0061] greaterThanOneFlagTabIdx is always 1,

[0062] ;

[0063] Where GolombCodeLength is the code length after Golomb-Rice encoding the absolute value of QCoeff[cIdx][x][y], and eqProbRateEstTab is the equal probability rate estimation table.

[0064] In one embodiment, after fine-tuning the non-zero transform coefficient, calculating a second coding cost corresponding to the fine-tuned non-zero transform coefficient includes:

[0065] The fine-tuning non-zero transform coefficients are calculated as follows:

[0066] ;

[0067] Among them, cost0[cIdx][x][y] is the coding cost of setting non-zero prequantization coefficients to zero, cost1[cIdx][x][y] is the coding cost of keeping the sign of non-zero prequantization coefficients unchanged but reducing the absolute value by one, and cost2[cIdx][x][y] is the coding cost of keeping the non-zero prequantization coefficients unchanged.

[0068] The second encoding cost is calculated as follows:

[0069] .

[0070] According to a second aspect of an embodiment of the present application, there is provided a rate-distortion optimization quantization device based on H.264, which is applied to an encoder that implements the H.264 / AVC standard. The device includes:

[0071] A receiving module, configured to receive a video frame to be processed;

[0072] a transform processing module, configured to divide and transform the video frame to be processed to obtain a plurality of first transform blocks;

[0073] a quantization processing module, configured to perform parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients;

[0074] The coefficient processing module is used to use the pre-quantization coefficients and the fine-tuning method and zeroing rule corresponding to the transform block specification of the first transform block to fine-tune and zero the corresponding coefficients in the first transform block to obtain the second transform coefficients after rate-distortion optimized quantization.

[0075] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the above method.

[0076] According to a fourth aspect of an embodiment of the present application, a computer program product is provided, on which a computer program is stored, characterized in that when the executable computer program in the computer program product is executed by a processor, the above method can be implemented.

[0077] Compared with the prior art, the present application has the following advantages: an encoder implements the H264 / AVC standard and receives a video frame to be processed; the video frame to be processed is divided and transformed to obtain multiple first transform blocks; first transform coefficients in the first transform block are parallel quantized using a rate-distortion optimized quantization algorithm to obtain prequantization coefficients; and the prequantization coefficients are fine-tuned using a fine-tuning method with the lowest coding cost. The fine-tuned prequantization coefficients and the zeroing rule corresponding to the transform block specification of the first transform block are used to zero the corresponding coefficients in the first transform block to obtain second transform coefficients after rate-distortion optimized quantization. Specifically, the transform coefficients in the transform block are first quantized with a fixed dead zone, and then each quantization coefficient is fine-tuned. After quantization and fine-tuning are completed, for a 4x4 transform block, the entire transform block is considered to be zeroed based on the rate-distortion cost. For an 8x8 transform block, the last non-zero quantized coefficient is considered to be zeroed based on the reverse scan order, and then the entire transform block is considered to be zeroed based on the rate-distortion cost. To minimize the consumption of hardware resources, the present invention does not require any floating-point operations, only integer addition, subtraction, and shift calculations up to 32 bits. Furthermore, the calculation of the otherwise complex coding cost is achieved using a simple table lookup. In summary, the technical solution provided by this application can process quantization coefficients in parallel, improve hardware processing speed, and reduce hardware processing delay. Furthermore, it can simultaneously adapt to the H.264 / AVC standard, share hardware resources as much as possible, and reduce the area of the rate-distortion optimization quantization module. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a block diagram of an encoder according to an exemplary embodiment.

[0079] Figure 2 The figure is a flowchart showing a rate-distortion optimized quantization method based on H264 according to an exemplary embodiment.

[0080] Figure 3 FIG. 4 is a flowchart showing a working process of RDOQ at the TU layer according to another exemplary embodiment.

[0081] Figure 4 FIG. 4 is a schematic diagram of a scanning order of a 4×4 TU according to an exemplary embodiment.

[0082] Figure 5 FIG. 4 is a schematic diagram of a scanning order of an 8x8 TU according to an exemplary embodiment.

[0083] Figure 6 The figure is a flowchart of coefficient fine-tuning according to an exemplary embodiment.

[0084] Figure 7 FIG. 4 is a flowchart showing how to set non-zero coefficients of an 8x8 TU at the end of a scanning order to zero according to an exemplary embodiment.

[0085] Figure 8 The figure is a block diagram showing a rate-distortion optimized quantization device based on H264 according to an exemplary embodiment.

[0086] Figure 9 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0087] Unless otherwise defined, the technical or scientific terms used in this specification and claims shall have the ordinary meanings understood by persons having ordinary skills in the technical field to which the invention belongs. Specific embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be noted that in the specific description of these embodiments, for the sake of clarity and clarity, this specification cannot provide a detailed description of all features of the actual embodiments. Without departing from the spirit and scope of the present invention, those skilled in the art may modify and replace the embodiments of the present invention, and the resulting embodiments are also within the scope of protection of the present invention.

[0088] Before introducing the rate-distortion optimization quantization method provided by this application, we first introduce the encoder and the position of the rate-distortion optimization quantization module in the encoder. Figure 1As shown, the encoder includes a transform module 11, a rate-distortion optimized quantization module 12, and an entropy coding module 13. The rate-distortion optimized quantization module 12 is located between the transform module 11 and the entropy coding module 13.

[0089] The encoder in this application is a video encoder. When a new video frame enters the encoder, it typically divides the video frame into a series of non-overlapping square or rectangular blocks. "Block" here essentially refers to a matrix. Given a target block, the encoder attempts to find the best prediction block for it and then subtracts the prediction block from the target block to obtain a residual block.

[0090] The transform block is configured to perform a frequency domain transform (typically, but not limited to, a discrete cosine transform) on the residual block. The output of the transform block is a transform coefficient (a transform coefficient matrix). In the H.264 / AVC video compression standard, the transform block is called a transform unit (TU).

[0091] The rate-distortion optimized quantization module (RDOQ) performs rate-distortion optimized quantization on the transform coefficients output by the transform block and outputs the quantization coefficient matrix (hereinafter referred to as the second quantization coefficient matrix) to the entropy coding module (CABAC). This module is responsible for quantizing the transform coefficients, fine-tuning nonzero coefficients, and conditionally setting them to zero.

[0092] The entropy coding module is configured to encode the quantization coefficient matrix output by the rate-distortion optimization quantization module. For example, CABAC (Context-Based Adaptive Binary Arithmetic Coding) encoding method may be used for encoding, but is not limited thereto.

[0093] Receive a video frame to be processed; divide and transform the video frame to obtain multiple first transform blocks; parallel quantize the first transform coefficients in the first transform block using a rate-distortion optimized quantization algorithm to obtain prequantization coefficients; then fine-tune the prequantization coefficients. Fine-tune method one sets non-zero prequantization coefficients to zero; fine-tune method two maintains the sign of the non-zero prequantization coefficients but reduces their absolute value by one; and fine-tune method three maintains the prequantization coefficients unchanged. The fine-tune method with the lowest coding cost is then selected. If the transform block is an 8x8 block, several non-zero coefficients at the end of the scan sequence are set to zero. Whether the non-zero coefficients are set to zero depends on whether the coding cost can be minimized. Finally, if the coding cost of setting all quantized coefficients of the entire first transform block to zero is less than the coding cost of not setting them to zero, then all quantized coefficients of the entire first transform block are set to zero; otherwise, the quantized coefficients remain unchanged. Following these steps, second transform coefficients are obtained after rate-distortion optimized quantization. This method can process quantization coefficients in parallel, improve hardware processing speed, reduce hardware processing delay, and can also adapt to the H264 / AVC standard, share hardware resources as much as possible, and reduce the area of the rate-distortion optimization quantization module.

[0094] The above briefly introduces the encoder and the position of the rate-distortion optimization quantization module in the encoder. The following is a detailed introduction to the rate-distortion optimization quantization module and its functions.

[0095] Figure 2 This is a flow chart showing a rate-distortion optimization quantization method based on H264 according to an exemplary embodiment. The rate-distortion optimization quantization method can be applied to an encoder. The encoder is adapted to the H264 / AVC standard. Figure 2 The rate-distortion optimization quantization method based on H264 may include the following steps:

[0096] Step 201: Receive a video frame to be processed.

[0097] Step 202: Divide and transform the video frame to be processed to obtain a plurality of first transformation blocks.

[0098] Step 203 : Perform parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients.

[0099] Step 204: Fine-tune the pre-quantization coefficient using a preset fine-tuning method.

[0100] Step 205 : Using the fine-tuned prequantization coefficients and the zeroing rule corresponding to the transform block size of the first transform block, the corresponding coefficients in the first transform block are set to zero to obtain second transform coefficients after rate-distortion optimized quantization.

[0101] First, the transform coefficients in the transform block are quantized with a fixed dead zone, and then each quantized coefficient is fine-tuned. After quantization and fine-tuning, for a 4x4 transform block, the entire transform block is considered zeroed based on rate-distortion cost. For an 8x8 transform block, the last non-zero quantized coefficient is considered zeroed based on the inverse scan order, and then the entire transform block is considered zeroed based on rate-distortion cost. To minimize hardware resource consumption, the present invention does not require any floating-point operations, requiring only integer addition, subtraction, and shift calculations up to 32 bits. Furthermore, this solution utilizes a simple table lookup to achieve the otherwise complex coding cost calculation. It should be noted that the first transform block includes at least one of the following transform block types: a 4x4 transform block and an 8x8 transform block. If the first transform block is an 8x8 transform block, the last non-zero quantized coefficient is considered zeroed based on the inverse scan order, and then the entire transform block is considered zeroed based on rate-distortion cost. This method effectively reduces scattered non-zero coefficients at the end of the 8x8 block, concentrating the non-zero coefficients in the upper left corner of the 8x8 block, thereby improving the compression performance of the entropy encoder. The H264 / AVC standard requires that the entropy encoder process the quantization coefficients of each TU in a predefined scanning order, such as a zigzag sequence. Therefore, when calculating the rate-distortion cost of a particular quantization coefficient, its context model depends on the coding state of all forward quantization coefficients. Therefore, the process must wait until all forward quantization coefficients are fully processed before it can begin, which greatly disrupts the hardware pipeline operation. However, the present invention fixes the value of the context model and does not need to follow the scanning order of the entropy coded quantization coefficients. Each quantization coefficient can be processed independently, thus greatly utilizing the pipeline to reduce hardware latency.

[0102] When a new frame enters the video encoder, it typically divides the frame into a series of non-overlapping square or rectangular blocks. Given a target block, the encoder attempts to find the best prediction block for it and then subtracts the prediction block from the target block to obtain a residual block. The transform module then performs a frequency domain transform (typically a discrete cosine transform) on the residual block. The output of the transform module is a transform coefficient block. The H264 / AVC video coding standard has two sizes of transform coefficient blocks: 4x4 and 8x8. In this patent, a transform block is also referred to as a TU.

[0103] like Figure 1 As shown in , RDOQ is located after the transform module. RDOQ first quantizes the transform coefficients using a uniform quantization offset, then adjusts the non-zero coefficients to minimize the rate-distortion penalty, and further sets the entire TU to zero when necessary. Figure 3 As shown in Figure 1, after parameter initialization, RDOQ starts to quantize each transform coefficient. There is no dependency between transform coefficients, so they can be processed in parallel. For non-zero quantization coefficients, RDOQ adjusts their absolute value. The specific steps are as follows: Figure 6As shown in the figure, after all quantization coefficients of a TU have been processed, RDOQ can set the entire TU to zero as needed. In addition, for an 8x8 TU, RDOQ attempts to set the quantization coefficients at the end with an absolute value of 1 to zero to save bit rate.

[0104] Pre-quantize the transform coefficients of the TU layer. The workflow of RDOQ at the TU layer is as follows: Figure 3 As shown, first initialize the context model of the syntax element. The context model list is:

[0105] ● significantCoeffCtx

[15]

[0106] ● lastSignificantCoeffCtx

[15]

[0107] ● greaterThanOneCoeffCtx[5]

[0108] ● levelAbsCoeffCtx[5]

[0109] Each element of these arrays is a 32-bit integer. These context models are initialized to predefined values, and the models are fixed throughout the entire process of RDOQ.

[0110] Then, RDOQ pre-quantizes one transform coefficient at a time from the TU. Let the quantization parameter be QP, x and y be the coordinates of the transform coefficient in the TU, and quantization is a right shift of the transform coefficient. The parameter defining the quantization shift, named quantizationShift, is generated as follows:

[0111]

[0112] Define the parameters of the quantization scaler, named quantizationScaler, and generate the following:

[0113]

[0114] Among them, scalingList4x4 can be obtained by querying the 4x4 quantization matrix table (that is, Table 1), and scalingList8x8 can be obtained by querying the 8x8 quantization matrix table (that is, Table 2).

[0115] Table 1

[0116]

[0117] Table 2

[0118]

[0119] In RDOQ, pre-quantization is specifically defined as:

[0120]

[0121] Where TCoeff[cIdx][x][y] is the transform coefficient, cIdx is the Y, U, V component index, sign[cIdx][x][y] is the sign of the transform coefficient, and QCoeff[cIdx][x][y] is the prequantization result.

[0122] The encoding cost calculation process is as follows:

[0123] RDOQ reduces the bitrate by minimizing the absolute value of nonzero coefficients, thereby optimizing the rate-distortion penalty. Given a quantized coefficient QCoeff[cIdx][x][y], if the quantized coefficient is nonzero, the coding cost of zeroing it is first calculated. Then, the coding cost of not zeroing the coefficient is calculated. After processing all quantized coefficients for a TU, RDOQ compares the coding cost of retaining the nonzero coefficients and zeroing the entire TU, and selects the method with the lower coding cost.

[0124] In order to obtain the coding cost, the quantization error should be calculated first:

[0125]

[0126] Since the quantization error is amplified by the transformation, the distortion is the reverse restoration of the quantization error:

[0127]

[0128] Among them, distortionScaler4x4 is determined based on the 4x4 distortion weight factor table (Table 3), and distortionScaler8x8 is determined based on the 8x8 distortion weight factor table (Table 4).

[0129] Table 3

[0130]

[0131] Table 4

[0132]

[0133] To calculate the complete encoding cost, in addition to the distortion, we also need to calculate the encoding bit rate. The bit rate can be modeled as:

[0134]

[0135] Where sigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantized coefficient is zero. The calculation method is:

[0136]

[0137] sigCoeffFlagTabIdx is determined by x, y, and the type of the current coding block. Furthermore, rateEstTab is the rate estimation table (Table 5, rateEstTab). Since the bitrate can be estimated simply by looking up the table, there's no need to invoke a complex entropy encoder, thus reducing complexity.

[0138] Table 5

[0139]

[0140] lastSigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantized coefficient is the last non-zero quantized coefficient in the scanning order within the TU. The calculation method is:

[0141]

[0142] Where lastSigCoeffFlagTabIdx is determined by x, y and the type of the current coding block. The H264 / AVC standard defines the scanning order of quantization coefficients within a TU. The scanning order of a 4x4 TU is as follows: Figure 4 As shown, the scanning order of 8x8 TU is as follows Figure 5 As shown. The entropy encoder writes the quantized coefficients into the bitstream according to this scanning order. Although the present invention does not need to follow this scanning order to process the quantized coefficients, the position of the last non-zero quantized coefficient is determined by this scanning order.

[0143] If QCoeff[cIdx][x][y] is non-zero, the absolute value of QCoeff[cIdx][x][y] must also be encoded. coeffAbsLevelRate[cIdx][x][y] is the rate estimate of the syntax element that encodes the absolute value of the non-zero quantized coefficient. It is calculated as follows:

[0144]

[0145] in,

[0146]

[0147] Where greaterThanOneFlagTabIdx is always 1.

[0148]

[0149] Where GolombCodeLength is the code length after the absolute value of QCoeff[cIdx][x][y] is encoded with Golomb-Rice code, and eqProbRateEstTab is determined by the equal probability rate estimation table (Table 6).

[0150] Table 6

[0151]

[0152] In summary, given the quantization coefficient QCoeff[cIdx][x][y], the transform coefficient TCoeff[cIdx][x][y], the quantization parameter QP and bitDepth, the encoding cost is:

[0153]

[0154] The encoding cost process after fine-tuning the coefficients is as follows:

[0155] For non-zero quantization coefficients, RDOQ attempts to reduce their absolute values by 1 or set them to zero to reduce the encoding bit rate. Figure 6 The details of coefficient fine-tuning are described. RDOQ fine-tunes the absolute value of the quantized coefficient but keeps its sign unchanged. RDOQ will try to test three potential quantized coefficients: 0, sign[cIdx][x][y]x(|QCoeff[cIdx][x][y]|-1), and QCoeff[cIdx][x][y]. The candidate coefficient with the smallest coding cost will be selected as the new quantized coefficient. For the first candidate coefficient 0, the coding cost cost0[cIdx][x][y] is calculated using Eq.12, where QCoeff[cIdx][x][y]=0. If |QCoeff[cIdx][x][y]|>1, the second candidate coefficient is sign[cIdx][x][y]x(|QCoeff[cIdx][x][y]|-1), and the coding cost cost1[cIdx][x][y] is calculated using Eq.12. The third candidate coefficient is the original QCoeff[cIdx][x][y]. Eq.12 is still applied to calculate the coding cost cost2[cIdx][x][y]. Then the new quantization coefficient is:

[0156]

[0157] The cost of encoding this quantized coefficient is:

[0158]

[0159] The specific process of setting the non-zero coefficients at the end of the 8x8 block to zero is as follows:

[0160] For 8x8 blocks, RDOQ will try to set the last positive and negative 1 coefficients in the reverse scanning order to zero to reduce the coding cost. Figure 7 The specific process is described. First, read the fine-tuned quantization coefficients in reverse scanning order. If |newQCoeff[cIdx][x][y]|>0, try to set all coefficients after the scanning order to zero and calculate the coding cost trim_0_cost[cIdx][x][y]. The specific calculation method is:

[0161]

[0162] Where i is the horizontal coordinate of the quantization coefficient in the 8x8 block, j is the vertical coordinate of the quantization coefficient in the 8x8 block, and the order of the summation operation is based on the inverse scanning order of the 8x8 block. zeroOutCost[cIdx][i][j] is to set the quantization coefficient here to zero and then calculate distortion[cIdx][i][j] according to Eq.5. Then, the coding cost orig_cost[cIdx][x][y] of the cutoff position (x, y) that has not been set to zero is calculated. The specific calculation method is:

[0163]

[0164] If trim_0_cost[cIdx][x][y]< , all coefficients after QCoeff[cIdx][x][y] are set to zero; otherwise, they are not set to zero.

[0165] In addition, if |QCoeff[cIdx][x][y]|>1, stop and proceed to the next step. Otherwise, continue to read the quantization coefficients and perform zeroing operations.

[0166] After all quantization coefficients of a TU are fine-tuned, RDOQ chooses from two quantization schemes: 1) zeroing the entire TU and 2) using the fine-tuned quantization coefficients newQCoeff[cIdx][x][y]. To achieve the optimal quantization result, RDOQ needs to calculate the coding cost of these two quantization schemes. For the first zeroing scheme, the coding cost of the TU is:

[0167]

[0168] Among them, for the second quantization scheme, the coding cost of TU is:

[0169]

[0170] If tuCost0 < tuCost1, the entire TU will be set to zero. Otherwise, each quantization coefficient of the TU will be newQCeoff[cIdx][x][y].

[0171] Figure 8 is a block diagram of a rate-distortion optimized quantization device based on H264 shown according to an exemplary embodiment. The rate-distortion optimized quantization device is applied to an encoder that conforms to the H264 / AVC standard. In the rate-distortion optimized quantization process, a general variable quantization algorithm is used for both bitrate estimation and distortion estimation. As Figure 8 shown, in this embodiment, the rate-distortion optimized quantization device includes:

[0172] A receiving module 81 for receiving a video frame to be processed.

[0173] A transform processing module 82 for partitioning and transform processing the video frame to be processed to obtain a plurality of first transform blocks.

[0174] A quantization processing module 83 for performing parallel quantization processing on the first transform coefficients in the first transform blocks by using a rate-distortion optimized quantization algorithm to obtain pre-quantization coefficients.

[0175] A coefficient processing module 84 for fine-tuning the pre-quantization coefficients by using a preset fine-tuning method; and using the fine-tuned pre-quantization coefficients and a zeroing rule corresponding to the transform block specification of the first transform block to zero the corresponding coefficients in the first transform block to obtain second transform coefficients after rate-distortion optimization quantization.

[0176] The preset fine-tuning method includes: Fine-tuning method one zeros non-zero pre-quantization coefficients;

[0177] Fine-tuning method two keeps the sign of non-zero pre-quantization coefficients unchanged but reduces the absolute value by one;

[0178] Fine-tuning method three keeps the pre-quantization coefficients unchanged and selects the fine-tuning method with the minimum coding cost.

[0179] The quantization processing module 83 is used to initialize the context model to a predefined value and then define quantization offset parameters and quantization scaling parameters;

[0180] Calculate pre-quantization coefficients based on the quantization offset parameters and the quantization scaling parameters; the calculation formula is as follows:

[0181]

[0182] Among them, TCoeff[cIdx][x][y] is the transform coefficient, cIdx is the Y, U, and V component indexes, x and y are the horizontal and vertical coordinates of the transform coefficient in the first transform block, sign[cIdx][x][y] is the sign of the transform coefficient, and QCoeff[cIdx][x][y] is the prequantization result.

[0183] Optionally, the first transform block includes: a 4×4 transform block and an 8×8 transform block.

[0184] The coefficient processing module 84 is used to calculate the quantization offset parameter in the following manner:

[0185] If the first transform block is a 4×4 transform block, the calculation formula of the quantization offset parameter is:

[0186]

[0187] If the first transform block is an 8×8 transform block, the calculation formula of the quantization offset parameter is:

[0188]

[0189] The coefficient processing module 84 is used to calculate the quantization scaling parameter in the following manner:

[0190] If the first transform block is a 4×4 transform block, the calculation formula of the quantization scaling parameter is:

[0191]

[0192] If the first transform block is an 8×8 transform block, the calculation formula of the quantization scaling parameter is:

[0193]

[0194] Among them, scalingList4x4 can be determined by querying the 4x4 quantization matrix table, and scalingList8x8 can be determined by querying the 8x8 quantization matrix table.

[0195] a coefficient processing module 84 for calculating first coding costs corresponding to non-zero transform coefficients and zero-set transform coefficients respectively using the prequantized coefficients, and determining a transform coefficient processing method with a smaller coding cost according to the calculation results of the first coding costs;

[0196] After fine-tuning the non-zero transform coefficient, calculating a second coding cost corresponding to the fine-tuned non-zero transform coefficient;

[0197] A zeroing rule is determined according to the second coding cost, and corresponding transform coefficients in the first transform block are set to zero to obtain second transform coefficients after rate-distortion optimized quantization.

[0198] a coefficient processing module 84 for fine-tuning the non-zero prequantized coefficients according to three preset fine-tuning methods, calculating first coding costs of the three fine-tuning methods, and selecting the fine-tuning method with the lowest coding cost based on the first coding costs;

[0199] The coding cost of the fine-tuned quantized coefficient is the second coding cost;

[0200] If it is an 8x8 block, according to the principle of minimizing coding cost, the non-zero quantized coefficients at the end of the scanning order are set to zero;

[0201] Also according to the principle of minimizing coding cost, all transform coefficients in the first transform block are set to zero to obtain second transform coefficients after rate-distortion optimized quantization.

[0202] The coefficient processing module 84 is used to set several non-zero quantized coefficients at the end of the scanning order to zero if it is an 8x8 transform block. For any position in the 8x8 transform block, if the coding cost of setting all the non-zero coefficients after the scanning order to zero is less than the coding cost of not setting them to zero, then all the non-zero coefficients after that are set to zero.

[0203] The coefficient processing module 84 is configured to set the entire transform block to zero if the coding cost of setting the entire transform block to zero is less than the coding cost of not setting the entire transform block to zero.

[0204] The coefficient processing module 84 is configured to calculate, after obtaining the prequantization coefficient QCoeff[cIdx][x][y], a coding cost after returning the non-zero prequantization coefficient to zero if the prequantization coefficient is non-zero;

[0205] Calculate the coding cost of pre-quantized coefficients not returning to zero;

[0206] According to the encoding costs of the three fine-tuning schemes, the fine-tuning scheme with the smallest encoding cost is selected.

[0207] If it is an 8x8 TU, the non-zero coefficients at the end of the reverse scan order are set to zero according to the principle of minimizing the coding cost.

[0208] After all prequantized coefficients in the first transform block are processed, the coding cost of retaining the non-zero coefficients is compared with the coding cost of setting the entire first transform block to zero, and then the processing method with the smaller coding cost is selected.

[0209] The coefficient processing module 84 is configured to, after obtaining the prequantization coefficient QCoeff[cIdx][x][y], further include:

[0210] Calculate quantization error and encoding bit rate;

[0211] The quantization error is calculated as follows:

[0212]

[0213] If the first transform block is a 4×4 transform block, the quantization error restoration process is as follows:

[0214]

[0215] If the first transform block is an 8×8 transform block, the quantization error restoration process is as follows:

[0216]

[0217] Among them, distortionScaler4x4 is determined based on the 4x4 distortion weight factor table, and distortionScaler8x8 is determined based on the 8x8 distortion weight factor table;

[0218] The encoding bit rate is calculated as follows:

[0219]

[0220] Among them, sigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is zero; lastSigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is the last non-zero quantization coefficient in the first transform block; coeffAbsLevelRate[cIdx][x][y] is the rate estimate of the syntax element that encodes the absolute value of the non-zero quantization coefficient.

[0221] The coefficient processing module 84 is used to calculate the sigCoeffFlagRate[cIdx][x][y] as follows:

[0222]

[0223] Where sigCoeffFlagTabIdx is determined by x, y and the type of the current coding block, and rateEstTab corresponds to Table 5.

[0224] The coefficient processing module 84 is used to calculate the lastSigCoeffFlagRate[cIdx][x][y] as follows:

[0225]

[0226] Where lastSigCoeffFlagTabIdx is determined by x, y and the type of the current coding block.

[0227] The coefficient processing module 84 is used to calculate the coeffAbsLevelRate[cIdx][x][y] as follows:

[0228]

[0229] in,

[0230]

[0231] greaterThanOneFlagTabIdx is always 1,

[0232]

[0233] Where GolombCodeLength is the code length after the absolute value of QCoeff[cIdx][x][y] is encoded with Golomb-Rice code, and eqProbRateEstTab is obtained by querying Table 6.

[0234] The coefficient processing module 84 is used to calculate the fine-tuning non-zero transform coefficients as follows:

[0235]

[0236] The second encoding cost is calculated as follows:

[0237] .

[0238] An embodiment of the present application also proposes an electronic device, including a processor and a memory; the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the H264-based rate-distortion optimization quantization method of any of the above embodiments.

[0239] The embodiments of the present application further provide a computer program product. When the executable computer program in the computer program product is executed by a processor, the rate-distortion optimized quantization method based on H264 of any of the above embodiments can be implemented.

[0240] Regarding the apparatus in the above embodiment, the specific manner in which the processor performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0241] Figure 9 1 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 1200 can be provided as a server. Figure 9The device 1200 includes a processing component 1222, which further includes one or more processors and a memory resource represented by a memory 1232 for storing instructions, such as an application, that can be executed by the processing component 1222. The application stored in the memory 1232 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1222 is configured to execute the instructions to perform the above-mentioned rate-distortion optimized quantization method based on H.264.

[0242] The device 1200 may also include a power supply component 1226 configured to perform power management of the device 1200, a wired or wireless network interface 1250 configured to connect the device 1200 to a network, and an input / output (I / O) interface 1258. The device 1200 may operate based on an operating system stored in the memory 1232, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.

[0243] In an exemplary embodiment, a computer program product is also provided. The computer program product may include a memory 1232. Instructions stored in the memory 1232 may be executed by the processing component 1222 of the device 1200 to perform the above method. For example, the storage medium computer program product may be software or a memory containing the software, such as a fixed memory ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0244] In the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The term "plurality" refers to two or more, unless otherwise clearly defined.

[0245] The above description of the embodiments is intended to facilitate understanding and application of the present application by those skilled in the art. It will be apparent that those skilled in the art can readily make various modifications to these embodiments and apply the general principles described herein to other embodiments without expending any creative effort. Therefore, the present application is not limited to the embodiments described herein, and improvements and modifications made by those skilled in the art based on the disclosure of this application without departing from the scope and spirit of this application are within the scope of this application.

Claims

1. A rate-distortion optimization quantization method based on H264, characterized in that: Applied to an encoder that implements the H264 / AVC standard, the method comprises: Receive video frames to be processed; Dividing and transforming the video frame to be processed to obtain a plurality of first transformation blocks; Performing parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain prequantization coefficients; specifically comprising: initializing a context model to a predefined value, defining a quantization offset parameter and a quantization scaling parameter; and calculating the prequantization coefficients based on the quantization offset parameter and the quantization scaling parameter; Fine-tuning the pre-quantization coefficient using a preset fine-tuning method; Using the fine-tuned prequantization coefficients and a zeroing rule corresponding to the transform block size of the first transform block, corresponding coefficients in the first transform block are set to zero to obtain second transform coefficients after rate-distortion optimized quantization; The first transform block is: a 4×4 transform block or an 8×8 transform block; The zeroing rule includes: for a 4x4 transform block, the entire transform block is considered to be zeroed according to the rate-distortion cost; for an 8x8 transform block, the fine-tuned prequantization coefficients are first read in sequence according to the reverse scanning order, and if the absolute value is greater than zero and less than or equal to 1, Calculate the coding cost of setting all coefficients after the scanning order to zero, and the coding cost without setting to zero; If the encoding cost after zeroing is less than the encoding cost without zeroing, All coefficients after the scan sequence are set to zero, otherwise, they are not set to zero; until a pre-quantized coefficient with an absolute value greater than 1 is read, stop; Then, the entire transform block is considered to be set to zero according to the rate-distortion cost.

2. The method according to claim 1, characterized in that The preset fine-tuning method includes: Fine-tuning method 1 sets the non-zero pre-quantization coefficients to zero; Fine-tuning method 2 keeps the sign of non-zero pre-quantization coefficient unchanged but reduces the absolute value by one; Fine-tuning method three keeps the pre-quantization coefficient unchanged and selects the fine-tuning method with the lowest coding cost.

3. The method according to claim 1, characterized in that The method of performing parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients includes: After initializing the context model to predefined values, define the quantization offset parameter and the quantization scaling parameter; The pre-quantization coefficient is calculated based on the quantization offset parameter and the quantization scaling parameter; the calculation formula is as follows: ； Where TCoeff[cIdx][x][y] is the transform coefficient, cIdx is the Y, U, and V component index, x and y are the horizontal and vertical coordinates of the transform coefficient in the first transform block, sign[cIdx][x][y] is the sign of the transform coefficient, and QCoeff[cIdx][x][y] is the prequantization result; is the quantization scaling parameter, is the quantization offset parameter.

4. The method according to claim 3, characterized in that The calculation method of the quantization offset parameter includes: If the first transform block is a 4×4 transform block, the calculation formula of the quantization offset parameter is: ； If the first transform block is an 8×8 transform block, the calculation formula of the quantization offset parameter is: ； Where QP is the quantization parameter and bitDepth is the bit width of the encoding.

5. The method according to claim 4, characterized in that The calculation method of the quantization scaling parameter includes: If the first transform block is a 4×4 transform block, the calculation formula of the quantization scaling parameter is: ； If the first transform block is an 8×8 transform block, the calculation formula of the quantization scaling parameter is: ； in, is the quantization scaling parameter, x and y are the horizontal and vertical coordinates of the change coefficient in the transform block, and QP is the quantization parameter; Among them, scalingList4x4 can be obtained by querying the 4X4 quantization matrix table, and scalingList8x8 can be obtained by querying the 8X8 quantization matrix table.

6. The method according to claim 5, characterized in that Fine-tuning the non-zero pre-quantization coefficients according to three preset fine-tuning methods, calculating first coding costs of the three fine-tuning methods, and selecting the fine-tuning method with the smallest coding cost according to the first coding costs; The coding cost of the fine-tuned quantization coefficient is the second coding cost.

7. The method according to claim 6, characterized in that After obtaining the pre-quantization coefficient QCoeff[cIdx][x][y], the method further includes: Calculate quantization error and encoding bit rate; The quantization error is calculated as follows: ； If the first transform block is a 4×4 transform block, the quantization error restoration process is as follows: ； If the first transform block is an 8×8 transform block, the quantization error restoration process is as follows: ； The distortionScaler4x4 is determined based on the 4x4 distortion weight factor table, and the distortionScaler8x8 is determined based on the 8x8 distortion weight factor table; The encoding bit rate is calculated as follows: ； Among them, sigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is zero; lastSigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is the last non-zero quantization coefficient in the first transform block; coeffAbsLevelRate[cIdx][x][y] is the rate estimate of the syntax element that encodes the absolute value of the non-zero quantization coefficient.

8. The method according to claim 7, characterized in that The sigCoeffFlagRate[cIdx][x][y] is calculated as follows: ； Where sigCoeffFlagTabIdx is determined by x, y and the type of the current coding block, and rateEstTab is the rate estimation table.

9. The method according to claim 8, characterized in that The lastSigCoeffFlagRate[cIdx][x][y] is calculated as follows: ； Where lastSigCoeffFlagTabIdx is determined by x, y and the type of the current coding block.

10. The method according to claim 7, characterized in that The coeffAbsLevelRate[cIdx][x][y] is calculated as follows: ； in, ； greaterThanOneFlagTabIdx is always 1, ； Where GolombCodeLength is the code length after the absolute value of QCoeff[cIdx][x][y] is encoded with a Golomb-Rice code, and eqProbRateEstTab is determined by the equal probability rate estimation table.

11. The method according to claim 6, characterized in that The fine-tuning of the non-zero transform coefficient and calculating a second coding cost corresponding to the fine-tuned non-zero transform coefficient includes: The fine-tuning non-zero transform coefficients are calculated as follows: ； The second encoding cost is calculated as follows: 。 12. A rate-distortion optimization quantization device based on H264, characterized in that: Applied to an encoder, the encoder implements the H264 / AVC standard, the device comprising: A receiving module, configured to receive a video frame to be processed; a transform processing module, configured to divide and transform the video frame to be processed to obtain a plurality of first transform blocks; a quantization processing module, configured to perform parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain prequantization coefficients; the module specifically comprising: initializing a context model to a predefined value, defining a quantization offset parameter and a quantization scaling parameter; and calculating the prequantization coefficients based on the quantization offset parameter and the quantization scaling parameter; a coefficient processing module, configured to fine-tune the prequantization coefficients using a preset fine-tuning method; and to set corresponding coefficients in the first transform block to zero using the fine-tuned prequantization coefficients and a zeroing rule corresponding to a transform block specification of the first transform block, thereby obtaining second transform coefficients after rate-distortion optimized quantization; The first transform block is: a 4×4 transform block or an 8×8 transform block; The zeroing rule includes: for a 4x4 transform block, the entire transform block is considered to be zeroed according to the rate-distortion cost; for an 8x8 transform block, the fine-tuned prequantization coefficients are first read in sequence according to the reverse scanning order, and if the absolute value is greater than zero and less than or equal to 1, Calculate the coding cost of setting all coefficients after the scanning order to zero, and the coding cost without setting to zero; If the encoding cost after zeroing is less than the encoding cost without zeroing, All coefficients after the scan sequence are set to zero, otherwise, they are not set to zero; until a pre-quantized coefficient with an absolute value greater than 1 is read, stop; Then, the entire transform block is considered to be set to zero according to the rate-distortion cost.

13. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method according to any one of claims 1 to 11.

14. A computer program product having a computer program stored thereon, characterized in that: When the executable computer program in the computer program product is executed by a processor, the method according to any one of claims 1 to 11 can be implemented.

Images