H264-based rate distortion optimization quantization method, device, equipment and product

Abstract

The invention relates to an H264-based rate distortion optimization quantization method, device, equipment and product. The method is applied to an encoder, the encoder executes the H264 / AVC standard, and the method comprises the following steps: receiving a video frame to be processed; performing division and transformation processing on the to-be-processed video frame to obtain a plurality of first transformation blocks; performing parallel quantization processing on the first transformation coefficient in the first transformation block by using a distortion optimization quantization algorithm to obtain a pre-quantization coefficient; and if the block is a 8 * 8 transform block, a plurality of non-zero coefficients at the end of the scanning sequence are zeroed, and whether the non-zero coefficients are zeroed depends on whether the coding cost can be minimized. Through the above steps, a second transformation coefficient after rate distortion optimization quantization is obtained. According to the method, the quantization coefficient can be processed in parallel, the processing speed of hardware is increased, the processing delay of the hardware is reduced, the method can adapt to the H264 / AVC standard at the same time, hardware resources are shared as much as possible, and the area of a rate distortion optimization quantization module is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of video compression, and in particular to a rate-distortion optimization quantization method, device, equipment and product based on H264. Background Art

[0002] In related technologies, scalar quantization cannot achieve optimal performance because of its fixed dead zone. Rate-distortion optimized quantization (RDOQ) can maximize compression performance because it takes rate-distortion cost into account during the quantization process. RDOQ technology compares the rate-distortion cost from multiple quantization schemes, and finally selects the one with the smallest rate-distortion cost to achieve optimal performance. The quantization scheme can be determined by different quantization parameters (QP) or by different quantization offsets. Among them, RDOQ technology originated from the H264 / AVC video coding standard.

[0003] In the related art, an early RDOQ method was proposed, which selects the optimal QP for each coding block to achieve the optimization purpose. In the related art, a further improved method is also proposed, which removes some unnecessary candidate QPs through some fast algorithms to reduce the number of traversals. However, these methods are not suitable for applications with rate control turned on, because rate control will further change the QP of each block to achieve the effect of balancing the bit rate. Among them, the key to RDOQ is to calculate the rate-distortion cost of each quantization scheme, which can be defined as:

[0004] Among them, C is the rate-distortion cost, D is the distortion between the original signal and the decoded signal, R is the encoding bit rate, and λ is the rate-distortion parameter. D can be obtained by calculating the difference between the original signal and the decoded signal. However, due to the sequential execution characteristics of the entropy encoder and the high complexity of the context model, the accurate bit rate R is usually difficult to obtain. A method of directly inputting the quantization coefficient into the entropy encoder to obtain the bit rate R is proposed in the related art, but it still requires a lot of calculations of the entropy encoder, so it is not suitable for low-latency scenarios.

[0005] In order to reduce RDOQ's dependence on the entropy encoder, RDOQ uses many techniques to quickly estimate the distortion D and the bit rate R. First, skip the inverse transform step and directly calculate the distortion D. Then, the bit rate R is estimated by querying a large number of trained entropy coding tables without relying on the entropy encoder. In the related art, a simple and effective RDOQ method applied to H265 / HEVC is proposed. This method attempts to change the spatial distribution of non-zero coefficients and reduce the absolute value of non-zero coefficients to minimize the rate-distortion cost. For the quantized residual coefficients, the position of the non-zero coefficients and the absolute value of the non-zero coefficients occupy most of the encoding bit rate. RDOQ attempts to fine-tune these positions and absolute values ​​so that entropy coding can compress information with higher efficiency. In general, RDOQ can reduce the encoding bit rate R, but it will increase the encoding distortion D, but it can achieve the purpose of improving the final encoding quality by minimizing the rate-distortion cost C.

[0006] In the related art, a fast algorithm is further proposed. Whether RDOQ is needed is determined by checking the product of the quantization coefficient and QP. If the product is less than a certain threshold, RDOQ is activated. Otherwise, RDOQ is turned off.

[0007] However, in the related art, the RDOQ method requires that the quantization coefficients arrive in a predefined order and process them in order, which is feasible for software implementation but not optimal for hardware implementation. Specifically, when the RDOQ module processes a specific quantization coefficient, the processing of the previous quantization coefficient needs to be completed. This dependency makes it difficult to process multiple quantization coefficients in parallel, thus failing to give full play to the parallel capabilities of the hardware.

[0008] The current RDOQ method of H264 / AVC encoder generally optimizes the coding cost by adjusting QP, but this method cannot be used together with rate control because rate control will further change QP. In addition, some RDOQ schemes use adaptive methods to generate quantization offset parameters to achieve the purpose of optimizing coding cost, and further propose to use a lookup table to estimate the coding rate of quantization coefficients. However, this scheme can only be applied to H265 / HEVC standard encoders and is not applicable to H264 / AVC. Summary of the invention

[0009] The purpose of this application is to provide a rate-distortion optimized quantization method, device, equipment and product based on H264, which can process quantization coefficients in parallel, reduce hardware processing delay, and adapt to H264 / AVC at the same time, share hardware resources as much as possible, and reduce the area of ​​the rate-distortion optimized quantization module.

[0010] According to a first aspect of an embodiment of the present application, a rate-distortion optimization quantization method based on H264 is provided, which is applied to an encoder, and the encoder implements the H264 / AVC standard; the method includes: Receive video frames to be processed; Dividing and transforming the video frame to be processed to obtain a plurality of first transformation blocks; Performing parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients; The prequantization coefficients and the zeroing rule corresponding to the transform block size of the first transform block are used to set the corresponding coefficients in the first transform block to zero, so as to obtain the second transform coefficients after rate-distortion optimized quantization.

[0011] In one implementation, the utilizing rate-distortion optimization quantization algorithm to perform parallel quantization processing on the first transform coefficients in the first transform block to obtain pre-quantization coefficients includes: After initializing the context model to a predefined value, a quantization offset parameter and a quantization scaling parameter are defined; The pre-quantization coefficient is calculated based on the quantization offset parameter and the quantization scaling parameter; the calculation formula is as follows: ; Among them, TCoeff[cIdx][x][y] is the transform coefficient, cIdx is the Y, U, and V component indexes, x and y are the horizontal and vertical coordinates of the transform coefficient in the transform block, sign[cIdx][x][y] is the sign of the transform coefficient, and QCoeff[cIdx][x][y] is the pre-quantization result.

[0012] In one implementation, the first transform block includes: a 4×4 transform block and an 8×8 transform block.

[0013] In one implementation, the calculation method of the quantization offset parameter includes: If the first transform block is a 4×4 transform block, the calculation formula of the quantization offset parameter is: ; If the first transform block is an 8×8 transform block, the calculation formula of the quantization offset parameter is: ; In one implementation, the calculation method of the quantization scaling parameter includes: If the first transform block is a 4×4 transform block, the calculation formula of the quantization scaling parameter is: ; If the first transform block is an 8×8 transform block, the calculation formula of the quantization scaling parameter is: ; Among them, scalingList4x4 can be obtained by querying the 4x4 quantization matrix table (that is, Table 1), and scalingList8x8 can be obtained by querying the 8x8 quantization matrix table (that is, Table 2).

[0014] In one embodiment, a non-zero pre-quantization coefficient is fine-tuned, and then the fine-tuned quantization coefficient is set to zero to obtain a second transform coefficient after rate-distortion optimized quantization, including: using the pre-quantization coefficient to calculate the first coding costs corresponding to three fine-tuning methods respectively, and determining a transform coefficient fine-tuning method with a small coding cost based on the calculation result of the first coding cost; after fine-tuning the non-zero transform coefficient, a second coding cost corresponding to the fine-tuned non-zero transform coefficient is calculated; according to the second coding cost, a zeroing rule is determined, and the corresponding transform coefficient in the first transform block is set to zero to obtain the second transform coefficient after rate-distortion optimized quantization.

[0015] In one implementation, the using the prequantization coefficients to calculate the first coding costs of the three fine-tuning methods, and determining the transform coefficient processing method with a small coding cost according to the calculation result of the first coding cost, includes: After obtaining the prequantization coefficient QCoeff[cIdx][x][y], if the prequantization coefficient is non-zero, the coding cost after returning the non-zero prequantization coefficient to zero is calculated, which is the first fine-tuning method; Calculate the coding cost of keeping the sign of the pre-quantization coefficient unchanged and reducing the absolute value by one, which is the second fine-tuning method; Calculate the coding cost of keeping the pre-quantization coefficient unchanged, which is the fine-tuning method three; Compare the encoding costs of the above three fine-tuning methods, and then select the processing method with the smallest encoding cost.

[0016] In one implementation, after obtaining the prequantization coefficient QCoeff[cIdx][x][y], the method further includes: Calculate quantization error and encoding bit rate; The quantization error is calculated as follows: ; If the first transform block is a 4×4 transform block, the quantization error restoration processing method is as follows: ; If the first transform block is an 8×8 transform block, the quantization error restoration processing method is as follows: ; Among them, distortionScaler4x4 is determined based on the 4x4 distortion weight factor table, and distortionScaler8x8 is determined based on the 8x8 distortion weight factor table; The encoding bit rate is calculated as follows: ; Among them, sigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is zero; lastSigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is the last non-zero quantization coefficient in the first transform block; coeffAbsLevelRate[cIdx][x][y] is the rate estimate of the syntax element that encodes the absolute value of the non-zero quantization coefficient.

[0017] In one implementation, the sigCoeffFlagRate[cIdx][x][y] is calculated as follows: ; Where sigCoeffFlagTabIdx is determined by x, y and the type of the current coding block, and rateEstTab is the rate estimation table.

[0018] In one implementation, the lastSigCoeffFlagRate[cIdx][x][y] is calculated as follows: ; Where lastSigCoeffFlagTabIdx is determined by x, y and the type of the current coding block.

[0019] In one implementation, the coeffAbsLevelRate[cIdx][x][y] is calculated as follows: ; in, ; greaterThanOneFlagTabIdx is always 1, ; Where GolombCodeLength is the code length after the absolute value of QCoeff[cIdx][x][y] is encoded with Golomb-Rice code, and eqProbRateEstTab is the equal probability rate estimation table.

[0020] In one implementation, after fine-tuning the non-zero transform coefficient, calculating a second coding cost corresponding to the fine-tuned non-zero transform coefficient includes: The fine-tuning non-zero transform coefficients are calculated as follows: ; Among them, cost0[cIdx][x][y] is the coding cost of setting the non-zero prequantization coefficient to zero, cost1[cIdx][x][y] is the coding cost of keeping the sign of the non-zero prequantization coefficient unchanged but reducing the absolute value by one, and cost2[cIdx][x][y] is the coding cost of keeping the non-zero prequantization coefficient unchanged.

[0021] The second encoding cost is calculated as follows: .

[0022] According to a second aspect of an embodiment of the present application, a rate-distortion optimization quantization device based on H264 is provided, which is applied to an encoder, and is applied to an encoder, wherein the encoder implements the H264 / AVC standard, and the device includes: A receiving module, used for receiving a video frame to be processed; A transformation processing module, used for dividing and transforming the video frame to be processed to obtain a plurality of first transformation blocks; A quantization processing module, configured to perform parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients; The coefficient processing module is used to use the pre-quantization coefficient and the fine-tuning method and zeroing rule corresponding to the transform block specification of the first transform block to fine-tune and zero the corresponding coefficients in the first transform block to obtain the second transform coefficient after rate-distortion optimized quantization.

[0023] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the above method.

[0024] According to a fourth aspect of an embodiment of the present application, a computer program product is provided, on which a computer program is stored, characterized in that when the executable computer program in the computer program product is executed by a processor, the above method can be implemented.

[0025] Compared with the prior art, the beneficial effects of the present application are: the encoder executes the H264 / AVC standard and receives a video frame to be processed; the video frame to be processed is divided and transformed to obtain multiple first transform blocks; the first transform coefficient in the first transform block is parallel quantized using a rate-distortion optimization quantization algorithm to obtain a prequantization coefficient; the prequantization coefficient is fine-tuned by selecting a fine-tuning method with the minimum coding cost. The corresponding coefficient in the first transform block is set to zero using the fine-tuned prequantization coefficient and the zeroing rule corresponding to the transform block specification of the first transform block to obtain the second transform coefficient after rate-distortion optimization quantization. Specifically, the transform coefficient in the transform block is first quantized with a fixed dead zone, and then each quantization coefficient is fine-tuned. After quantization and fine-tuning are completed, for a 4x4 transform block, the entire transform block is considered to be set to zero according to the rate-distortion cost. For an 8x8 transform block, the last non-zero quantization coefficient is considered to be set to zero according to the reverse scanning order, and then the entire transform block is considered to be set to zero according to the rate-distortion cost. In order to minimize the consumption of hardware resources, the present invention does not require any floating-point operations, only integer addition, subtraction and shift calculations of up to 32 bits are required. Moreover, the calculation of the originally complex coding cost is realized by using a simple table lookup. In summary, the technical solution provided by the present application can process quantization coefficients in parallel, improve the processing speed of hardware, reduce the processing delay of hardware, and can adapt to the H264 / AVC standard at the same time, share hardware resources as much as possible, and reduce the area of ​​the rate-distortion optimization quantization module. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a block diagram of an encoder according to an exemplary embodiment.

[0027] Figure 2 The figure is a flow chart showing a rate-distortion optimized quantization method based on H264 according to an exemplary embodiment.

[0028] Figure 3 is a flowchart of the RDOQ of the TU layer according to another exemplary embodiment.

[0029] Figure 4 is a schematic diagram of a scanning order of a 4x4 TU according to an exemplary embodiment.

[0030] Figure 5 is a schematic diagram of a scanning order of an 8x8 TU according to an exemplary embodiment.

[0031] Figure 6 The figure is a flowchart of coefficient fine-tuning according to an exemplary embodiment.

[0032] Figure 7 The flowchart of setting non-zero coefficients of an 8x8 TU at the end of a scanning order to zero is shown according to an exemplary embodiment.

[0033] Figure 8 It is a block diagram of a rate-distortion optimized quantization device based on H264 according to an exemplary embodiment.

[0034] Fig. 9 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0035] Unless otherwise defined, the technical terms or scientific terms used in this specification and claims shall have the usual meanings understood by persons with ordinary skills in the technical field to which the invention belongs. The specific embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be noted that in the specific description of these embodiments, in order to provide a concise description, it is impossible for this specification to provide a detailed description of all the features of the actual embodiments. Without departing from the spirit and scope of the present invention, those skilled in the art may modify and replace the embodiments of the present invention, and the resulting embodiments are also within the scope of protection of the present invention.

[0036] Before introducing the rate-distortion optimization quantization method provided by this application, the encoder and the position of the rate-distortion optimization quantization module in the encoder are first introduced. Figure 1 As shown, the encoder includes a transform module 11, a rate-distortion optimization quantization module 12 and an entropy coding module 13. The rate-distortion optimization quantization module 12 is located between the transform module 11 and the entropy coding module 13.

[0037] The encoder in this application is a video encoder. When a new video frame enters the encoder, the encoder usually divides the video frame into a series of non-overlapping square or rectangular blocks. The "block" here actually refers to a matrix. Given a target block, the encoder will try to find the best prediction block for it, and then subtract the prediction block from the target block to obtain a residual block.

[0038] The transform block is configured to perform a frequency domain transform on the residual block (generally a discrete cosine transform, but not limited to this). The output of the transform block is a transform coefficient (a transform coefficient matrix). The video compression standard is H264 / AVC, and the transform block is called a transform unit (TU).

[0039] The rate-distortion optimization quantization module RDOQ is used to perform rate-distortion optimization quantization on the transform coefficients output by the transform block, and output the quantization coefficient matrix (hereinafter referred to as the second quantization coefficient matrix) to the entropy coding module CABAC. The rate-distortion optimization quantization module is used to complete the quantization processing of the transform coefficients, fine-tune the non-zero coefficients, and selectively set them to zero according to conditions.

[0040] The entropy coding module is configured to encode the quantization coefficient matrix output by the rate-distortion optimization quantization module, for example, CABAC (Context-Based Adaptive Binary Arithmetic Coding) coding method can be used for encoding, but is not limited thereto.

[0041] Receive a video frame to be processed; divide and transform the video frame to be processed to obtain multiple first transform blocks; use a rate-distortion optimization quantization algorithm to perform parallel quantization processing on the first transform coefficients in the first transform block to obtain prequantization coefficients; then fine-tune the prequantization coefficients, fine-tuning method one sets the non-zero prequantization coefficients to zero, fine-tuning method two keeps the sign of the non-zero prequantization coefficients unchanged but reduces the absolute value by one, and fine-tuning method three keeps the prequantization coefficients unchanged, and then selects the fine-tuning method with the smallest coding cost. If it is an 8x8 transform block, then set several non-zero coefficients at the end of the scanning order to zero, and whether the non-zero coefficients are set to zero depends on whether the coding cost can be minimized. Finally, if the coding cost of setting all quantization coefficients of the entire first transform block to zero is less than the coding cost of not setting them to zero, then set all quantization coefficients of the entire first transform block to zero, otherwise, keep the quantization coefficients unchanged. After the above steps, the second transform coefficients after rate-distortion optimization quantization are obtained. The method can process quantization coefficients in parallel, improve the processing speed of hardware, reduce the processing delay of hardware, and can adapt to the H264 / AVC standard at the same time, share hardware resources as much as possible, and reduce the area of ​​the rate-distortion optimization quantization module.

[0042] The above briefly introduces the encoder and the position of the rate-distortion optimization quantization module in the encoder. The following is a detailed introduction to the rate-distortion optimization quantization module and its functions.

[0043] Figure 2 This is a flow chart of a rate-distortion optimization quantization method based on H264 according to an exemplary embodiment. The rate-distortion optimization quantization method can be applied to an encoder. The encoder is adapted to the H264 / AVC standard. Figure 2 The rate-distortion optimization quantization method based on H264 may include the following steps: Step 201: receiving a video frame to be processed.

[0044] Step 202: divide and transform the video frame to be processed to obtain a plurality of first transformation blocks.

[0045] Step 203: Perform parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients.

[0046] Step 204: fine-tune the prequantization coefficient using a preset fine-tuning method.

[0047] Step 205 , using the fine-tuned prequantization coefficients and the zeroing rule corresponding to the transform block size of the first transform block, the corresponding coefficients in the first transform block are set to zero to obtain the second transform coefficients after rate-distortion optimized quantization.

[0048] First, the transform coefficients in the transform block are quantized with a fixed dead zone, and then each quantization coefficient is fine-tuned. After quantization and fine-tuning are completed, for the 4x4 transform block, the entire transform block is considered to be set to zero according to the rate-distortion cost. For the 8x8 transform block, the last non-zero quantization coefficient is considered to be set to zero according to the inverse scanning order, and then the entire transform block is considered to be set to zero according to the rate-distortion cost. In order to minimize the consumption of hardware resources, the present invention does not require any floating-point operations, and only requires integer addition, subtraction and shift calculations of up to 32 bits. Moreover, this scheme uses a simple table lookup to achieve the calculation of the originally complex coding cost. It should be noted that the first transform block includes: at least one transform block type of 4×4 transform block and 8×8 transform block. If the first transform block is an 8×8 transform block, the last non-zero quantization coefficient is considered to be set to zero according to the inverse scanning order, and then the entire transform block is considered to be set to zero according to the rate-distortion cost. This method can effectively reduce the scattered non-zero coefficients at the end of the 8x8 block, so that the non-zero coefficients are concentrated in the upper left corner of the 8x8 block, thereby improving the compression performance of the entropy encoder. The H264 / AVC standard requires that the entropy encoder needs to follow a predefined scanning order to process the quantization coefficients of each TU, such as a zigzag order. Therefore, when calculating the rate-distortion cost of a certain quantization coefficient, its context model depends on the coding state of all forward quantization coefficients. Therefore, it is necessary to wait until all forward quantization coefficients are fully processed before starting, which greatly destroys the pipeline operation of the hardware. However, the present invention fixes the value of the context model, and does not need to follow the scanning order of the entropy coding quantization coefficients. Each quantization coefficient can be processed independently, so the pipeline can be greatly utilized to reduce hardware delay.

[0049] When a new frame enters the video encoder, the encoder usually divides the frame into a series of non-overlapping square or rectangular blocks. Given a target block, the encoder tries to find the best prediction block for it, and then subtracts the prediction block from the target block to obtain the residual block. The transform module then performs a frequency domain transform (usually a discrete cosine transform) on the residual block. The output of the transform module is a transform coefficient block. In the H264 / AVC video coding standard, there are two sizes of transform coefficient blocks: 4x4 and 8x8. In this patent, the transform block is also called TU.

[0050] like Figure 1 As shown in Figure 1, RDOQ is located after the transform module. RDOQ first quantizes the transform coefficients using a uniform quantization offset, then adjusts the non-zero coefficients to minimize the rate-distortion penalty, and further sets the entire TU to zero when necessary. Figure 3As shown in Figure 1, after the parameters are initialized, RDOQ starts to quantize each transform coefficient. There is no dependency between the transform coefficients, so they can be processed in parallel. For non-zero quantization coefficients, RDOQ adjusts their absolute values. The specific steps are as follows: Figure 6 As shown in Figure 2. When all quantization coefficients of a TU are processed, RDOQ can set the entire TU to zero as needed. In addition, for an 8x8 TU, RDOQ attempts to set the quantization coefficients at the end with an absolute value of 1 to zero to save bit rate.

[0051] Pre-quantize the transform coefficients of the TU layer. The workflow of RDOQ at the TU layer is as follows: Figure 3 As shown, first initialize the context model of the syntax element, the context model list is: ● significantCoeffCtx

[15] ● lastSignificantCoeffCtx

[15] ● greaterThanOneCoeffCtx[5] ● levelAbsCoeffCtx[5] Each element of these arrays is a 32-bit integer, these context models are initialized to predefined values, and the models are fixed throughout the entire process of RDOQ.

[0052] Then, RDOQ takes one transform coefficient from TU for pre-quantization each time. Let the quantization parameter be QP, x and y are the coordinates of the transform coefficient in TU, and quantization is the right shift operation of this transform coefficient. Among them, the parameter defining the quantization shift is named quantizationShift, which is generated as follows:

[0053] Defining the parameters of quantization scaling, named quantizationScaler, produces the following:

[0054] Among them, scalingList4x4 can be obtained by querying the 4x4 quantization matrix table (that is, Table 1), and scalingList8x8 can be obtained by querying the 8x8 quantization matrix table (that is, Table 2).

[0055] Table 1

[0056] Table 2

[0057] In RDOQ, pre-quantization is specifically defined as:

[0058] Where TCoeff[cIdx][x][y] is the transform coefficient, cIdx is the Y, U, V component index, sign[cIdx][x][y] is the sign of the transform coefficient, and QCoeff[cIdx][x][y] is the pre-quantization result.

[0059] The encoding cost calculation process is as follows: RDOQ optimizes the rate-distortion cost by reducing the absolute value of non-zero coefficients to reduce the bit rate. Given a quantized coefficient QCoeff[cIdx][x][y], if the quantized coefficient is non-zero, the coding cost of returning this coefficient to zero is first calculated. Then, the coding cost of not returning the coefficient to zero is calculated. When all quantized coefficients of a TU are processed, RDOQ compares the coding cost of retaining the non-zero coefficients and setting the entire TU to zero, and then selects the method with the smaller coding cost.

[0060] In order to obtain the coding cost, the quantization error should be calculated first:

[0061] Since the quantization error is amplified by the transformation, the reverse restoration of the quantization error is distortion:

[0062] Among them, distortionScaler4x4 is determined based on the 4x4 distortion weight factor table (Table 3), and distortionScaler8x8 is determined based on the 8x8 distortion weight factor table (Table 4).

[0063] Table 3

[0064] Table 4

[0065] To calculate the complete encoding cost, in addition to the distortion, we also need to calculate the encoding bitrate. The bitrate can be modeled as:

[0066] Where sigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is zero. The calculation method is:

[0067] Where sigCoeffFlagTabIdx is determined by x, y and the type of the current coding block. In addition, rateEstTab is a rate estimation table (Table 5rateEstTab). Since the bit rate can be estimated by simply looking up the table, there is no need to call a complex entropy encoder, thereby reducing complexity.

[0068] Table 5

[0069] lastSigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantized coefficient is the last non-zero quantized coefficient in the scanning order within the TU. The calculation method is:

[0070] Where lastSigCoeffFlagTabIdx is determined by x, y and the type of the current coding block. The H264 / AVC standard defines the scanning order of quantization coefficients within a TU. The scanning order of a 4x4 TU is as follows: Figure 4 As shown, the scanning order of 8x8 TU is as follows Figure 5 As shown. The entropy encoder will write the quantized coefficients into the bitstream according to this scanning order. Although the present invention does not need to follow this scanning order to process the quantized coefficients, the position of the last non-zero quantized coefficient is determined by this scanning order.

[0071] If QCoeff[cIdx][x][y] is non-zero, the absolute value of QCoeff[cIdx][x][y] must also be encoded. coeffAbsLevelRate[cIdx][x][y] is the rate estimate of the syntax element that encodes the absolute value of the non-zero quantized coefficient. It is calculated as follows:

[0072] in,

[0073] Among them, greaterThanOneFlagTabIdx is always 1.

[0074]

[0075] Where GolombCodeLength is the code length after the absolute value of QCoeff[cIdx][x][y] is encoded with Golomb-Rice code, and eqProbRateEstTab is determined by the equal probability rate estimation table (Table 6).

[0076] Table 6

[0077] In summary, given the quantization coefficient QCoeff[cIdx][x][y], the transform coefficient TCoeff[cIdx][x][y], the quantization parameter QP and bitDepth, the encoding cost is:

[0078] The encoding cost process after fine-tuning the coefficients is as follows: For non-zero quantization coefficients, RDOQ attempts to reduce their absolute values ​​by 1 or set them to zero to reduce the encoding bit rate. Figure 6 The details of coefficient fine-tuning are described in detail. RDOQ fine-tunes the absolute value of the quantized coefficient but keeps its sign bit unchanged. RDOQ will try to test three potential quantized coefficients, namely 0, sign[cIdx][x][y]x(|QCoeff[cIdx][x][y]|-1) and QCoeff[cIdx][x][y]. The candidate coefficient with the smallest coding cost will be selected as the new quantized coefficient. For the first candidate coefficient 0, apply Eq.12 to calculate the coding cost cost0[cIdx][x][y], where QCoeff[cIdx][x][y]=0. If |QCoeff[cIdx][x][y]|>1, the second candidate coefficient is sign[cIdx][x][y]x(|QCoeff[cIdx][x][y]|-1), and apply Eq.12 to calculate the coding cost cost1[cIdx][x][y]. The third candidate coefficient is the original QCoeff[cIdx][x][y], and Eq.12 is still used to calculate the coding cost cost2[cIdx][x][y]. Then the new quantization coefficient is:

[0079] The cost of encoding this quantized coefficient is:

[0080] The specific process of setting the non-zero coefficients at the end of the 8x8 block to zero is as follows: For 8x8 blocks, RDOQ will try to set the last positive and negative 1 coefficients in the reverse scanning order to zero to reduce the coding cost. Figure 7 The specific process is described. First, read the fine-tuned quantization coefficients in reverse scanning order. If |newQCoeff[cIdx][x][y]|>0, try to set all coefficients after the scanning order to zero and calculate the coding cost trim_0_cost[cIdx][x][y]. The specific calculation method is:

[0081] where i is the abscissa of the quantization coefficient in the 8x8 block, j is the ordinate of the quantization coefficient in the 8x8 block, and the order of the summation operation is according to the inverse scan order of the 8x8 block. zeroOutCost[cIdx][i][j] is to calculate distortion[cIdx][i][j] according to Eq.5 after setting the quantization coefficient here to zero. Then, calculate the encoding cost orig_cost[cIdx][x][y] of the cut-off position (x, y) without setting to zero. The specific calculation method is as follows:

[0082] If trim_0_cost[cIdx][x][y]<, then all the coefficients after QCoeff[cIdx][x][y] are set to zero, otherwise, they are not set to zero.

[0083] In addition, if |QCoeff[cIdx][x][y]|>1, stop and go to the next step, otherwise, continue to read the quantization coefficient and perform the zeroing operation.

[0084] When all the quantization coefficients of the TU are fine-tuned, RDOQ selects from two quantization schemes: 1) zero out the entire TU and 2) use the fine-tuned quantization coefficient newQCoeff[cIdx][x][y]. For the optimal quantization result, RDOQ needs to calculate the encoding costs of these two quantization schemes. For the first zeroing scheme, the encoding cost of the TU is:

[0085] where, for the second quantization scheme, the encoding cost of the TU is:

[0086] If tuCost0<tuCost1, the entire TU will be zeroed out. Otherwise, each quantization coefficient of the TU will be newQCeoff[cIdx][x][y].

[0087] Figure 8 is a block diagram of a rate-distortion optimized quantization device based on H264 shown according to an exemplary embodiment. The rate-distortion optimized quantization device is applied to an encoder that adapts to the H264 / AVC standard. During the rate-distortion optimized quantization process, for both the bitrate estimation and the distortion estimation, a general variable quantization algorithm is used. As Figure 8 shown, in this embodiment, the rate-distortion optimized quantization device includes: A receiving module 81 for receiving a video frame to be processed.

[0088] The transformation processing module 82 is used to divide and transform the video frame to be processed to obtain multiple first transformation blocks.

[0089] The quantization processing module 83 is used to perform parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients.

[0090] The coefficient processing module 84 is used to fine-tune the prequantization coefficient using a preset fine-tuning method; using the fine-tuned prequantization coefficient and the zeroing rule corresponding to the transform block specification of the first transform block, the corresponding coefficient in the first transform block is set to zero to obtain the second transform coefficient after rate-distortion optimized quantization.

[0091] The preset fine-tuning methods include: fine-tuning method 1 sets non-zero pre-quantization coefficients to zero; Fine-tuning method 2 keeps the sign of non-zero pre-quantization coefficient unchanged but reduces the absolute value by one; Fine-tuning method three keeps the pre-quantization coefficient unchanged and selects the fine-tuning method with the lowest coding cost.

[0092] A quantization processing module 83, configured to define a quantization offset parameter and a quantization scaling parameter after initializing the context model to a predefined value; The pre-quantization coefficient is calculated based on the quantization offset parameter and the quantization scaling parameter; the calculation formula is as follows:

[0093] Among them, TCoeff[cIdx][x][y] is the transform coefficient, cIdx is the Y, U, and V component index, x and y are the horizontal and vertical coordinates of the transform coefficient in the first transform block, sign[cIdx][x][y] is the sign of the transform coefficient, and QCoeff[cIdx][x][y] is the pre-quantization result.

[0094] Optionally, the first transform block includes: a 4×4 transform block and an 8×8 transform block.

[0095] The coefficient processing module 84 is used to calculate the quantization offset parameter, including: If the first transform block is a 4×4 transform block, the calculation formula of the quantization offset parameter is:

[0096] If the first transform block is an 8×8 transform block, the calculation formula of the quantization offset parameter is:

[0097] The coefficient processing module 84 is used to calculate the quantization scaling parameter, including: If the first transform block is a 4×4 transform block, the calculation formula of the quantization scaling parameter is:

[0098] If the first transform block is an 8×8 transform block, the calculation formula of the quantization scaling parameter is:

[0099] Among them, scalingList4x4 can be determined by querying the 4x4 quantization matrix table, and scalingList8x8 can be determined by querying the 8x8 quantization matrix table.

[0100] A coefficient processing module 84, configured to calculate first coding costs corresponding to non-zero transform coefficients and zero transform coefficients respectively using the prequantization coefficients, and determine a transform coefficient processing method with a small coding cost according to a result of calculating the first coding cost; After fine-tuning the non-zero transform coefficient, calculating a second encoding cost corresponding to the fine-tuned non-zero transform coefficient; A zeroing rule is determined according to the second coding cost, and corresponding transform coefficients in the first transform block are set to zero to obtain second transform coefficients after rate-distortion optimization quantization.

[0101] A coefficient processing module 84, configured to fine-tune the non-zero prequantized coefficients according to three preset fine-tuning methods, calculate first encoding costs of the three fine-tuning methods, and select a fine-tuning method with a minimum encoding cost according to the first encoding cost; The coding cost of the fine-tuned quantized coefficient is the second coding cost; If it is an 8x8 block, according to the principle of minimizing coding cost, the non-zero quantized coefficients at the end of the scanning order are set to zero; Also according to the principle of minimizing coding cost, all transform coefficients in the first transform block are set to zero to obtain the second transform coefficients after rate-distortion optimized quantization.

[0102] The coefficient processing module 84 is used to set several non-zero quantized coefficients at the end of the scanning order to zero if it is an 8x8 transform block. For any position of the 8x8 transform block, if the coding cost after setting all the non-zero coefficients after the scanning order to zero is less than the coding cost of not setting them to zero, then all the subsequent non-zero coefficients are set to zero.

[0103] The coefficient processing module 84 is configured to set the entire transform block to zero if the coding cost of setting the entire transform block to zero is less than the coding cost of not setting the entire transform block to zero.

[0104] A coefficient processing module 84 is used for, after obtaining the prequantization coefficient QCoeff[cIdx][x][y], if the prequantization coefficient is non-zero, calculating the coding cost after returning the non-zero prequantization coefficient to zero; Calculate the coding cost of pre-quantized coefficients not returning to zero; According to the encoding costs of the three fine-tuning schemes, the fine-tuning scheme with the smallest encoding cost is selected.

[0105] If it is an 8x8 TU, the non-zero coefficients at the end of the reverse scan order are set to zero according to the principle of minimizing the coding cost.

[0106] After all pre-quantized coefficients in the first transform block are processed, the coding cost of retaining the non-zero coefficients and the coding cost of setting the entire first transform block to zero are compared, and then the processing method with the smaller coding cost is selected.

[0107] The coefficient processing module 84 is used for obtaining the pre-quantization coefficient QCoeff[cIdx][x][y] and further comprising: Calculate quantization error and encoding bit rate; The quantization error is calculated as follows:

[0108] If the first transform block is a 4×4 transform block, the quantization error restoration processing method is as follows:

[0109] If the first transform block is an 8×8 transform block, the quantization error restoration processing method is as follows:

[0110] Among them, distortionScaler4x4 is determined based on the 4x4 distortion weight factor table, and distortionScaler8x8 is determined based on the 8x8 distortion weight factor table; The encoding bit rate is calculated as follows:

[0111] Among them, sigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is zero; lastSigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is the last non-zero quantization coefficient in the first transform block; coeffAbsLevelRate[cIdx][x][y] is the rate estimate of the syntax element that encodes the absolute value of the non-zero quantization coefficient.

[0112] The coefficient processing module 84 is used to calculate the sigCoeffFlagRate[cIdx][x][y] as follows:

[0113] Among them, sigCoeffFlagTabIdx is determined by x, y and the type of the current coding block, and rateEstTab corresponds to Table 5.

[0114] The coefficient processing module 84 is used to calculate the lastSigCoeffFlagRate[cIdx][x][y] as follows:

[0115] Where lastSigCoeffFlagTabIdx is determined by x, y and the type of the current coding block.

[0116] The coefficient processing module 84 is used to calculate the coeffAbsLevelRate[cIdx][x][y] as follows:

[0117] in,

[0118] greaterThanOneFlagTabIdx is always 1,

[0119] Wherein GolombCodeLength is the code length after the absolute value of QCoeff[cIdx][x][y] is encoded with Golomb-Rice code, and eqProbRateEstTab is obtained by querying Table 6.

[0120] The coefficient processing module 84 is used to calculate the fine-tuning non-zero transform coefficients as follows:

[0121] The second encoding cost is calculated as follows: .

[0122] An embodiment of the present application also proposes an electronic device, including a processor and a memory; the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the rate-distortion optimized quantization method based on H264 of any of the above embodiments.

[0123] The embodiments of the present application further provide a computer program product. When an executable computer program in the computer program product is executed by a processor, the rate-distortion optimization quantization method based on H264 of any of the above embodiments can be implemented.

[0124] Regarding the device in the above embodiment, the specific manner in which the processor performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0125] Fig. 9 1 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 1200 may be provided as a server. Fig. 9 , the device 1200 includes a processing component 1222, which further includes one or more processors, and a memory resource represented by a memory 1232 for storing instructions that can be executed by the processing component 1222, such as an application. The application stored in the memory 1232 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1222 is configured to execute instructions to perform the above-mentioned rate-distortion optimized quantization method based on H264.

[0126] The device 1200 may also include a power component 1226 configured to perform power management of the device 1200, a wired or wireless network interface 1250 configured to connect the device 1200 to a network, and an input / output (I / O) interface 1258. The device 1200 may operate based on an operating system stored in the memory 1232, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.

[0127] In an exemplary embodiment, a computer program product is also provided, and the computer program product may include a memory 1232, and the instructions stored in the memory 1232 may be executed by the processing component 1222 of the device 1200 to complete the above method. For example, the storage medium computer program product may be software, or a memory containing software such as a fixed memory ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device.

[0128] In the present invention, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. The term "plurality" refers to two or more than two, unless otherwise clearly defined.

[0129] The above description of the embodiments is to facilitate those of ordinary skill in the art to understand and apply the present application. It is obvious that those familiar with the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without having to pay creative labor. Therefore, the present application is not limited to the embodiments herein, and improvements and modifications made by those skilled in the art based on the contents disclosed in the present application without departing from the scope and spirit of the present application are within the scope of the present application.

Claims

1. A rate-distortion optimization quantization method based on H264, characterized in that: Applied to an encoder, the encoder implements the H264 / AVC standard, the method comprising: Receive video frames to be processed; Dividing and transforming the video frame to be processed to obtain a plurality of first transformation blocks; Performing parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients; Fine-tuning the prequantization coefficient using a preset fine-tuning method; Using the fine-tuned prequantization coefficients and the zeroing rule corresponding to the transform block size of the first transform block, corresponding coefficients in the first transform block are set to zero to obtain second transform coefficients after rate-distortion optimized quantization.

2. The method according to claim 1, characterized in that The preset fine-tuning method comprises: Fine-tuning method 1 sets the non-zero prequantization coefficients to zero; Fine-tuning method 2 keeps the sign of non-zero pre-quantization coefficient unchanged but reduces the absolute value by one; Fine-tuning method three keeps the pre-quantization coefficient unchanged and selects the fine-tuning method with the lowest coding cost.

3. The method according to claim 1, characterized in that The method of performing parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients includes: After initializing the context model to a predefined value, a quantization offset parameter and a quantization scaling parameter are defined; The pre-quantization coefficient is calculated based on the quantization offset parameter and the quantization scaling parameter; the calculation formula is as follows: ； Among them, TCoeff[cIdx][x][y] is the transform coefficient, cIdx is the Y, U, and V component index, x and y are the horizontal and vertical coordinates of the transform coefficient in the first transform block, sign[cIdx][x][y] is the sign of the transform coefficient, and QCoeff[cIdx][x][y] is the pre-quantization result.

4. The method according to claim 1 or 2, characterized in that: The first transform block includes: a 4×4 transform block and an 8×8 transform block.

5. The method according to claim 4, characterized in that The calculation method of the quantization offset parameter includes: If the first transform block is a 4×4 transform block, the calculation formula of the quantization offset parameter is: ； If the first transform block is an 8×8 transform block, the calculation formula of the quantization offset parameter is: ； Where QP is the quantization parameter and bitDepth is the bit width of the encoding.

6. The method according to claim 4, characterized in that The calculation method of the quantization scaling parameter includes: If the first transform block is a 4×4 transform block, the calculation formula of the quantization scaling parameter is: ； If the first transform block is an 8×8 transform block, the calculation formula of the quantization scaling parameter is: ； Among them, scalingList4x4 can be determined by querying the 4x4 quantization matrix table, and scalingList8x8 can be determined by querying the 8x8 quantization matrix table.

7. The method according to claim 3, characterized in that The prequantization coefficients are fine-tuned, and corresponding coefficients in the first transform block are set to zero according to a zeroing rule corresponding to a transform block specification of the first transform block, so as to obtain second transform coefficients after rate-distortion optimized quantization.

8. The method according to claim 7, characterized in that Fine-tune the non-zero prequantization coefficients according to three preset fine-tuning methods, calculate first encoding costs of the three fine-tuning methods, and select the fine-tuning method with the smallest encoding cost according to the first encoding cost; The coding cost of the fine-tuned quantized coefficient is the second coding cost; If it is an 8x8 block, according to the principle of minimizing coding cost, the non-zero quantized coefficients at the end of the scanning order are set to zero; Also according to the principle of minimizing coding cost, all transform coefficients in the first transform block are set to zero to obtain the second transform coefficients after rate-distortion optimized quantization.

9. The method according to claim 7, characterized in that: If it is an 8x8 transform block, several non-zero quantized coefficients at the end of the scanning order are set to zero. For any position in the 8x8 transform block, if the coding cost of setting all the non-zero coefficients after the scanning order to zero is less than the coding cost of not setting them to zero, then all the non-zero coefficients thereafter are set to zero.

10. The method according to claim 7, characterized in that: If the coding cost of setting the entire transform block to zero is less than the coding cost of not setting it to zero, the entire transform block is set to zero.

11. The method according to claim 8, characterized in that After obtaining the pre-quantization coefficient QCoeff[cIdx][x][y], the method further includes: Calculate quantization error and encoding bit rate; The quantization error is calculated as follows: ； If the first transform block is a 4×4 transform block, the quantization error restoration processing method is as follows: ； If the first transform block is an 8×8 transform block, the quantization error restoration processing method is as follows: ； Among them, distortionScaler4x4 is determined based on the 4x4 distortion weight factor table, and distortionScaler8x8 is determined based on the 8x8 distortion weight factor table; The encoding bit rate is calculated as follows: ； Among them, sigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is zero; lastSigCoeffFlagRate[cIdx][x][y] is the rate estimate of the syntax element that specifies whether the current quantization coefficient is the last non-zero quantization coefficient in the first transform block; coeffAbsLevelRate[cIdx][x][y] is the rate estimate of the syntax element that encodes the absolute value of the non-zero quantization coefficient.

12. The method according to claim 11, characterized in that The sigCoeffFlagRate[cIdx][x][y] is calculated as follows: ； Where sigCoeffFlagTabIdx is determined by x, y and the type of the current coding block, and rateEstTab is the rate estimation table.

13. The method according to claim 12, characterized in that The lastSigCoeffFlagRate[cIdx][x][y] is calculated as follows: ； Where lastSigCoeffFlagTabIdx is determined by x, y and the type of the current coding block.

14. The method according to claim 11, characterized in that The coeffAbsLevelRate[cIdx][x][y] is calculated as follows: ； in, ； greaterThanOneFlagTabIdx is always 1, ； Where GolombCodeLength is the code length after the absolute value of QCoeff[cIdx][x][y] is encoded with Golomb-Rice code, and eqProbRateEstTab is determined by the equal probability rate estimation table.

15. The method according to claim 7, characterized in that The fine-tuning of the non-zero transform coefficient and calculating a second coding cost corresponding to the fine-tuned non-zero transform coefficient includes: The fine-tuning non-zero transform coefficients are calculated as follows: ； The second encoding cost is calculated as follows: 。 16. A rate-distortion optimization quantization device based on H264, characterized in that: Applied to an encoder, applied to an encoder, the encoder implements the H264 / AVC standard, the device comprising: A receiving module, used for receiving a video frame to be processed; A transformation processing module, used for dividing and transforming the video frame to be processed to obtain a plurality of first transformation blocks; A quantization processing module, configured to perform parallel quantization processing on the first transform coefficients in the first transform block using a rate-distortion optimization quantization algorithm to obtain pre-quantization coefficients; The coefficient processing module is used to fine-tune the prequantization coefficient using a preset fine-tuning method; use the fine-tuned prequantization coefficient and the zeroing rule corresponding to the transform block specification of the first transform block to set the corresponding coefficient in the first transform block to zero, so as to obtain the second transform coefficient after rate-distortion optimized quantization.

17. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method according to any one of claims 1 to 15.

18. A computer program product having a computer program stored thereon, characterized in that: When the executable computer program in the computer program product is executed by a processor, the method according to any one of claims 1 to 15 can be implemented.

Images