A Rate-Distortion Optimized Quantization Method Based on Scanline Parallelism

Abstract

This invention belongs to the field of AVS3 video coding, specifically involving a rate-distortion optimized quantization method based on scan-line parallelism. After completing the pre-quantization process, the pre-quantization coefficients are calculated in parallel using a column scan sequence. Multiple sets of pre-quantization coefficient data are calculated each time, and the optimal quantization coefficient is determined by comparing multiple sets of data. Then, parallel decision-making is performed on non-zero positions to determine the optimal non-zero position on the scan line. Finally, a shift operation is performed to set the last-bit coefficient to zero. This invention, in RDOQ, proposes to perform parallel processing while preserving local dependencies under the condition of conforming to the zig-zag scan pattern through locality optimization. This invention reduces coding loss while maintaining good performance.

Description

Technical Field

[0001] This invention belongs to the field of AVS3 video coding, specifically relating to a rate-distortion optimized quantization method based on scanline parallelism. Background Technology

[0002] With the explosive growth of video data in recent years, related video compression coding standards have been continuously proposed. Among them, the AVS3 video compression coding standard can effectively reduce the burden of video data transmission and storage. Rate-distortion optimized quantization (RDOQ) technology provides significant coding gains in the third-generation audio and video coding standard (AVS3), but it involves complex calculations of the optimal quantization coefficients, which increases the complexity of quantization to some extent.

[0003] The RDOQ technology for AVS3 is mainly optimized in the following four stages.

[0004] 1. Pre-quantification process stage.

[0005] 2. Optimal Coefficient Decision (OCD) stage.

[0006] 3. The final non-zero position decision (LNPD) stage.

[0007] 4. Zero value setting (ZS) stage.

[0008] In RDOQ, all coefficients are searched sequentially in a zig-zag scan order, resulting in low parallelism and hindering real-time processing. While methods using fixed dependencies have been employed to improve parallelism, they significantly impact coding performance. Due to the complexity of RDOQ technology, the parallel computation problem in this direction has not yet been effectively solved. Current parallel scan computation methods in RDOQ employ fixed-cost dependency approaches, leading to substantial performance losses. Summary of the Invention

[0009] To address the aforementioned technical problems in the existing technology, this invention provides a rate-distortion optimized quantization method based on scan-line parallelism, comprising the following steps: After completing the pre-quantization process, the pre-quantization coefficients are calculated in parallel using a column scan sequence; multiple sets of pre-quantization coefficient data are calculated each time, and the optimal quantization coefficient is determined by comparing multiple sets of data; then, parallel decision-making is performed on non-zero positions to determine the optimal non-zero position on the scan line; finally, the shifted last-bit coefficient is set to zero.

[0010] Furthermore, in the column scanning order, the run values ​​of the coordinate points of the first column and the last row are initialized according to formula (3), and the level values ​​are initialized according to formula (4). The specific formulas are as follows:

[0011]

[0012]

[0013] Where, the level value represents the optimal quantization value that minimizes the RD cost, i.e., the optimal coefficient quantization value; run represents the number of zeros between two non-zero coefficients, i.e., the run length; assuming the first column and the last row, i represents the row coordinate of the current TU block, and j represents the column coordinate of the current TU block. is the prequantization coefficient in the i-th row and j-th column of formula (1).

[0014] Furthermore, local dependency retention is adopted, and the scan lines are divided into solid scan lines and dashed scan lines.

[0015] Furthermore, for solid scan lines:

[0016] Implement the function to obtain the dependency value of the previous coordinate point in the scan line region;

[0017] Define L (i,j) The decision variables are as follows:

[0018]

[0019] Where i∈[0, height-1] represents the row coordinate of the current TU block, and j∈[0, width-1] represents the instance coordinate of the current TU block; the value of the decision variable of the current coordinate point is determined based on the prequantization coefficient of the current coordinate point;

[0020] The update of the dependency on run between two non-zero coefficient coordinate blocks is reflected by the decision variable, as shown in the following formula:

[0021]

[0022] in The run value is the coordinate point (i, j) of the k-th scan line;

[0023] The update of the current dependency level is determined by the previous non-zero optimal coefficient. Under this premise, the specific formula is as follows:

[0024]

[0025] Where i∈[0, height-2] represents the row coordinate of the current TU block, and j∈[1, width-1] represents the instance coordinate of the current TU block; It is the previous non-zero optimal coefficient.

[0026] Furthermore, for dashed line scanning:

[0027] The dependencies at the virtual scan line are assigned values. The dependency run is calculated according to formula (3), and the dependency level is calculated according to formula (4). That is, the values ​​of run and level are assigned in advance on the virtual scan line.

[0028] Furthermore, it also includes RD-cost calculation:

[0029] For each scan line, the data involved in the calculation includes: the x and y coordinates of the last position last_x and last_y, the cost of the last non-zero position last_one, the cumulative cost of the entire scan line temp_cost, and the cost from the start of the scan line to the optimal non-zero position end_pos_cost.

[0030] For calculating the RD cost on the scan line, the positions before and after the solid scan line are directly compared by accumulating the RD-cost, as shown in the following formula:

[0031] temp_cost k +last_one k <end_pos_cost k-1 +last_one k-1 (8)

[0032] Where k and k-1 are two adjacent points on the scan line; if they are less than 1, the current position is considered optimal.

[0033] Conversely, for dashed scan lines, the difference comparison method is used when calculating RD-cost, thus achieving positive cumulative RD-cost comparison. The formula is as follows:

[0034] temp_cost k +end_pos_cost k-1 -last_one k +last_one k-1 >0 (9)

[0035] If the above expression is greater than zero, then the current position is considered optimal.

[0036] The method of this invention, within RDOQ, proposes to achieve parallel processing while preserving local dependencies through locality optimization, provided that the zig-zag scanning pattern is followed. This invention reduces coding loss while maintaining good performance. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the RDO process in AVS3;

[0038] Figure 2 This is a schematic diagram of the RDOQ process in AVS3;

[0039] Figure 3 This is a schematic diagram of the zig-zag scanning process;

[0040] Figure 4 This is a schematic diagram of local dependency preservation for solid scan lines;

[0041] Figure 5 This is a schematic diagram of the improved RDOQ process in a parallel architecture. Detailed Implementation

[0042] The invention will now be further described with reference to the accompanying drawings.

[0043] In the AVS3 mode decision-making process, the optimal mode is determined by calculating the rate-distortion cost of the corresponding prediction mode. For example... Figure 1 As shown, the transformed coefficients need to be quantized during the pattern decision-making process. Quantization can be achieved using scalar quantization or RDOQ. While RDOQ, combined with rate-distortion optimization, offers significant performance improvements, it also introduces greater computational complexity.

[0044] like Figure 2 As shown, RDOQ mainly consists of four stages.

[0045] During the pre-quantization phase, RDOQ performs a pre-quantization operation for each transform coefficient based on the quantization parameter (QP). First, it calculates the initial scalar quantization value L of the transform coefficient at each frequency position (u, v). sq (u,v), the specific pre-quantization calculation formula is as follows:

[0046]

[0047] Where c(u,v), q, and θ are the transform coefficients, the quantization step size, and the quantization offset, respectively.

[0048] In the Optimal Coefficient Decision (OCD) stage, it is necessary to consider multiple candidates (0, |l...) i |-1,|l i |) A detailed search is performed based on zig-zag scanning to determine the optimal value output for each quantization coefficient. Table 1 shows the results. i | and optional quantization values ​​0, |l i |-1 and |l i The relationship between |.

[0049] Among them, zig-zag scans, such as Figure 3 As shown.

[0050] <![CDATA[|l i |]]> Candidate value list 0 0 2 0,1,2 ... ... N 0,N,N-1

[0051] Table 1. Quantification coefficient table.

[0052] The TU is traversed in zig-zag scan order. For each coefficient, the RD cost is calculated for each possible quantization value, and the one with the smallest RD cost is selected as the optimal quantization value. The specific formula is as follows:

[0053]

[0054] Here, err represents the corresponding distortion rate, λ represents the Lagrange operator, and rate represents the coding rate of the candidate value, which is updated through entropy coding and the context model. In AVS3, rate can be obtained from the circular table, and the index of the table is related to the context-dependent value.

[0055] The Last Non-zero Position Decision (LNPD) stage is also based on the rate-distortion optimization criterion. The scan begins with the first non-zero coefficient in the current TU block. If the current position is a non-zero coefficient, it is assumed to be the last non-zero coefficient position. The accumulated RD cost is calculated and compared with previous values, and the position with the lower cost is selected as the last non-zero position. This process is repeated until the last coefficient in the TU is reached. Finally, the coefficients after the last non-zero position are set to zero, thus obtaining the final non-zero quantized coefficient positions.

[0056] During the Zero Set phase, all coefficients following the last non-zero coefficient are set to zero. This is done in zig-zag order, such as... Figure 3 As shown.

[0057] RDOQ parallel column scanning works column-by-column, scanning the coordinates of each row corresponding to the current column and initializing or updating the dependencies of each coordinate. Since dependencies play a crucial role in calculating the RD loss value, two important dependencies, run and level, are introduced below.

[0058] For operations that depend on `run` and `level`, the `level` value represents the optimal quantization value that minimizes the RD cost, i.e., the optimal coefficient quantization value. `run` represents the number of zeros between two non-zero coefficients, i.e., the run length. The `run` and `level` values ​​will affect subsequent RD costs, such as `temp_cost`, `end_pos_cost`, and `last_one`.

[0059] Based on RDOQ parallel column scanning, due to the fixed column scanning order, some coordinate points cannot obtain the previous dependency value, which inevitably causes loss during the encoding process and reduces the encoding performance.

[0060] To address the aforementioned problems, we propose a local dependency-preserving optimization algorithm. This algorithm utilizes solid and dashed scan lines, as follows: Figure 4 As shown, preserving local dependencies achieves better coding performance and reduces coding loss.

[0061] like Figure 5 As shown, after the prequantization process is completed, the prequantization coefficients are calculated in parallel using a column scan sequence. Multiple sets of prequantization coefficient data are calculated each time, and the optimal quantization coefficients are determined by comparing these multiple sets of data.

[0062] Then, parallel decision-making is performed on the non-zero positions to determine the optimal non-zero position on the scan line. Finally, the last bit coefficient of the shift operation is set to zero.

[0063] The specific plan mainly consists of the following three processes:

[0064] 1. Initialization 2. Preservation of local dependencies 3. RD-cost calculation.

[0065] 1. Initialization:

[0066] like Figure 4 As shown, the run values ​​of the coordinate points in the first column and the last row are initialized according to formula (3), and the level values ​​are initialized according to formula (4). The specific formulas are as follows:

[0067]

[0068]

[0069] In this context, assuming the first column and last row are specified, i represents the row coordinate of the current TU block, and j represents the column coordinate of the current TU block. is the prequantization coefficient in the i-th row and j-th column of formula (1).

[0070] For example, such as Figure 4 As shown, for the (0,0) coordinate point, if the currently obtained prequantization value is 6, run will be calculated according to formula (3). (0,0) The value is 0, level (0,0) The value is 6. If the previously obtained prequantization value is 0, then run (0,0) The value is 1, level (0,0) The value is 0. Similarly, the coordinates of the first column and the last row can be initialized in sequence according to formula (3) and formula (4), which will provide a basis for subsequent calculations.

[0071] 2. Local dependency preservation:

[0072] During dependency retention, due to the influence of column-based input, some coordinate blocks cannot obtain the previous dependency value. Therefore, it is necessary to distinguish between solid scan lines and dashed scan lines. Dashed scan lines and solid scan lines are as follows: Figure 4 As shown.

[0073] Solid scan line:

[0074] The solid scan line region can obtain the dependency value of the previous coordinate point, and thus be updated. For example, as... Figure 4 As shown, the run and level values ​​at coordinates (2,0) can be obtained from initialization. Using run and level, the optimal coefficients can be obtained in the optimal coefficient decision-making process. (Formula 2). In the next iteration, the previous optimal coefficient will be used as the level value at coordinate (1,1) according to formula (7), and the run value will be obtained according to formula (5) and formula (6). Then, the optimal coefficient decision process will be carried out again for run and level, and so on.

[0075] Define L (i,j) The decision variables are as follows:

[0076]

[0077] Where i∈[0, height-1] represents the row coordinate of the current TU block, and j∈[0, width-1] represents the instance coordinate of the current TU block. The value of the decision variable for the current coordinate point is determined based on the prequantization coefficient of the current coordinate point.

[0078] The update of the dependency on run between two non-zero coefficient coordinate blocks is reflected by the decision variable, as shown in the following formula:

[0079]

[0080] in For the k-th scan line (e.g.) Figure 4 As shown, the scan line at coordinate point (0,0) is called the 0th scan line. The run value at coordinate point (i,j) is the same as that at coordinate point (i,j).

[0081] Taking the second scan line as an example, the points on this scan line are (2,0), (1,1), and (0,2). Assuming that the prequantization coefficients of the two coordinate points (2,0) and (1,1) are both 0, and the prequantization coefficient of the coordinate point (0,2) is non-zero, then according to formulas (5) and (6)... Since run represents the number of zeros between two non-zero coefficients, run needs to be set to 0 when the quantization coefficient is not zero.

[0082] The update of the current dependency level is determined by the previous non-zero optimal coefficient. Under this premise, the specific formula is as follows:

[0083]

[0084] Where i∈[0, height-2] represents the row coordinate of the current TU block, and j∈[1, width-1] represents the instance coordinate of the current TU block.

[0085] It is a non-zero optimal coefficient in formula (2).

[0086] For solid scan lines, update the dependent run according to formula (5-6) and update the level according to formula (7).

[0087] Dashed scan line:

[0088] For coordinate points on the dashed scan line, data dependency is a potential obstacle to parallelism. For example, after the first column of scanning is completed, since the scanning is always performed from right to left, the coordinate points on the dashed scan line cannot obtain the dependency corresponding to the previous coordinate point based on the scanning direction. This can be illustrated using the secondary diagonal as an example:

[0089] Coordinates (3,0) (e.g.) Figure 4 As shown, a column scan is first performed to complete the initialization process. The coordinate (2,1) is then calculated. However, the dependency value at coordinate (2,1) needs to be provided by coordinate (1,2), so the dependency cannot meet the requirements of parallel column scan. Therefore, the dependency at the virtual scan line needs to be assigned a value.

[0090] The dependency on run is calculated according to formula (3), and the dependency on level is calculated according to formula (4). That is, run and level are assigned values ​​in advance on the virtual scan line. This method effectively reduces the error caused by data dependency.

[0091] 3. RD-cost calculation:

[0092] For each scan line, the data involved in the calculation mainly includes: the horizontal and vertical coordinates of the last position last_x and last_y, the cost of the last non-zero position last_one, the cumulative cost of the entire scan line temp_cost, and the cost from the start of the scan line to the optimal non-zero position end_pos_cost.

[0093] For calculating the RD cost on the scan line, the positions before and after the solid scan line are directly compared by accumulating the RD-cost, as shown in the following formula:

[0094] temp_cost k +last_one k<end_pos_cost k-1 +last_one k-1 (8)

[0095] Here, k and k-1 are two adjacent points on the scan line. If the value is less than 1, the current position is considered optimal.

[0096] Conversely, for dashed scan lines, a difference comparison method is used when calculating RD-cost, thereby achieving positive cumulative RD-cost comparison. The formula is as follows:

[0097] temp_cost k +end_pos_cost k-1 -last_one k +last_one k-1 >0 (9)

[0098] If the above expression is greater than zero, then the current position is considered optimal.

[0099] Locality optimization algorithm pseudocode

[0100]

[0101] 9 Zero value setting

[0102] 10 END.

Claims

1. A rate-distortion optimized quantization method based on scanline parallelism, characterized in that... The following steps are taken: After completing the pre-quantization process, the pre-quantization coefficients are calculated in parallel using column scanning order; multiple sets of pre-quantization coefficient data are calculated each time, and the optimal quantization coefficient is determined by comparing multiple sets of data; then, parallel decision-making is performed on non-zero positions to determine the optimal non-zero position on the scan line. Finally, the shift last bit coefficient is set to zero. In the column scanning order, the run value of the coordinate points of the first column and the last row is initialized according to formula (3), and the level value is initialized according to formula (4). The specific formulas are as follows: Where, the level value represents the optimal quantization value that minimizes the RD cost, i.e., the optimal coefficient quantization value; run represents the number of zeros between two non-zero coefficients, i.e., the run length; assuming the first column and the last row, i represents the row coordinate of the current TU block, and j represents the column coordinate of the current TU block. The prequantization coefficients in the i-th row and j-th column of formula (1); Local dependency preservation is employed, and the scan lines are divided into solid lines and dashed lines; For solid scan lines: Implement the function to obtain the dependency value of the previous coordinate point in the scan line region; Define L (i,j) The decision variables are as follows: Where i∈[0, height-1] represents the row coordinate of the current TU block, and j∈[0, width-1] represents the column coordinate of the current TU block; the value of the decision variable of the current coordinate point is determined based on the prequantization coefficient of the current coordinate point; The update of the dependency on run between two non-zero coefficient coordinate blocks is reflected by the decision variable, as shown in the following formula: in The run value is the coordinate point (i, j) of the k-th scan line; The update of the current dependency level is determined by the previous non-zero optimal coefficient. Under this premise, the specific formula is as follows: Where i∈[0, height-2] represents the row coordinate of the current TU block, and j∈[1, width-1] represents the column coordinate of the current TU block; This is the previous non-zero optimal coefficient; For dashed line scan: The dependencies at the virtual scan line are assigned values. The dependency run is calculated according to formula (3), and the dependency level is calculated according to formula (4). That is, the values ​​of run and level are assigned in advance on the virtual scan line.

2. The rate-distortion optimized quantization method based on scanline parallelism as described in claim 1, characterized in that: It also includes RD-cost calculation: For each scan line, the data involved in the calculation includes: the x and y coordinates of the last position last_x and last_y, the cost of the last non-zero position last_one, the cumulative cost of the entire scan line temp_cost, and the cost from the start of the scan line to the optimal non-zero position end_pos_cost. For calculating the RD cost on the scan line, the positions before and after the solid scan line are directly compared by accumulating the RD-cost, as shown in the following formula: temp_cost k +last_one k <end_pos_cost k-1 +last_one k-1 (8) Where k and k-1 are two adjacent points on the scan line; if they are less than 1, the current position is considered optimal. Conversely, for dashed scan lines, the difference comparison method is used when calculating RD-cost, thus achieving positive cumulative RD-cost comparison. The formula is as follows: temp_cost k +end_pos_cost k-1 -last_one k +last_one k-1 >0 (9) If the above expression is greater than zero, then the current position is considered optimal.

Images