Hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization

Abstract

The present invention discloses a hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization. By acquiring coefficient data of a single frame image, the coefficient data is divided into blocks based on the scanning areas of multiple channels, and the blocks are traversed in parallel between multiple channels; an extended array and a two-dimensional array are constructed for coefficient traversal within the blocks. The extended array stores reference coding coefficient information of the previous block and coding coefficient information of the current block. In the two-dimensional array, the first-dimensional array stores each coding coefficient information of the current block, and the second-dimensional array stores the previous reference coding coefficient information corresponding to the coding coefficient information of each current block; when traversing the blocks, each coding coefficient of the current block and its corresponding previous reference coding coefficient are preprocessed according to the result of the array after the end of the previous loop, and the second-dimensional array is used to calculate a flag bit; all blocks of multiple channels are traversed, the context index increment of the flag bit of the current coding coefficient is calculated, and the results are output sequentially.

Description

Technical Field

[0001] The present invention belongs to the technical field of video coding, and in particular relates to a hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization. Background Art

[0002] AVS3 uses Advanced Entropy Code (AEC) entropy coding, which consists of three steps: binarization, context modeling, and binary arithmetic coding. AVS3 video coding abandons the coefficient group (CG)-based transform coefficient coding approach and instead uses SRCC (Scan Region-based Coefficient Coding), which employs an inverse zig-zag scanning scheme. After determining the transport block (TB) mode for a coding unit, a coded block flag (CBF) is issued to indicate whether the TB contains nonzero coefficients. If a TB contains nonzero coefficients, a residual coding process encodes the position and value of each nonzero coefficient within the TB. SRCC encodes this information. After determining the scan region, SRCC calculates a significance flag for each position within the region. The significance flag indicates whether the position has a nonzero coefficient, and only nonzero coefficients are transmitted. Using SRCC as the coding method improves coding efficiency even more for high-resolution sequences, with most of the gains coming from the selection of the scan region and the carefully chosen context model set for amplitude coding.

[0003] The existing binarization algorithm based on AVS3 entropy coding has the following main problems:

[0004] 1. There are a lot of strong data dependencies in the context model update and arithmetic coding process, and the algorithm hardware design has poor timing;

[0005] 2. The hardware structure is complex and it is difficult to meet the processing requirements of 16 pixels / circle. Summary of the Invention

[0006] To address the shortcomings of the existing technology, reduce the difficulty of hardware implementation at high throughput, optimize timing logic, and improve coding efficiency, the present invention adopts the following technical solutions:

[0007] A hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization includes the following steps:

[0008] Step S1: Obtain coefficient data of a single frame image, divide the coefficient data into blocks based on the scanning areas of multiple channels, and perform block traversal in parallel across multiple channels;

[0009] Step S2: constructing an extended array and a two-dimensional array for coefficient traversal within the block. The extended array stores the reference coding coefficient information of the previous block and the coding coefficient information of the current block. In the two-dimensional array, the first-dimensional array stores the coding coefficient information of the current block, and the second-dimensional array stores the previous reference coding coefficient information corresponding to the coding coefficient information of each current block. When traversing the block, based on the array result after the previous loop, each coding coefficient of the current block and its corresponding previous (5) reference coding coefficients are preprocessed. The second-dimensional array is used to calculate flags (such as the importance flag, GT1 and GT2). Since the data areas do not overlap, data dependencies are eliminated, thereby facilitating the execution of parallel loops.

[0010] Step S3: traverse all blocks of multiple channels, calculate the context index increment of the flag bit of the current coding coefficient, and output the results sequentially.

[0011] Furthermore, in step S1, an array for storing the component parameters of each channel and the starting block information is added, and the channel coefficient array is expanded to a structure type for storing multiple channels, which is used for parallel execution of block traversal among multiple channels.

[0012] Furthermore, in step S1, the origin of the scanning area of ​​each channel is obtained respectively, and the number of blocks of each channel is calculated with a number of coefficients as one block. The blocks of the channel are traversed in sequence, and the traversal within each channel starts from the last block and traverses forward to the first block, and the traversal within the block is also from back to front.

[0013] Furthermore, the extended array in step S2 has a total of 21 elements, the first 5 elements store the five coding coefficient information of the previous cycle, and the last 16 elements store the 16 coding coefficient information of the current cycle. At the end of the cycle, the last 5 elements are moved to the first 5.

[0014] Furthermore, in the two-dimensional array in step S2, the 16 elements of the first dimension correspond to the 16 coding coefficient information scanned in the current loop. According to the extended array of the current loop, the first 5 coding coefficient information corresponding to each coding coefficient information is stored in the second dimension, and the corresponding second-dimensional array elements are directly passed in when calculating the flag bit.

[0015] Furthermore, the flags in step S2 include importance flags, and the extended array of the importance flags stores the non-zero data of the reference coding coefficients of the previous block and the non-zero data of the coding coefficients of the current block. In the two-dimensional array of the importance flags, the first-dimensional array stores the non-zero data of the corresponding previous reference coding coefficients according to the extended array elements of the current importance flags; in step S3, the blocks are traversed based on the non-zeroness of the coding coefficients to calculate the context index increment of the importance flag of the current coding coefficient, and the coding coefficient array of the importance flags is obtained, and the array is output sequentially.

[0016] Furthermore, the flag in step S2 also includes GT1, and the extended array of GT1 stores data on whether the reference coding coefficient of the previous block is greater than 1 and whether the coding coefficient of the current block is greater than 1. In the two-dimensional array of GT1, the first-dimensional array stores data on whether the corresponding previous reference coding coefficient is greater than 1 based on the extended array element of the current importance flag; in step S3, the blocks are traversed based on whether the coding coefficient is greater than 1 to calculate the context index increment of the current coding coefficient greater than 1, obtain the importance flag and the coding coefficient array of GT1, and output the array sequentially.

[0017] Furthermore, the flag in step S2 also includes GT2, and the extended array of GT2 stores data on whether the reference coding coefficient of the previous block is greater than 2 and whether the coding coefficient of the current block is greater than 2. In the two-dimensional array of GT2, the first-dimensional array stores data on whether the corresponding previous reference coding coefficient is greater than 2 based on the extended array element of the current importance flag; in step S3, the blocks are traversed based on whether the coding coefficient is greater than 2 to calculate the context index increment of the current coding coefficient greater than 2, and the coding coefficient arrays of the importance flag, GT1 and GT2 are obtained, GT1 is stored in the even bits, and GT2 is stored in the odd bits, and the arrays are output sequentially.

[0018] Furthermore, in step S3, the redundant part of the coding coefficient greater than 2 is subjected to exponential Golomb coding and output.

[0019] Furthermore, in step S3, the sign bit variables of the block non-zero coefficients are encoded and output using a bypass method.

[0020] The advantages and beneficial effects of the present invention are:

[0021] The present invention performs real-time parallel pipeline processing in the hardware architecture, thereby improving the throughput of the hardware design. Moreover, while ensuring high throughput, it reduces the algorithm data dependency, thereby lowering the difficulty of hardware implementation and optimizing the timing logic. The present invention is applicable to the field of AVS3 codec chip design, and the pipeline hardware structure can process 16-pixel data encoding in one cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of calculating three-channel data in an embodiment of the present invention.

[0023] Figure 2 1 is a scanning sequence diagram for multi-channel processing in an embodiment of the present invention.

[0024] Figure 3a This is a schematic diagram of the data coefficient preprocessing method in the prior art.

[0025] Figure 3b 2 is a schematic diagram of a data coefficient preprocessing method according to an embodiment of the present invention.

[0026] Figure 4 1 is a schematic diagram of the processing of outputting GT0, 1, and 2 data in an embodiment of the present invention.

[0027] Figure 5 2 is a schematic diagram of the processing of residual coefficients and symbol data in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not intended to limit the present invention.

[0029] Traditional entropy coding binarization algorithms are mostly based on software algorithm optimization. Due to the data dependence and computational complexity of the binarization algorithm, its implementation in hardware becomes difficult. Moreover, with the rapidly growing demand for real-time video coding, it is difficult to meet the SRCC processing requirements in real-time video coding.

[0030] In view of this, the present invention discloses a hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization. According to the parameter list of the AVS3 entropy coding binarization algorithm, the input is the logarithmic length and width of the single frame image, the origin coordinates of the scanning area of ​​the three channels, the starting block sequence number, the coefficient data of the single frame and cycle_num. Through reverse zig-zag scanning, preprocessing is performed and the binarized data stream is calculated and output. The calculation process is as follows: Figure 1 As shown, the following steps are included:

[0031] 1. Optimize single-channel processing to multi-channel processing; obtain coefficient data of a single frame image, divide the coefficient data into blocks based on the scanning area of ​​multiple channels, and perform block traversal in parallel across multiple channels;

[0032] In the prior art, each call to the enc_eco_srcc function only processes data for a single color channel. This function needs to be called three times for each pixel processed. To achieve a hardware performance of 16 pixels / cycle, it is not possible to wait for one channel to be processed before processing the next. Therefore, the present invention optimizes the enc_eco_srcc function so that it can process three channels in one call.

[0033] Compared with the original function, the num_sig array is added to store the YUV components sr_x and sr_y respectively; the line_idx_in_lcu array is added to store the starting blocks of each channel respectively; the array coef that originally stored a single channel coefficient is expanded to a variable of type structure to store three channel coefficients.

[0034] At the beginning of the function, the origin of the scanning area of ​​each channel is taken out respectively, and the number of blocks of each channel is calculated with 16 coefficients as a block (blocked in reverse zig-zag scanning order), and then the blocks of the three channels are traversed in turn, such as Figure 2 As shown, the channels are traversed in the order of YUV. The traversal within each channel starts from the last block and traverses forward to the first block, and traverses from back to front within the block.

[0035] 2. Preprocess the previous coded coefficients; construct an extended array and a two-dimensional array for coefficient traversal within the block. In the extended array, the reference coding coefficient information of the previous block and the coding coefficient information of the current block are stored. In the two-dimensional array, the first-dimensional array stores the coding coefficient information of the current block, and the second-dimensional array stores the previous reference coding coefficient information corresponding to the coding coefficient information of each current block; when traversing the block, according to the result of the array after the end of the previous loop, preprocess each coding coefficient of the current block and its corresponding previous (5) reference coding coefficients, pack the data area corresponding to each coding coefficient, and calculate the flag bit (such as the importance flag bit, GT1 and GT2) based on the second-dimensional array; since the data areas do not overlap, data dependency is removed, which facilitates the execution of the parallel loop.

[0036] After SRCC determines the scanning area, it calculates the significance flag (SignificanceFlag) for each position in the area. The significance flag indicates whether there is a non-zero coefficient at that position. Only non-zero coefficients need to be transmitted. In addition, GT1 and GT2 are calculated to indicate coefficients greater than 1 and greater than 2. One of the steps in calculating the significance flag, GT1, and GT2 is to calculate the previously coded coefficients. coefficients). When the SRCC method performs context modeling during the entropy coding of its syntax elements, a context index ctxIdx is used to represent the context. The calculation of this index requires one of the parameters ctxIdxInc, whose value is num_gt0+1 or num_gt12+1, where num_gt0 represents the number of the first five coded non-zero coefficients; if GT1 is being calculated, num_gt12 is the number of the first five coded coefficients greater than 1; if GT2 is being calculated, num_gt12 is the number of the first five coded coefficients greater than 2. The addition of 1 is the addition of 1 to the number of coefficients that meet the significance flag in the five coefficients scanned before the coded coefficient, or the addition of 1 to the number of coefficients that meet GT1 or GT2 in the five non-zero coefficients scanned before the coefficient.

[0037] In the prior art (HPM16.0), for each block, the 16 coefficients in the block are first traversed from back to front to determine whether the coefficient is 0. If it is not zero, the absolute value of the coefficient is recorded in the abs_coef array, and the sign bit is recorded in the bit of the coef_signs variable (1 represents negative, 0 represents positive). At the same time, the importance flag is calculated. At this time, the non-zeroness of the five coefficients scanned before the current coefficient is required (stored by the prev_0val[5] array). Then, the prev_0val array is updated, all elements are shifted back one position, and the non-zeroness of the coefficient scanned this time is written to the first position of the array for use in the next scan. When the 16 coefficients in a block are scanned, if there is a non-zero coefficient in the block, each non-zero coefficient is traversed and GT1 and GT2 are calculated according to the absolute value recorded in the abs_coef array. At this time, the absolute value of the first five non-zero coefficients scanned before the current coefficient is required. The result is encoded and output.

[0038] Specifically, the prior art uses two queues of size 5, prev_0val and prev_12val. After each coefficient is scanned, the nonzero value of the element is queued in prev_0val. If the coefficient is nonzero, its absolute value is queued in prev_12val. Because the scanning method is zigzag, data is read randomly, and caching coefficients can reduce the time required to access previous coefficients in each loop. However, this method is not conducive to running loops in parallel, as each loop relies on the results of the previous loop, resulting in data dependencies.

[0039] In the present invention, multiple arrays are additionally used, namely the importance flag, GT1, and GT2. Each item constructs an extended array and a two-dimensional array respectively, and expands the existing prev_0val[5] array (5 elements) to 16+5 elements. In this way, when traversing each block, the data area corresponding to each of the first five coefficients corresponding to each coefficient of the block can be pre-processed and packaged according to the results of these arrays after the end of the previous loop. Since the data areas do not overlap, data dependencies are eliminated, thereby facilitating the parallel operation of the loop, as shown in Figure 3 and the following table:

[0040]

[0041] Among them, sig_coeff16_ctx, cur_12val_para, cur_12val_gt1_para represent the extended arrays of the importance flag, GT1, and GT2, respectively, and represent the two-dimensional arrays of the importance flag, GT1, and GT2, respectively.

[0042] 3. Separate calculation and output; traverse all blocks of multiple channels, calculate the context index increment of the flag bit of the current coefficient, and output the results sequentially.

[0043] In the output result stage, due to parallel computing, the timing of output results will also be uncertain. To solve this problem, the function can be divided into two main parts, that is, calculation and output separation. The first part is responsible for traversing all the blocks of the three channels and calculating some key results, such as the importance flag, GT1, GT2 and the remaining absolute level. This part is computationally intensive and requires careful analysis of each block to determine its importance and its relationship with other blocks. After completing these calculations, these results are filled into the corresponding variables. The process design is as follows: Figure 4 、 Figure 5As shown in the figure, the second part outputs the results in sequence based on the data in the variables. This step is executed sequentially and depends on the results calculated in the first part. In this way, the accuracy and order of the output results can be guaranteed, and reliable results can be obtained even in parallel computing.

[0044] In the prior art, i is traversed from 0 to 15 in steps one, and ctx_gt0 (the context index increment storing the importance flag of the current coefficient) is calculated through the com_get_ctx_gt0_inc function and the various parameters in the previous text. If the coefficient is not 0, the absolute value of the coefficient is recorded in the abs_coef array, and the sign bit is recorded in the bit of the coef_signs variable (1 represents negative, 0 represents positive), and the importance flag is calculated using the prev_0val array. Then, each non-zero coefficient is traversed, and GT1 and GT2 are calculated according to the absolute value recorded in the abs_coef array and the data stored in the prev_12val array, and the result is encoded and output. Then, the excess part of the coefficient with an absolute value greater than 2 is output by exponential Columbus coding. Finally, the variable coef_signs that records the sign bit of the non-zero coefficient of this block is encoded and output using the bypass method.

[0045] In the present invention, the new method now traverses i from 0 to 15 step by step, and calculates ctx_gt0 (storing the context index increment of the importance flag of the current coefficient) through the com_get_ctx_gt0_inc function and the various parameters in the previous text and stores it in ctx_gt0_coeff_part[i]; then, similarly, ctx_gt1 (storing the context index increment of the current coefficient greater than one) and ctx_gt2 (storing the context index increment of the current coefficient greater than two) are calculated through com_get_ctx_gt1_inc, and stored in the gt1_ctx_part_array array (the even bits, i = 2k, store GT1; the odd bits, i = 2k+1, store GT2), because GT1 refers to a coefficient greater than 1, and GT2 refers to a coefficient greater than 2. Therefore, GT1 needs to be calculated only when the coefficient is greater than 1, and gt1_enable is 1 at this time. GT2 needs to be calculated only when the coefficient is greater than 2, and gt1_symbol is 1 at this time (indicating that GT2 needs to be calculated next). Store the gt1_enable of all traversed coefficients into gt1_enable_array, and store gt1_symbol into gt1_symbol_array. Finally, output gt1_ctx_part_array, gt1_enable_array, and gt1_symbol_array, such as Figure 4 and the following table:

[0046]

[0047] Then, the excess part of the coefficient greater than 2 (i.e. the value of the coefficient -2) needs to be output by exponential Golomb coding, and the variable coef_signs that records the sign bit of the non-zero coefficient of the block is encoded and output using the bypass method, such as Figure 5 As shown in the following table:

[0048]

[0049] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization, characterized in that The steps include: Step S1: Obtain coefficient data of a single frame image, divide the coefficient data into blocks based on the scanning areas of multiple channels, and perform block traversal in parallel across multiple channels; Step S2: constructing an extended array and a two-dimensional array for coefficient traversal within the block, wherein the extended array stores the reference coding coefficient information of the previous block and the coding coefficient information of the current block, and the two-dimensional array stores the coding coefficient information of the current block in the first dimension, and the second dimension stores the previous reference coding coefficient information corresponding to the coding coefficient information of each current block; When traversing the blocks, each coding coefficient of the current block and its corresponding previous reference coding coefficient are preprocessed according to the result of the array after the previous loop. The second-dimensional array is used to calculate the flag bit; Step S3: traverse all blocks of multiple channels, calculate the context index increment of the flag bit of the current coding coefficient, and output the results sequentially.

2. The hardware pipelining method based on scan line parallel entropy coding binarization optimization according to claim 1 is characterized in that: In the step S1, an array for storing the component parameters of each channel and the starting block information is added, and the channel coefficient array is expanded to a structure type for storing multiple channels, which is used for parallel execution of block traversal among multiple channels.

3. The hardware pipelining method based on scan line parallel entropy coding binarization optimization according to claim 1 is characterized in that: In step S1, the origin of the scanning area of ​​each channel is obtained respectively, and the number of blocks of each channel is calculated with a number of coefficients as one block. The blocks of the channel are traversed in sequence, and the traversal within each channel starts from the last block and traverses forward to the first block, and the traversal within the block is also from back to front.

4. The hardware pipelining method based on scan line parallel entropy coding binarization optimization according to claim 1 is characterized in that: The extended array in step S2 has a total of 21 elements, the first 5 elements store the five coding coefficient information of the previous cycle, and the last 16 elements store the 16 coding coefficient information of the current cycle. At the end of the cycle, the last 5 elements are moved to the first 5.

5. The hardware pipelining method based on scan line parallel entropy coding binarization optimization according to claim 1 is characterized in that: In the two-dimensional array in step S2, the 16 elements of the first dimension correspond to the 16 coding coefficient information scanned in the current loop. According to the extended array of the current loop, the first 5 coding coefficient information corresponding to each coding coefficient information is stored in the second dimension, and the corresponding second-dimensional array elements are directly passed in when calculating the flag bit.

6. The hardware pipelining method based on scan line parallel entropy coding binarization optimization according to claim 1, characterized in that: The flags in step S2 include an importance flag, an extended array of the importance flag stores the non-zero data of the reference coding coefficient of the previous block and the non-zero data of the coding coefficient of the current block, and a first-dimensional array of the two-dimensional array of the importance flag stores the non-zero data of the corresponding previous reference coding coefficient according to the extended array element of the current importance flag; In step S3, the blocks are traversed based on the non-zeroness of the coding coefficients to calculate the context index increment of the importance flag of the current coding coefficient, obtain the coding coefficient array of the importance flag, and output the array in sequence.

7. The hardware pipelining method based on scan line parallel entropy coding binarization optimization according to claim 6, characterized in that: The flag bit in step S2 further includes GT1, the extended array of GT1 stores data indicating whether the reference coding coefficient of the previous block is greater than 1 and whether the coding coefficient of the current block is greater than 1, and in the two-dimensional array of GT1, the first-dimensional array stores data indicating whether the corresponding previous reference coding coefficient is greater than 1 according to the extended array element of the current importance flag bit; In step S3, the blocks are traversed based on whether the coding coefficient is greater than 1 to calculate the context index increment when the current coding coefficient is greater than 1, obtain the importance flag and the coding coefficient array of GT1, and output the array in sequence.

8. The hardware pipelining method based on scan line parallel entropy coding binarization optimization according to claim 7, characterized in that: The flag bit in step S2 further includes GT2, the extended array of GT2 stores data indicating whether the reference coding coefficient of the previous block is greater than 2 and whether the coding coefficient of the current block is greater than 2, and in the two-dimensional array of GT2, the first-dimensional array stores data indicating whether the corresponding previous reference coding coefficient is greater than 2 according to the extended array element of the current importance flag bit; In step S3, the blocks are traversed based on whether the coding coefficient is greater than 2 to calculate the context index increment when the current coding coefficient is greater than 2, obtain the importance flag, the coding coefficient arrays of GT1 and GT2, and output the arrays in sequence.

9. The hardware pipelining method based on scan line parallel entropy coding binarization optimization according to claim 8, characterized in that: In step S3, the redundant part of the coding coefficient greater than 2 is subjected to exponential Golomb coding and outputted.

10. The hardware pipelining method based on scan line parallel entropy coding binarization optimization according to claim 1, characterized in that: In step S3, the sign bit variables of the block non-zero coefficients are encoded and output using a bypass method.

Images