Hardware pipeline implementation method based on scanning line parallel entropy coding binarization optimization

By using the hardware flow implementation method of scanning line parallel entropy encoding in AVS3 entropy encoding, the problem of data dependence and implementation complexity in the existing technology is solved, high throughput and simplified hardware design is achieved, and it is suitable for AVS3 codec chips 16pixel/cycle processing.

CN118972589AActive Publication Date: 2024-11-15HANGZHOU DIANZI UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411026348.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-15
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

The existing binary algorithm based on AVS3 entropy encoding has a large number of strong data dependencies and complex hardware structure implementation, which is difficult to meet the processing requirements of 16pixel/cycle.

Method used

Using the hardware flow implementation method based on parallel entropy encoding of scanning lines, by obtaining coefficient data of a single frame image, blocking coefficient data based on scanning areas of multiple channels, performing chunking traversal in parallel, and building an extended array and a two-dimensional array to remove it. Data dependency, calculates flag bits and context index increments, and performs sequential output.

Benefits of technology

It improves the throughput rate of hardware design, reduces algorithm data dependence, reduces hardware implementation difficulty, and realizes processing of 16 pixels of data encoding in one loop, suitable for AVS3 codec chip design.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118972589A_ABST
    Figure CN118972589A_ABST
Patent Text Reader

Abstract

The invention discloses a hardware pipeline implementation method based on scanning line parallel entropy coding binarization optimization, and the method comprises the steps: obtaining coefficient data of a single-frame image, carrying out the blocking of the coefficient data based on a scanning region of a plurality of channels, and carrying out the parallel execution of block traversal among the plurality of channels; an expansion array and a two-dimensional array are constructed for coefficient traversal in the blocks, reference coding coefficient information of the previous block and coding coefficient information of the current block are stored in the expansion array, and coding coefficient information of the current block is stored in a first dimensional array in the two-dimensional array; the second dimension array stores previous reference coding coefficient information corresponding to the coding coefficient information of each current block; during block traversal, each coding coefficient of the current block and the corresponding previous reference coding coefficient are preprocessed according to the result of the array after the last cycle is ended, and the second dimension array is used for calculating a flag bit; and traversing all blocks of the plurality of channels, calculating a context index increment of a flag bit of the current coding coefficient, and outputting results in sequence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of video encoding, and in particular relates to a hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization. Background Art

[0002] AVS3 uses the entropy coding method of Advanced Entropy Code (AEC), which is divided into three processes: binarization, context modeling, and binary arithmetic coding. AVS3 video coding abandons the transform coefficient coding method based on coefficient group CG (Coefficient Group) and uses SRCC (Scan Region-based Coefficient Coding) as the coding method, which adopts an inverse zig-zag type scan. After determining the transport block (TB) mode for a coding unit, a coding block flag (CBF) will be issued to indicate whether the TB has non-zero coefficients. If a TB contains non-zero coefficients, there will be a residual coding process to encode the position and value of each non-zero coefficient in the TB. SRCC is a scheme for encoding this information. After SRCC determines the scan area, the significance flag is calculated for each position in the area. The significance flag indicates whether there is a non-zero coefficient at the position, and only non-zero coefficients need to be transmitted. Using SRCC as the coding method, the coding efficiency is improved more for high-resolution sequences, and most of the gain comes from the selection of the scan area and the carefully selected context model set for amplitude coding.

[0003] The existing binarization algorithm based on AVS3 entropy coding has the following main problems:

[0004] 1. There is a lot of strong data dependency in the context model update and arithmetic coding process, and the algorithm hardware design timing is poor;

[0005] 2. The hardware structure is complex and it is difficult to meet the processing requirements of 16 pixels / circle. Summary of the invention

[0006] In order to solve the shortcomings of the prior art, reduce the difficulty of hardware implementation at high throughput, optimize timing logic, and improve coding efficiency, the present invention adopts the following technical solutions:

[0007] A hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization includes the following steps:

[0008] Step S1: Obtain coefficient data of a single frame image, divide the coefficient data into blocks based on the scanning areas of multiple channels, and perform block traversal in parallel among multiple channels;

[0009] Step S2: construct an extended array and a two-dimensional array for coefficient traversal in the block, in which the reference coding coefficient information of the previous block and the coding coefficient information of the current block are stored in the extended array, and in the two-dimensional array, the first dimension array stores the coding coefficient information of the current block, and the second dimension array stores the previous reference coding coefficient information corresponding to the coding coefficient information of each current block; when traversing the blocks, pre-process each coding coefficient of the current block and its corresponding previous (5) reference coding coefficients according to the result of the array after the last loop, and the second dimension array is used to calculate the flag bit (such as the importance flag bit, GT1 and GT2); since the data area does not overlap, the data dependency is removed, so that it is convenient to run the loop in parallel;

[0010] Step S3: traverse all blocks of multiple channels, calculate the context index increment of the flag bit of the current coding coefficient, and output the results sequentially.

[0011] Furthermore, in the step S1, an array for storing the component parameters of each channel and the starting block information is added, and the channel coefficient array is expanded to a structure type for storing multiple channels, which is used for parallel execution of block traversal among multiple channels.

[0012] Furthermore, in the step S1, the origin of the scanning area of ​​each channel is obtained respectively, and the number of blocks of each channel is calculated with a number of coefficients as one block. The blocks of the channel are traversed in turn, and the traversal within each channel starts from the last block and traverses forward to the first block, and the traversal within the block is also from back to front.

[0013] Furthermore, the extended array in step S2 has a total of 21 elements, the first 5 elements store the five coding coefficient information of the previous cycle, and the last 16 elements store the 16 coding coefficient information of the current cycle. At the end of the cycle, the last 5 elements are moved to the first 5.

[0014] Furthermore, in the two-dimensional array in step S2, 16 elements of the first dimension correspond to 16 coding coefficient information scanned in the current loop, and according to the extended array of the current loop, the first 5 coding coefficient information corresponding to each coding coefficient information is stored in the second dimension, and the corresponding second-dimensional array elements are directly passed in when calculating the flag.

[0015] Furthermore, the flags in step S2 include importance flags, and an extended array of importance flags stores the non-zero data of the reference coding coefficients of the previous block and the non-zero data of the coding coefficients of the current block. In the two-dimensional array of importance flags, the first-dimensional array stores the non-zero data of the corresponding previous reference coding coefficients according to the extended array elements of the current importance flags. In step S3, the blocks are traversed based on the non-zeroness of the coding coefficients to calculate the context index increment of the importance flag of the current coding coefficient, and the coding coefficient array of the importance flags is obtained, and the array is output sequentially.

[0016] Furthermore, the flag in step S2 also includes GT1, and the extended array of GT1 stores data on whether the reference coding coefficient of the previous block is greater than 1 and whether the coding coefficient of the current block is greater than 1. In the two-dimensional array of GT1, the first-dimensional array stores data on whether the corresponding previous reference coding coefficient is greater than 1 according to the extended array element of the current importance flag; in step S3, the blocks are traversed based on whether the coding coefficient is greater than 1 to calculate the context index increment of the current coding coefficient greater than 1, and the importance flag and the coding coefficient array of GT1 are obtained, and the arrays are output sequentially.

[0017] Furthermore, the flag in step S2 also includes GT2, and the extended array of GT2 stores data on whether the reference coding coefficient of the previous block is greater than 2 and whether the coding coefficient of the current block is greater than 2. In the two-dimensional array of GT2, the first-dimensional array stores data on whether the corresponding previous reference coding coefficient is greater than 2 according to the extended array element of the current importance flag; in step S3, the blocks are traversed based on whether the coding coefficient is greater than 2 to calculate the context index increment of the current coding coefficient greater than 2, and the coding coefficient arrays of the importance flag, GT1 and GT2 are obtained, GT1 is stored in the even bits, GT2 is stored in the odd bits, and the arrays are output sequentially.

[0018] Furthermore, in step S3, the redundant part of the coding coefficient greater than 2 is outputted by exponential Golomb coding.

[0019] Furthermore, in step S3, the sign bit variable of the block non-zero coefficient is encoded and outputted using a bypass method.

[0020] The advantages and beneficial effects of the present invention are:

[0021] The present invention performs real-time parallel pipeline processing in the hardware architecture, improves the throughput of the hardware design, and, under the premise of ensuring high throughput, reduces the algorithm data dependence, thereby reducing the difficulty of hardware implementation and optimizing the timing logic; the present invention is suitable for the field of AVS3 codec chip design, and the pipeline hardware structure can realize data encoding of 16 pixels in one cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of calculating three-channel data in an embodiment of the present invention.

[0023] Figure 2 1 is a scanning sequence diagram for multi-channel processing in an embodiment of the present invention.

[0024] Figure 3a It is a schematic diagram of a data coefficient preprocessing method in the prior art.

[0025] Figure 3b It is a schematic diagram of a data coefficient preprocessing method according to an embodiment of the present invention.

[0026] Figure 4 It is a schematic diagram of the data processing of outputting GT0, 1, and 2 in an embodiment of the present invention.

[0027] Figure 5 It is a schematic diagram of the processing of residual coefficients and symbol data in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0029] Traditional entropy coding binarization algorithms are mostly based on software algorithm optimization. Due to the data dependence and computational complexity of the binarization algorithm, its implementation in hardware becomes difficult. And with the rapidly growing demand for real-time video coding, it is difficult to meet the SRCC processing requirements in real-time video coding.

[0030] In view of this, the present invention discloses a hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization. According to the parameter list of the AVS3 entropy coding binarization algorithm, the input is the logarithmic length and width of the single frame image, the origin coordinates of the scanning area of ​​the three channels, the starting block sequence number, the coefficient data of the single frame and cycle_num. Through the reverse zig-zag scanning, pre-processing is performed to calculate and output the binarized data stream. The calculation part of the process is as follows Figure 1 As shown, the following steps are included:

[0031] 1. Optimize single-channel processing to multi-channel processing; obtain coefficient data of a single frame image, divide the coefficient data into blocks based on the scanning areas of multiple channels, and perform block traversal in parallel among multiple channels;

[0032] In the prior art, each call of the enc_eco_srcc function only processes data of a single color channel, and the function needs to be called three times for each pixel processed. In order to achieve the hardware performance of 16 pixels / cycle, it is not possible to wait for one channel to be processed before processing the next channel. Therefore, the present invention optimizes the enc_eco_srcc function so that one call can process three channels.

[0033] Compared with the original function, the num_sig array is added to store the yuv components sr_x and sr_y respectively; the line_idx_in_lcu array is added to store the starting blocks of each channel respectively; the array coef that originally stored a single channel coefficient is expanded to a variable of type structure to store three channel coefficients.

[0034] At the beginning of the function, the origin of the scanning area of ​​each channel is taken out respectively, and the number of blocks of each channel is calculated with 16 coefficients as a block (the blocks are divided in reverse zig-zag scanning order), and then the blocks of the three channels are traversed in turn, such as Figure 2 As shown, the channels are traversed in the order of YUV, and the traversal within each channel starts from the last block and traverses forward to the first block, and traverses from back to front within the block.

[0035] 2. Preprocess the previous coded coefficients; construct an extended array and a two-dimensional array for coefficient traversal in the block, in which the reference coding coefficient information of the previous block and the coding coefficient information of the current block are stored in the extended array, and in the two-dimensional array, the first dimension array stores the coding coefficient information of the current block, and the second dimension array stores the previous reference coding coefficient information corresponding to the coding coefficient information of each current block; when traversing the blocks, preprocess each coding coefficient of the current block and its corresponding previous (5) reference coding coefficients according to the result of the array after the last loop, pack the data area corresponding to each coding coefficient, and calculate the flag bit (such as the importance flag bit, GT1 and GT2) based on the second dimension array; since the data areas do not overlap, the data dependency is removed, which facilitates the execution of the parallel running loop.

[0036] After SRCC determines the scanning area, it calculates the significance flag for each position in the area. The significance flag indicates whether there is a non-zero coefficient at that position. Only non-zero coefficients need to be transmitted. In addition, GT1 and GT2 are calculated to indicate coefficients greater than 1 and greater than 2. One of the steps in calculating the significance flag, GT1, and GT2 is to calculate the previously coded coefficients. coefficients), when the SRCC method performs context modeling in the process of entropy coding its syntax elements, the context index ctxIdx is used to represent the context, and one of the parameters ctxIdxInc is required to calculate the index, and its value is num_gt0+1 or num_gt12+1, where num_gt0 represents the number of the first five coded non-zero coefficients; if GT1 is calculated, num_gt12 is the number of the first five coded coefficients greater than 1; if GT2 is calculated, num_gt12 is the number of the first five coded coefficients greater than 2, and adding 1 means adding 1 to the number of coefficients that meet the importance flag in the five coefficients scanned before the coded coefficient, or adding 1 to the number of coefficients that meet GT1 or GT2 in the five non-zero coefficients scanned before the coefficient.

[0037] In the prior art (HPM16.0), for each block, the 16 coefficients in the block are first traversed from back to front to determine whether the coefficient is 0. If not, the absolute value of the coefficient is recorded in the abs_coef array, and the sign bit is recorded in the bit of the coef_signs variable (1 represents negative, 0 represents positive). At the same time, the importance flag is calculated. At this time, the non-zeroness of the five coefficients scanned before the current coefficient is required (stored by the prev_0val[5] array), and then the prev_0val array is updated, all elements are moved back one position, and the non-zeroness of the coefficient scanned this time is written to the first position of the array for use in the next scan. When the 16 coefficients in a block are scanned, if there are non-zero coefficients in the block, each non-zero coefficient is traversed, and GT1 and GT2 are calculated according to the absolute value recorded in the abs_coef array. At this time, the absolute values ​​of the first five non-zero coefficients scanned stored in the prev_12val array are required, and the result is encoded and output.

[0038] That is, two queues of size 5, prev_0val and prev_12val, are used in the prior art. After each coefficient is scanned, the non-zeroness of the element is queued into prev_0val. If the coefficient is a non-zero coefficient, its absolute value is queued into prev_12val. Since the scanning method is zigzag, the data is read randomly, and the cache coefficient can reduce the time to address the previous coefficient in each loop. However, this method is not conducive to parallel operation of loops, because each loop depends on the result calculated in the previous loop, and there is data dependency.

[0039] In the present invention, multiple arrays are additionally used, namely, the importance flag, GT1, and GT2. Each item constructs an extended array and a two-dimensional array respectively, and expands the existing prev_0val[5] array (5 elements) to 16+5 elements. In this way, when traversing each block, the data area corresponding to each coefficient of the first five coefficients corresponding to each coefficient of the block can be pre-processed and packaged according to the results of these arrays after the previous loop ends. Since the data areas do not overlap, the data dependency is removed, which facilitates the parallel operation of the loop, as shown in Figure 3 and the following table:

[0040]

[0041] Among them, sig_coeff16_ctx, cur_12val_para, and cur_12val_gt1_para respectively represent the extended arrays of the importance flag, GT1, and GT2, and respectively represent the two-dimensional arrays of the importance flag, GT1, and GT2.

[0042] 3. Separate calculation from output; traverse all blocks of multiple channels, calculate the context index increment of the flag bit of the current coefficient, and output the results sequentially.

[0043] In the output result stage, due to parallel computing, the timing of output results will also be uncertain. In order to solve this problem, the function can be divided into two main parts, that is, calculation and output separation. The first part is responsible for traversing all the blocks of the three channels and calculating some key results, such as the importance flag, GT1, GT2 and the remaining absolute level. This part is computationally intensive and requires careful analysis of each block to determine its importance and relationship with other blocks. After completing these calculations, these results are filled into the corresponding variables. The process design is as follows: Figure 4 , Figure 5As shown in the figure, the second part outputs the results in order according to the data in the variables. This step is executed sequentially and depends on the results calculated in the first part. In this way, the accuracy and order of the output results can be ensured, and reliable results can be obtained even in parallel computing.

[0044] In the prior art, i is traversed from 0 to 15 in steps of one, and ctx_gt0 (the context index increment storing the importance flag of the current coefficient) is calculated through the com_get_ctx_gt0_inc function and the various parameters in the previous text. If the coefficient is not 0, the absolute value of the coefficient is recorded in the abs_coef array, and the sign bit is recorded in the bit of the coef_signs variable (1 represents negative, 0 represents positive), and the importance flag is calculated using the prev_0val array. Then traverse each non-zero coefficient, calculate GT1 and GT2 according to the absolute value recorded in the abs_coef array and the data stored in the prev_12val array, and encode the result for output. Then, the excess part of the coefficient with an absolute value greater than 2 is output by exponential Columbus encoding. Finally, the variable coef_signs that records the sign bit of the non-zero coefficient of this block is encoded and output using the bypass method.

[0045] In the present invention, i is traversed from 0 to 15 in step one in the new method, and ctx_gt0 (storing the context index increment of the importance flag of the current coefficient) is calculated through the com_get_ctx_gt0_inc function and the various parameters in the previous text and stored in ctx_gt0_coeff_part[i]; then similarly, ctx_gt1 (storing the context index increment of the current coefficient greater than one) and ctx_gt2 (storing the context index increment of the current coefficient greater than two) are calculated through com_get_ctx_gt1_inc, and stored in the gt1_ctx_part_array array (the even bits, i=2k, store GT1; the odd bits, i=2k+1, store GT2), because GT1 refers to a coefficient greater than 1, and GT2 refers to a coefficient greater than 2. Therefore, GT1 needs to be calculated only when the coefficient is greater than 1, and gt1_enable is 1 at this time. GT2 needs to be calculated only when the coefficient is greater than 2, and gt1_symbol is 1 at this time (indicating that GT2 needs to be calculated next). Store the gt1_enable of all traversed coefficients into gt1_enable_array, and store gt1_symbol into gt1_symbol_array. Finally, output gt1_ctx_part_array, gt1_enable_array, and gt1_symbol_array, such as Figure 4 And the following table shows:

[0046]

[0047] Then, the redundant part of the coefficient greater than 2 (i.e. the value of the coefficient -2) needs to be output by exponential Golomb coding, and the variable coef_signs recording the sign bit of the non-zero coefficient of the block is encoded and output by the bypass method, such as Figure 5 As shown in the following table:

[0048]

[0049] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization, Features The steps include: Step S1: Obtain coefficient data of a single frame image, divide the coefficient data into blocks based on the scanning areas of multiple channels, and perform block traversal in parallel among multiple channels; Step S2: constructing an extended array and a two-dimensional array for coefficient traversal in the block, wherein the extended array stores the reference coding coefficient information of the previous block and the coding coefficient information of the current block, and the two-dimensional array stores the coding coefficient information of the current block in the first dimension, and the second dimension stores the previous reference coding coefficient information corresponding to the coding coefficient information of each current block; When traversing the blocks, each coding coefficient of the current block and its corresponding previous reference coding coefficient are preprocessed according to the result of the array after the previous loop ends. The second-dimensional array is used to calculate the flag bit; Step S3: traverse all blocks of multiple channels, calculate the context index increment of the flag bit of the current coding coefficient, and output the results sequentially.

2. The hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization according to claim 1, Features: In the step S1, an array for storing the component parameters of each channel and the starting block information is added, and the channel coefficient array is expanded to a structure type for storing multiple channels, which is used for parallel execution of block traversal among multiple channels.

3. The hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization according to claim 1, Features: In step S1, the origin of the scanning area of ​​each channel is obtained respectively, and the number of blocks of each channel is calculated with a number of coefficients as one block. The blocks of the channel are traversed in sequence, and the traversal within each channel starts from the last block and traverses forward to the first block, and the traversal within the block is also from back to front.

4. The hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization according to claim 1, Features: The extended array in step S2 has 21 elements in total, the first 5 elements store the five coding coefficient information of the previous cycle, and the last 16 elements store the 16 coding coefficient information of the current cycle. At the end of the cycle, the last 5 elements are moved to the first 5.

5. The hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization according to claim 1, Features: In the two-dimensional array in step S2, the 16 elements of the first dimension correspond to the 16 coding coefficient information scanned in the current cycle. According to the extended array of the current cycle, the first 5 coding coefficient information corresponding to each coding coefficient information is stored in the second dimension, and the corresponding second-dimensional array elements are directly passed in when calculating the flag bit.

6. The hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization according to claim 1, Features: The flags in step S2 include importance flags, the extended array of importance flags stores the non-zero data of the reference coding coefficients of the previous block and the non-zero data of the coding coefficients of the current block, and in the two-dimensional array of importance flags, the first-dimensional array stores the non-zero data of the corresponding previous reference coding coefficients according to the extended array elements of the current importance flags; In the step S3, the blocks are traversed based on the non-zeroness of the coding coefficients to calculate the context index increment of the importance flag of the current coding coefficient, obtain the coding coefficient array of the importance flag, and output the array in sequence.

7. The hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization according to claim 6, Features: The flag bit in step S2 also includes GT1, the extended array of GT1 stores data on whether the reference coding coefficient of the previous block is greater than 1 and whether the coding coefficient of the current block is greater than 1, and in the two-dimensional array of GT1, the first-dimensional array stores data on whether the corresponding previous reference coding coefficient is greater than 1 according to the extended array element of the current importance flag bit; In step S3, the blocks are traversed based on whether the coding coefficient is greater than 1 to calculate the context index increment when the current coding coefficient is greater than 1, obtain the importance flag and the coding coefficient array of GT1, and output the array in sequence.

8. The hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization according to claim 7, Features: The flag bit in step S2 also includes GT2, the extended array of GT2 stores data on whether the reference coding coefficient of the previous block is greater than 2 and whether the coding coefficient of the current block is greater than 2, and in the two-dimensional array of GT2, the first-dimensional array stores data on whether the corresponding previous reference coding coefficient is greater than 2 according to the extended array element of the current importance flag bit; In step S3, the blocks are traversed based on whether the coding coefficient is greater than 2 to calculate the context index increment when the current coding coefficient is greater than 2, and the importance flag, GT1 and GT2 coding coefficient arrays are obtained, and the arrays are output sequentially.

9. The hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization according to claim 8, Features: In step S3, the redundant part of the coding coefficient greater than 2 is outputted by performing exponential Golomb coding.

10. The hardware pipeline implementation method based on scan line parallel entropy coding binarization optimization according to claim 1, Features: In step S3, the sign bit variables of the block non-zero coefficients are encoded and output using a bypass method.

Citation Information

Patent Citations

  • Image coding and decoding method and device and storage medium

    CN116132685A

  • Coding and decoding method of coefficient, encoder, decoder and computer storage medium

    CN116982317A

  • Variable bit rate image compression method and system based on dual-resolution context network

    CN117812265A

  • Method, apparatus, and storage medium for encoding / decoding feature map for machine vision

    US20230336780A1