HEVC (High Efficiency Video Coding) conversion coefficient code rate estimation device and method based on FPGA (Field Programmable Gate Array)
By designing a transform coefficient code rate estimation device based on FPGA in HEVC video encoding, using table lookup and linear estimation methods, the problem of low bit rate estimation efficiency in HEVC encoding is solved, and high parallelism and high efficiency bit rate calculation is achieved , suitable for the needs of transform blocks of various sizes.
Patent Information
- Application Number
- CN202411211428.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-12
AI Technical Summary
In HEVC video encoding, it is difficult for the prior art to quickly estimate the code rate under the premise of parallel computing, to meet the needs of multiple size transformation blocks, and the context model needs to be calculated one by one during the rate distortion optimization calculation process, which affects efficiency.
Design a HEVC transform coefficient code rate estimation device based on FPGA, including coefficient scanning module, buffer module, absolute value module, syntax element processing module and code rate estimation module. It uses table lookup and linear estimation methods to quickly estimate the code rate. Supports parallel computing and is suitable for multiple transform blocks of different sizes.
It realizes high parallelism and high efficiency bit rate estimation, reduces resource consumption, and improves bit rate calculation speed, and is suitable for HEVC encoders that require high throughput.
Smart Images

Figure CN118945354A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of video coding, and in particular relates to an FPGA-based HEVC transform coefficient bit rate estimation device and method. Background Art
[0002] With the rapid development of video streaming and the popularization of 5G mobile communication technology, people have an increasing demand for video quality. Major streaming platforms have begun to provide videos with higher definition and higher frame rate. In order to compress high-definition and high-frame rate videos more efficiently, the Joint Collaborative Team on Video Coding (JCT-VC) has proposed a new generation of high-efficiency video coding standard (HEVC).
[0003] During video transmission and storage, videos with low bitrate R (Rate) are more conducive to transmission and storage, but the distortion D (Distortion) will be greatly increased. On the contrary, high-quality videos with low distortion will increase the bitrate, increasing the pressure on network transmission. In order to minimize the distortion under a certain bitrate limit, a technology used in video coding is rate distortion optimization RDO (Rate Distortion Optimization). This technology searches for a set of parameters with the smallest distortion as the optimal video coding parameters under a certain bitrate limit. In the existing hybrid coding framework, rate distortion optimization is usually performed independently on a coding unit basis to ensure optimal data coding within a single coding unit.
[0004] The rate-distortion model currently commonly used in HEVC to describe the relationship between distortion and bit rate is the R-λ model. By using the Lagrange multiplier method and introducing the Lagrange coefficient λ, the constrained maximum value problem is converted into a non-constrained maximum value problem: minJ J=D+λR.
[0005] Among them, J represents the rate-distortion cost, D and R represent distortion and bit rate respectively, which can be applied to CTU (Coding Tree Unit), CU (Coding Unit), PU (Prediction Unit) and TU (Transform Unit).
[0006] In the rate-distortion optimization process of the transform unit, it is necessary to find a block of appropriate size for optimal data encoding. For intra-frame prediction, it is necessary to calculate and select the optimal block combination from 4×4, 8×8, and 16×16 to store and encode data. For inter-frame prediction, it is necessary to calculate and select the optimal block combination from 4×4, 8×8, 16×16, and 32×32 to store and encode data. In order to reduce the cycle cost of rate-distortion optimization calculation and improve throughput, one idea is to expand the rate-distortion optimization calculation into four calculation modules of 4×4, 8×8, 16×16, and 32×32 to calculate simultaneously to improve throughput. The design of the code rate calculation module should meet the needs of parallel calculation and adapt to transform blocks of various sizes.
[0007] In addition, in order to obtain the bit rate R, in normal entropy coding of HEVC, it is necessary to obtain the bit rate through context-adaptive binary arithmetic coding (CABAC). This process requires binarization of syntax elements, obtaining probabilities from context models, updating context models, and arithmetic coding. In the two steps of updating the context model and arithmetic coding, the value of a codeword of each syntax element will affect the encoding result of the next codeword, that is, the bit rates calculated by different transform block combinations will be different. In this case, the rate-distortion optimization calculation process can only calculate the context models one by one to obtain the accurate bit rate, which is contrary to the concept of parallel computing. Summary of the invention
[0008] In order to solve the above technical problems, the present invention provides an FPGA-based HEVC transform coefficient bit rate estimation device and method, which can quickly estimate the bit rate to obtain performance close to that of accurate bit rate calculation while meeting the needs of parallel computing and adapting to transform blocks of various sizes. Therefore, it is more suitable for occasions requiring high throughput, such as FPGA-based HEVC encoders.
[0009] The technical solution adopted by the present invention is: a HEVC transform coefficient bit rate estimation device based on FPGA, comprising: a coefficient scanning module, a buffer module, an absolute value module, a syntax element processing module, and a bit rate estimation module.
[0010] The coefficient scanning module is connected to the buffer module and the syntax element processing module respectively, the buffer module is connected to the absolute value module, the absolute value module is connected to the syntax element processing module, and the syntax element processing module is connected to the bit rate estimation module.
[0011] The bit rate estimation device inputs a transform block, that is, a coefficient block after transformation and quantization. The transform block size is 4×4, 8×8, 16×16, and 32×32, with a total of four sizes. Four transform coefficients are fixedly input each time until a full-size transform block is completed.
[0012] The coefficient scanning module includes: a counter, an output buffer module, a scanning sequence number module, 5 or calculation modules, and 4 comparison modules; the coefficient scanning module is used to scan the input coefficients to obtain the coordinate position of the last non-zero coefficient in the transformation block.
[0013] The counter in the coefficient scanning module is connected to the output buffer module and the scanning module respectively. The input coefficient is connected to the 5 OR calculation modules OR0~OR4. The scanning sequence module and the OR calculation module are connected to the 4 comparison modules. All the results are finally connected to the output buffer module. The counter module is used to record the order and number of each input data. The scanning sequence module converts the count value into the scanning order of the input data. The output buffer module is used to temporarily store data. OR0~OR3 of the 5 OR calculation modules is used to calculate whether the current input coefficient is zero, and OR4 is used to calculate whether all input coefficients are zero. COMP0~COMP3 of the 4 comparators is used to calculate the coefficient with the largest coordinate among the four coefficients, and COMP4 is used to calculate the coefficient with the largest coordinate among the new and old coefficients.
[0014] The buffer module is used to overcome the problem of inconsistency between the input sequence and the scanning sequence, temporarily store the input transformation data, and then output the data in the scanning sequence when outputting the data.
[0015] The absolute value module is used to calculate the positive and negative signs and absolute values of the transformation coefficients.
[0016] The syntax element processing module is used to convert the position information and value information of the non-zero transform coefficient into specific syntax element information for subsequent bit rate estimation. The syntax element processing module is implemented by a state machine. The state machine is used to convert the non-zero coefficient position information and coefficient value information into syntax element information. The syntax element information includes: syntax element value syntax_value, syntax element model starting number syntax_id and syntax element model type syntax_type.
[0017] The code rate estimation module comprises: a binarization module, a register, a probability estimation table module, a local context model module, an accumulator, and a selector; and is used to estimate the code rate of each syntax element.
[0018] The binarization module in the bit rate estimation module is connected to the register, which is divided into two paths. One path is for linear estimation and connected to the selector, and the other path is for table lookup estimation and connected to the local context model module and the probability estimation table module. The probability estimation table is connected to the selector, and the selector is connected to the accumulator.
[0019] The binarization module converts the syntax elements into binary strings using a binarization method. The register is used to temporarily store the binary string and output each bit of the binary string. The local context model module is used to synchronize the state of the global context model. The probability estimation table module stores the bit rate table for table lookup to obtain the bit rate. The selector is used to select the bit rate estimated by table lookup or linear estimation. The accumulator accumulates the estimated bit rates of all syntax elements and outputs the final bit rate.
[0020] Furthermore, in the syntax element processing module, the state machine conversion process is specifically as follows:
[0021] A1, initially in the IDLE state, if all coefficients are 0, then enter step A9, that is, enter the END state to end the bit rate estimation of the current transform block, if the coefficients are not all 0, then enter step A2 to start processing the non-zero coefficient position information;
[0022] A2, enter the LAST_X_PREFIX and LAST_Y_PREFIX states in sequence, process the coordinate prefixes last_sig_coeff_x_prefix and last_sig_coeff_y_prefix, if the coordinate suffix is not 0, enter LAST_SUFFIX, process the coordinate suffixes last_sig_coeff_x_prefix and last_sig_coeff_y_prefix, if the coordinate suffix is 0, do not process the suffix, after processing the coordinates, enter step A3 to start processing the absolute value information of the coefficients;
[0023] A3, enter the CSBF state, start processing from the sub-block where the last non-zero coefficient is located, if the csbf of the current coefficient sub-block is 0, it remains in the CSBF state and processes the next coefficient sub-block, if the current is 1 or it is the last sub-block, that is, csbf==1ORlast_cg==1, enter step A4, and further process the coefficient information of the current sub-block;
[0024] A4, enter the SIG_COEFF_FLAG state, process the 16 transform coefficients in the sub-block in the scanning order, if the coefficient is 0, the sig_coeff_flag flag at the current position is 0, otherwise sig_coeff_flag is 1, and only the coefficients with sig_coeff_flag=1 are further processed in the future, if all the coefficients are 0 and in the last sub-block, that is, csbf==0&last_cg==1, then end all processing and enter step A9, otherwise enter step A5;
[0025] A5. Enter the COEFF_G1_FLAG state. If the coefficient is greater than 1, coeff_abs_level_greater1_flag is 1. If it is equal to 1, it is 0. Only the first 8 non-zero coefficients are marked. The unmarked ones are 0 by default.
[0026] A6. If the absolute value of the coefficient is less than or equal to 2, directly enter step A7. Otherwise, if the absolute value of the coefficient is greater than 2, first enter the COEFF_G2_FLAG state, only mark the first coefficient greater than 2, set coeff_abs_level_greater2_flag = 1, and the unmarked ones are defaulted to 0, and then enter step A7;
[0027] A7, enter the COEFF_SIGN_FLAG state, mark the signs of all current coefficients, if the mark is a positive number, coeff_sign_flag is 0, if it is a negative number, it is 1, if there is a coefficient with a residual value, that is, coeff_remain! = 0, enter step A8, that is, enter the COEFF_ABS_REMAIN state, if all coefficients have no residual values and it is not the last sub-block, that is, last_cg == 0, then return to the CSBF state of step A3 to process the next sub-block, if all coefficients have no residual values and it is the last sub-block, that is, last_cg == 1, enter step A9;
[0028] A8, enter the COEFF_ABS_REMAIN state, mark the residual value, and will not mark it as a zero residual value. If the current sub-block is not the last one, that is, last_cg==0, then after processing all coefficients, return to the CSBF state of step A3 to process the next sub-block. If the current sub-block is the last one, that is, last_cg==1, enter step A9;
[0029] Among them, the numerical expression of the remaining value coeff_abs_remain is as follows:
[0030] coeff_abs_remain=|V|-1-coeff_abs_level_greater1_flag-coeff_abs_level_greater2_flag, where V represents the coefficient value.
[0031] A9, entering the END state, in which no operation or output is performed, and the device waits for all operations of other modules of the bit rate estimation device to be completed and then returns to the IDLE state to wait for the next transformation block.
[0032] Furthermore, in the syntax element processing module, all syntax elements are specifically as follows:
[0033] last_sig_coeff_x_prefix: There are 18 grammatical element models in total, estimated using a table lookup.
[0034] last_sig_coeff_y_prefix: There are 18 grammatical element models in total, estimated using a table lookup.
[0035] last_sig_coeff_x_suffix: No syntax element model, using linear estimation.
[0036] last_sig_coeff_y_suffix: No syntax element model, using linear estimation.
[0037] coded_sub_block_flag: There are 4 syntax element models in total, using linear estimation.
[0038] sig_coeff_flag: There are 42 syntax element models in total, using linear estimation.
[0039] coeff_abs_g1_flag: There are 24 syntax element models in total, using linear estimation.
[0040] coeff_abs_g2_flag: There are 8 syntax element models in total, using linear estimation.
[0041] coeff_sign_flag: No syntax element model, use linear estimation.
[0042] coeff_abs_level_remain: No syntax element model, use linear estimation.
[0043] The present invention also provides a HEVC transform coefficient rate estimation method based on FPGA, which is applied to the rate estimation device, and the specific steps are as follows:
[0044] S1, the input end of the bit rate estimation device inputs a transform block, enters a coefficient scanning module to perform coefficient scanning, and obtains the last non-zero coefficient coordinate position;
[0045] Four transform coefficients are input each time until a full-size transform block is completed. The input coefficients are bitwise ORed to detect whether the current coefficient is 0. After receiving the enable signal en, the counter starts to record the number and order of input transform coefficients, and generates a scan number based on the count value. The detection value and the corresponding scan number are concatenated to obtain a comparison value, which is then compared with the comparison values of the remaining coefficients to select the largest one until the input ends. The scan number in the maximum comparison value is taken out and converted into the last non-zero coefficient coordinates last_sig_x and last_sig_y. After scanning, the transform coefficient is generated according to the scan number to generate the address w_addr and the data itself w_coeff. All data will be temporarily stored in the output buffer module, and the final output data will be output by the output buffer module.
[0046] The coefficient scanning methods include: horizontal scanning from right to left, vertical scanning from bottom to top, and diagonal scanning from bottom right to top left.
[0047] After each 4×4 sub-block is input, the CSBF of the current sub-block is calculated. If all 16 coefficients of the sub-block are 0, the CSBF is 0, otherwise it is 1.
[0048] S2, the coefficient w_coeff scanned by the coefficient scanning module is stored in the buffer module according to the address w_addr generated by the scanning sequence number, and the subsequent data is taken out from the buffer module according to the scanning sequence and output to the absolute value module, the absolute value of the coefficient is taken, and the amplitude and sign of the coefficient are obtained;
[0049] Among them, the absolute value module reads the coefficient r_coeff from the buffer module by reading the address r_addr, and then outputs the transformation coefficient sign sign and the coefficient absolute value abs_coeff to the syntax element processing module.
[0050] S3, the syntax element processing module converts the non-zero coefficient position information and the remaining coefficient value information into syntax element information;
[0051] S4, a bit rate estimation module binarizes the syntax element obtained in step S3 to obtain a binary string bin, and performs bit rate estimation on the bin to obtain a bit rate of the syntax element;
[0052] The syntax element value syntax_value is binarized to obtain the binary string bin. The syntax_id determines the binarization method. Fixed-length coding does not need to be converted in FPGA, but truncated Rice code will convert syntax elements into prefixes and suffixes.
[0053] Then the code rate is estimated for each bit of the bin, including table lookup estimation and linear estimation, as follows:
[0054] (1) Table lookup estimation: Take each bit of the bin, obtain the corresponding context model offset value in the register according to the bin length, add it to the syntax_id to get the index, use the index to get the probability state value and the most likely symbol MPM from the local context model module, use the probability state value to look up two long and short code rates from the bit rate estimation table, then compare MPM with the bin. If the bin is equal to the MPM, select the short code rate. If the bin is not equal to the MPM, select the long code rate.
[0055] (2) Linear estimation: Get the bin length, then the code rate = bin length × 1.
[0056] S5. The rate accumulator accumulates the rates of all syntax elements obtained in step S4 and rounds them off, outputting the total rate as the estimated rate of the current transform block, that is, obtaining a rate estimate of the transform block.
[0057] Beneficial effects of the present invention: The device of the present invention includes: a coefficient scanning module, a buffer module, an absolute value module, a syntax element processing module, and a bit rate estimation module. The present invention provides a high-parallelism and high-efficiency bit rate estimation method for HEVC encoder rate distortion optimization calculation, which can quickly estimate the bit rate to meet the needs of parallel calculation and adapt to the needs of various sizes of transformation blocks to obtain performance close to that of accurate bit rate calculation. Two methods based on table lookup and linear estimation are adopted to avoid updating the context model and arithmetic coding process in entropy coding, which not only reduces resource consumption but also improves the bit rate calculation speed, and supports parallel calculation, which is suitable for simultaneous bit rate estimation of multiple different size transformation blocks, and is suitable for occasions such as FPGA-based HEVC encoders that require high throughput. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 It is a structural diagram of an FPGA-based HEVC transform coefficient bit rate estimation device of the present invention.
[0059] Figure 2 4 is a structural diagram of a coefficient scanning module in an embodiment of the present invention.
[0060] Figure 3 4 is a structural diagram of a bit rate estimation module in an embodiment of the present invention.
[0061] Figure 4 4 is a flow chart of the state machine conversion process of the syntax element processing module in an embodiment of the present invention.
[0062] Figure 5 A schematic diagram for introducing all syntax elements in an embodiment of the present invention.
[0063] Figure 6 Schematic diagram of the coefficient scanning method and sequence in an embodiment of the present invention.
[0064] Figure 7 It is a schematic diagram of simulation results of HEVC transform coefficient bit rate estimation under FPGA in an embodiment of the present invention. DETAILED DESCRIPTION
[0065] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0066] like Figure 1 As shown, an FPGA-based HEVC transform coefficient bit rate estimation device of the present invention includes: a coefficient scanning module, a buffer module, an absolute value module, a syntax element processing module, and a bit rate estimation module.
[0067] The coefficient scanning module is connected to the buffer module and the syntax element processing module respectively, the buffer module is connected to the absolute value module, the absolute value module is connected to the syntax element processing module, and the syntax element processing module is connected to the bit rate estimation module.
[0068] The bit rate estimation device inputs a transform block, that is, a coefficient block after transformation and quantization. The transform block size is 4×4, 8×8, 16×16, and 32×32, with a total of four sizes. Four transform coefficients are fixedly input each time until a full-size transform block is completed.
[0069] like Figure 2 As shown, the coefficient scanning module includes: a counter, an output buffer module, a scanning sequence number module, 5 or calculation modules, and 4 comparison modules; the coefficient scanning module is used to scan the input coefficient to obtain the coordinate position of the last non-zero coefficient in the transformation block.
[0070] The counter in the coefficient scanning module is connected to the output buffer module and the scanning module respectively. The input coefficient is connected to the 5 OR calculation modules OR0~OR4. The scanning sequence module and the OR calculation module are connected to the 4 comparison modules. All the results are finally connected to the output buffer module. The counter module is used to record the order and number of each input data. The scanning sequence module converts the count value into the scanning order of the input data. The output buffer module is used to temporarily store data. OR0~OR3 of the 5 OR calculation modules is used to calculate whether the current input coefficient is zero, and OR4 is used to calculate whether all input coefficients are zero. COMP0~COMP3 of the 4 comparators is used to calculate the coefficient with the largest coordinate among the four coefficients, and COMP4 is used to calculate the coefficient with the largest coordinate among the new and old coefficients.
[0071] The buffer module is used to overcome the problem of inconsistency between the input sequence and the scanning sequence, temporarily store the input transformation data, and then output the data in the scanning sequence when outputting the data.
[0072] The absolute value module is used to calculate the positive and negative signs and absolute values of the transformation coefficients.
[0073] The syntax element processing module is used to convert the position information and value information of the non-zero transform coefficient into specific syntax element information for subsequent bit rate estimation. The syntax element processing module is implemented by a state machine. The state machine is used to convert the non-zero coefficient position information and coefficient value information into syntax element information. The syntax element information includes: syntax element value syntax_value, syntax element model starting number syntax_id and syntax element model type syntax_type.
[0074] like Figure 3 As shown, the bit rate estimation module includes: a binarization module, a register, a probability estimation table module, a local context model module, an accumulator, and a selector; and is used to estimate the bit rate of each syntax element.
[0075] The binarization module in the bit rate estimation module is connected to the register, which is divided into two paths. One path is for linear estimation and connected to the selector, and the other path is for table lookup estimation and connected to the local context model module and the probability estimation table module. The probability estimation table is connected to the selector, and the selector is connected to the accumulator.
[0076] The binarization module converts the syntax elements into binary strings using a binarization method. The register is used to temporarily store the binary string and output each bit of the binary string. The local context model module is used to synchronize the state of the global context model. The probability estimation table module stores the bit rate table for table lookup to obtain the bit rate. The selector is used to select the bit rate estimated by table lookup or linear estimation. The accumulator accumulates the estimated bit rates of all syntax elements and outputs the final bit rate.
[0077] like Figure 4 As shown, in this embodiment, in the syntax element processing module, the state machine conversion process is specifically as follows:
[0078] A1, initially in the IDLE state, if all coefficients are 0, then enter step A9, that is, enter the END state to end the bit rate estimation of the current transform block, if the coefficients are not all 0, then enter step A2 to start processing the non-zero coefficient position information;
[0079] A2, enter the LAST_X_PREFIX and LAST_Y_PREFIX states in sequence, process the coordinate prefixes last_sig_coeff_x_prefix and last_sig_coeff_y_prefix, if the coordinate suffix is not 0, enter LAST_SUFFIX, process the coordinate suffixes last_sig_coeff_x_prefix and last_sig_coeff_y_prefix, if the coordinate suffix is 0, do not process the suffix, after processing the coordinates, enter step A3 to start processing the absolute value information of the coefficients;
[0080] A3, enter the CSBF state, start processing from the sub-block where the last non-zero coefficient is located, if the csbf (coded_sub_block_flag) of the current coefficient sub-block is 0, then remain in the CSBF state and process the next coefficient sub-block, if the current is 1 or it is the last sub-block, that is, csbf==1ORlast_cg==1, enter step A4, and further process the coefficient information of the current sub-block;
[0081] A4, enter the SIG_COEFF_FLAG state, process the 16 transform coefficients in the sub-block in the scanning order, if the coefficient is 0, the sig_coeff_flag flag at the current position is 0, otherwise sig_coeff_flag is 1, and only the coefficients with sig_coeff_flag=1 are further processed in the future, if all the coefficients are 0 and in the last sub-block, that is, csbf==0&last_cg==1, then end all processing and enter step A9, otherwise enter step A5;
[0082] A5. Enter the COEFF_G1_FLAG state. If the coefficient is greater than 1, coeff_abs_level_greater1_flag is 1. If it is equal to 1, it is 0. Only the first 8 non-zero coefficients are marked. The unmarked ones are 0 by default.
[0083] A6. If the absolute value of the coefficient is less than or equal to 2, directly enter step A7. Otherwise, if the absolute value of the coefficient is greater than 2, first enter the COEFF_G2_FLAG state, only mark the first coefficient greater than 2, set coeff_abs_level_greater2_flag = 1, and the unmarked ones are defaulted to 0, and then enter step A7;
[0084] A7, enter the COEFF_SIGN_FLAG state, mark the signs of all current coefficients, if the mark is a positive number, coeff_sign_flag is 0, if it is a negative number, it is 1, if there is a coefficient with a residual value, that is, coeff_remain! = 0, enter step A8, that is, enter the COEFF_ABS_REMAIN state, if all coefficients have no residual values and it is not the last sub-block, that is, last_cg == 0, then return to the CSBF state of step A3 to process the next sub-block, if all coefficients have no residual values and it is the last sub-block, that is, last_cg == 1, enter step A9;
[0085] A8, enter the COEFF_ABS_REMAIN state, mark the residual value, and will not mark it as a zero residual value. If the current sub-block is not the last one, that is, last_cg==0, then after processing all coefficients, return to the CSBF state of step A3 to process the next sub-block. If the current sub-block is the last one, that is, last_cg==1, enter step A9;
[0086] Among them, the numerical expression of the remaining value coeff_abs_remain is as follows:
[0087] coeff_abs_remain=|V|-1-coeff_abs_level_greater1_flag-coeff_abs_level_greater2_flag, where V represents the coefficient value.
[0088] A9, entering the END state, in which no operation or output is performed, and the device waits for all operations of other modules of the bit rate estimation device to be completed and then returns to the IDLE state to wait for the next transformation block.
[0089] like Figure 5 As shown, in this embodiment, in the syntax element processing module, all syntax elements are specifically as follows:
[0090] last_sig_coeff_x_prefix: There are 18 grammatical element models in total, estimated using a table lookup.
[0091] last_sig_coeff_y_prefix: There are 18 grammatical element models in total, estimated using a table lookup.
[0092] last_sig_coeff_x_suffix: No syntax element model, using linear estimation.
[0093] last_sig_coeff_y_suffix: No syntax element model, using linear estimation.
[0094] coded_sub_block_flag: There are 4 syntax element models in total, using linear estimation.
[0095] sig_coeff_flag: There are 42 syntax element models in total, using linear estimation.
[0096] coeff_abs_g1_flag: There are 24 syntax element models in total, using linear estimation.
[0097] coeff_abs_g2_flag: There are 8 syntax element models in total, using linear estimation.
[0098] coeff_sign_flag: No syntax element model, use linear estimation.
[0099] coeff_abs_level_remain: No syntax element model, use linear estimation.
[0100] This embodiment also provides a HEVC transform coefficient rate estimation method based on FPGA, which is applied to the rate estimation device, and the specific steps are as follows:
[0101] S1, the input end of the bit rate estimation device inputs a transform block, enters a coefficient scanning module to perform coefficient scanning, and obtains the last non-zero coefficient coordinate position;
[0102] Four transform coefficients are input each time until a complete transform block is completed. In order to overcome the inconsistency between the input order and the scanning order, the input coefficients are bitwise ORed to detect whether the current coefficient is 0. After receiving the enable signal en, the counter starts to record the number and order of input transform coefficients, and generates a scan number based on the count value. The detection value is concatenated with the corresponding scan number to obtain a comparison value, which is then compared with the comparison values of the remaining coefficients to select the largest one until the input ends. The scan number in the maximum comparison value is taken out and converted into the last non-zero coefficient coordinates last_sig_x and last_sig_y. After scanning, the transform coefficient is generated according to the scan number to generate the address w_addr and the data itself w_coeff. All data will be temporarily stored in the output buffer module, and the final output data will be output by the output buffer module.
[0103] The coefficient scanning method and order are as follows Figure 6 As shown, including: horizontal scanning from right to left ( Figure 6 (a)) Vertical scanning from bottom to top ( Figure 6 (b)) and scan diagonally from bottom right to top left ( Figure 6 (c)).
[0104] After each 4×4 sub-block is input, the CSBF of the current sub-block is calculated. If all 16 coefficients of the sub-block are 0, the CSBF is 0, otherwise it is 1.
[0105] S2, the coefficient w_coeff scanned by the coefficient scanning module is stored in the buffer module according to the address w_addr generated by the scanning sequence number, and the subsequent data is taken out from the buffer module according to the scanning sequence and output to the absolute value module, the absolute value of the coefficient is taken, and the amplitude and sign of the coefficient are obtained;
[0106] Among them, the absolute value module reads the coefficient r_coeff from the buffer module by reading the address r_addr, and then outputs the transformation coefficient sign sign and the coefficient absolute value abs_coeff to the syntax element processing module.
[0107] S3, the syntax element processing module converts the non-zero coefficient position information and the remaining coefficient value information into syntax element information;
[0108] S4, a bit rate estimation module binarizes the syntax element obtained in step S3 to obtain a binary string bin, and performs bit rate estimation on the bin to obtain a bit rate of the syntax element;
[0109] The syntax element value syntax_value is binarized to obtain the binary string bin. The syntax_id determines the binarization method. Fixed-length coding does not need to be converted in FPGA, but truncated Rice code will convert syntax elements into prefixes and suffixes.
[0110] Then the code rate is estimated for each bit of the bin, including table lookup estimation and linear estimation, as follows:
[0111] (1) Table lookup estimation: Take each bit of the bin, obtain the corresponding context model offset value in the register according to the bin length, add it to the syntax_id to get the index, use the index to get the probability state value and the most likely symbol MPM from the local context model module, use the probability state value to look up two long and short code rates from the bit rate estimation table, then compare MPM with the bin. If the bin is equal to the MPM, select the short code rate. If the bin is not equal to the MPM, select the long code rate.
[0112] (2) Linear estimation: Get the bin length, then the code rate = bin length × 1.
[0113] S5. The rate accumulator accumulates the rates of all syntax elements obtained in step S4 and rounds them off, outputting the total rate as the estimated rate of the current transform block, that is, obtaining a rate estimate of the transform block.
[0114] Figure 7This is the simulation result of HEVC transform coefficient rate estimation under FPGA in this embodiment. The figure shows the result of parallel calculation of three transform blocks, namely 8×8, 16×16 and 32×32, in which 8×8 calculates the rate 16 times, 16×16 calculates the rate 4 times, and 32×32 calculates the rate once. It can be seen from the figure that under simulation conditions, the present invention can be applied to multiple transform blocks of different sizes for simultaneous rate estimation. In addition, the rate results calculated by transform blocks of different sizes are different, and the sum of the small block rate is also different from the sum of the large block rate, which reflects that the calculation of transform blocks of different sizes is indispensable.
[0115] In summary, the present invention provides a high-parallelism and high-efficiency rate estimation method for the rate-distortion optimization calculation of the HEVC encoder, which can quickly estimate the rate to obtain performance close to that of the precise rate calculation under the premise of meeting the needs of parallel calculation and adapting to the needs of various sizes of transformation blocks. It adopts two methods based on table lookup and linear estimation, which eliminates the need to update the context model and arithmetic coding process in entropy coding, reduces resource consumption and improves the rate calculation speed, and supports parallel calculation, which is suitable for simultaneous rate estimation of multiple transformation blocks of different sizes, and is suitable for occasions such as FPGA-based HEVC encoders that require high throughput.
[0116] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.
Claims
1. A HEVC transform coefficient bit rate estimation device based on FPGA, comprising: Coefficient scanning module, buffer module, absolute value module, syntax element processing module, bit rate estimation module; The coefficient scanning module is connected to the buffer module and the syntax element processing module respectively, the buffer module is connected to the absolute value module, the absolute value module is connected to the syntax element processing module, and the syntax element processing module is connected to the bit rate estimation module; The bit rate estimation device inputs a transform block, i.e., a coefficient block after transformation and quantization, and the transform block size is 4×4, 8×8, 16×16, and 32×32, with four transform coefficients being inputted each time until a transform block of a complete size is completed; The coefficient scanning module includes: a counter, an output buffer module, a scanning sequence number module, 5 or calculation modules, and 4 comparison modules; the coefficient scanning module is used to scan the input coefficients to obtain the coordinate position of the last non-zero coefficient in the transformation block; The counter in the coefficient scanning module is connected to the output buffer module and the scanning module respectively, the input coefficient is connected to 5 OR calculation modules OR0~OR4, the scanning sequence module and the OR calculation module are connected to 4 comparison modules, and all the results are finally connected to the output buffer module; the counter module is used to record the order and number of each input data, the scanning sequence module converts the count value into the scanning order of the input data, and the output buffer module is used to temporarily store data; OR0~OR3 of the 5 OR calculation modules is used to calculate whether the current input coefficient is zero, and OR4 is used to calculate whether all input coefficients are zero; COMP0~COMP3 of the 4 comparators is used to calculate the coefficient with the largest coordinate among the four coefficients, and COMP4 is used to calculate the coefficient with the largest coordinate among the new and old coefficients; The buffer module is used to overcome the problem of inconsistency between the input order and the scanning order, temporarily store the input transformation data, and then output the data in the scanning order when outputting the data; The absolute value module is used to calculate the positive and negative signs and absolute values of the transform coefficients; The syntax element processing module is used to convert the position information and value information of the non-zero transform coefficients into specific syntax element information for subsequent bit rate estimation. The syntax element processing module is implemented by a state machine. The state machine is used to convert the non-zero coefficient position information and coefficient value information into syntax element information. The syntax element information includes: syntax element value syntax_value, syntax element model starting number syntax_id and syntax element model type syntax_type; The code rate estimation module includes: a binarization module, a register, a probability estimation table module, a local context model module, an accumulator, and a selector; and is used to estimate the code rate of each syntax element; The binarization module in the bit rate estimation module is connected to the register, and the register is divided into two paths, one path is connected to the selector for linear estimation, and the other path is connected to the local context model module and the probability estimation table module for table lookup estimation, the probability estimation table is connected to the selector, and the selector is connected to the accumulator; The binarization module converts the syntax elements into binary strings using a binarization method. The register is used to temporarily store the binary string and output each bit of the binary string. The local context model module is used to synchronize the state of the global context model. The probability estimation table module stores the bit rate table for table lookup to obtain the bit rate. The selector is used to select the bit rate estimated by table lookup or linear estimation. The accumulator accumulates the estimated bit rates of all syntax elements and outputs the final bit rate.
2. The HEVC transform coefficient bit rate estimation device based on FPGA according to claim 1, characterized in that: In the syntax element processing module, the state machine conversion process is specifically as follows: A1, initially in the IDLE state, if all coefficients are 0, then enter step A9, that is, enter the END state to end the bit rate estimation of the current transform block, if the coefficients are not all 0, then enter step A2 to start processing the non-zero coefficient position information; A2, enter the LAST_X_PREFIX and LAST_Y_PREFIX states in sequence, process the coordinate prefixes last_sig_coeff_x_prefix and last_sig_coeff_y_prefix, if the coordinate suffix is not 0, enter LAST_SUFFIX, process the coordinate suffixes last_sig_coeff_x_prefix and last_sig_coeff_y_prefix, if the coordinate suffix is 0, do not process the suffix, after processing the coordinates, enter step A3 to start processing the absolute value information of the coefficients; A3, enter the CSBF state, start processing from the sub-block where the last non-zero coefficient is located, if the csbf of the current coefficient sub-block is 0, it remains in the CSBF state and processes the next coefficient sub-block, if the current is 1 or it is the last sub-block, that is, csbf==1ORlast_cg==1, enter step A4, and further process the coefficient information of the current sub-block; A4, enter the SIG_COEFF_FLAG state, process the 16 transform coefficients in the sub-block in the scanning order, if the coefficient is 0, the sig_coeff_flag flag at the current position is 0, otherwise sig_coeff_flag is 1, and only the coefficients with sig_coeff_flag=1 are further processed in the future, if all the coefficients are 0 and in the last sub-block, that is, csbf==0&last_cg==1, then end all processing and enter step A9, otherwise enter step A5; A5. Enter the COEFF_G1_FLAG state. If the coefficient is greater than 1, coeff_abs_level_greater1_flag is 1. If it is equal to 1, it is 0. Only the first 8 non-zero coefficients are marked. The unmarked ones are 0 by default. A6. If the absolute value of the coefficient is less than or equal to 2, directly enter step A7. Otherwise, if the absolute value of the coefficient is greater than 2, first enter the COEFF_G2_FLAG state, only mark the first coefficient greater than 2, set coeff_abs_level_greater2_flag = 1, and the unmarked ones are defaulted to 0, and then enter step A7; A7, enter the COEFF_SIGN_FLAG state, mark the signs of all current coefficients, if the mark is a positive number, coeff_sign_flag is 0, if it is a negative number, it is 1, if there is a coefficient with a residual value, that is, coeff_remain! = 0, enter step A8, that is, enter the COEFF_ABS_REMAIN state, if all coefficients have no residual values and it is not the last sub-block, that is, last_cg == 0, then return to the CSBF state of step A3 to process the next sub-block, if all coefficients have no residual values and it is the last sub-block, that is, last_cg == 1, enter step A9; A8, enter the COEFF_ABS_REMAIN state, mark the residual value, and will not mark it as a zero residual value. If the current sub-block is not the last one, that is, last_cg==0, then after processing all coefficients, return to the CSBF state of step A3 to process the next sub-block. If the current sub-block is the last one, that is, last_cg==1, enter step A9; Among them, the numerical expression of the remaining value coeff_abs_remain is as follows: coeff_abs_remain=|V|-1-coeff_abs_level_greater1_flag-coeff_abs_level_greater2_flag, where V represents the coefficient value; A9, entering the END state, in which no operation or output is performed, and the device waits for all operations of other modules of the bit rate estimation device to be completed and then returns to the IDLE state to wait for the next transformation block.
3. The HEVC transform coefficient bit rate estimation device based on FPGA according to claim 1, characterized in that: In the grammatical element processing module, all grammatical elements are as follows: last_sig_coeff_x_prefix: There are 18 grammatical element models, estimated using a table lookup; last_sig_coeff_y_prefix: There are 18 grammatical element models, estimated using a lookup table; last_sig_coeff_x_suffix: no syntax element model, using linear estimation; last_sig_coeff_y_suffix: no grammatical element model, using linear estimation; coded_sub_block_flag: There are 4 syntax element models in total, using linear estimation; sig_coeff_flag: There are 42 syntax element models in total, using linear estimation; coeff_abs_g1_flag: There are 24 syntax element models in total, using linear estimation; coeff_abs_g2_flag: There are 8 syntax element models in total, using linear estimation; coeff_sign_flag: no syntax element model, use linear estimation; coeff_abs_level_remain: No syntax element model, use linear estimation.
4. A HEVC transform coefficient rate estimation method based on FPGA, applied to the rate estimation device, the specific steps are as follows: S1, the input end of the bit rate estimation device inputs a transform block, enters a coefficient scanning module to perform coefficient scanning, and obtains the last non-zero coefficient coordinate position; Four transform coefficients are input each time until a complete transform block is completed. The input coefficients are bitwise ORed to detect whether the current coefficient is 0. After receiving the enable signal en, the counter starts to record the number and order of the input transform coefficients. The scan number is generated according to the count value. The detection value and the corresponding scan number are spliced to obtain the comparison value, which is then compared with the comparison values of the remaining coefficients to select the largest one until the input ends. The scan number in the largest comparison value is taken out and converted into the last non-zero coefficient coordinates last_sig_x and last_sig_y. After scanning, the transform coefficients are generated according to the scan number to generate the address w_addr and the data itself w_coeff. All data will be temporarily stored in the output buffer module, and the final output data will be output by the output buffer module. Coefficient scanning methods include: Scan horizontally from right to left, scan vertically from bottom to top, and scan diagonally from bottom right to top left; Among them, after completing the input of a 4×4 sub-block, the CSBF of the current sub-block is calculated. If all 16 coefficients of the sub-block are 0, the CSBF is equal to 0, otherwise it is 1; S2, the coefficient w_coeff scanned by the coefficient scanning module is stored in the buffer module according to the address w_addr generated by the scanning sequence number, and the subsequent data is taken out from the buffer module according to the scanning sequence and output to the absolute value module, the absolute value of the coefficient is taken, and the amplitude and sign of the coefficient are obtained; The absolute value module reads the coefficient r_coeff from the buffer module by reading the address r_addr, and then outputs the transformation coefficient sign sign and the coefficient absolute value abs_coeff to the syntax element processing module; S3, the syntax element processing module converts the non-zero coefficient position information and the remaining coefficient value information into syntax element information; S4, a bit rate estimation module binarizes the syntax element obtained in step S3 to obtain a binary string bin, and performs bit rate estimation on the bin to obtain a bit rate of the syntax element; The syntax element value syntax_value is binarized to obtain a binary string bin. The syntax_id determines the binarization method. Fixed-length coding does not need to be converted in FPGA, but truncated Rice code will convert syntax elements into prefixes and suffixes. Then the code rate is estimated for each bit of the bin, including table lookup estimation and linear estimation, as follows: (1) Table lookup estimation: Take each bit of the bin, obtain the corresponding context model offset value in the register according to the bin length, add it to the syntax_id to get the index, use the index to get the probability state value and the most likely symbol MPM from the local context model module, use the probability state value to look up two long and short code rates from the code rate estimation table, then compare MPM with bin. If bin is equal to MPM, select the short code rate. If bin is not equal to MPM, select the long code rate. (2) Linear estimation: Get the bin length, then the code rate = bin length × 1; S5. The rate accumulator accumulates the rates of all syntax elements obtained in step S4 and rounds them off, outputting the total rate as the estimated rate of the current transform block, that is, obtaining a rate estimate of the transform block.
Citation Information
Cited By
Processor, chip, device and code rate estimation method
CN120956905A
Video encoder, video encoding method, electronic equipment, storage medium and computer program product
CN121691685A
