Code rate estimation method suitable for hardware implementation

Abstract

The invention provides a code rate estimation method suitable for hardware implementation. The code rate estimation method comprises the following steps: S1, optimizing a context probability model; s2, independently designing a bit processing unit (BPU) for updating and storing a context model value; and S3, carrying out hardware design on the entropy coding process realized by hardware. According to the invention, a'context modeling of binarized bin 'module in the CABAC entropy coding process is optimized, and an algorithm and a hardware pipeline structure are provided, so that the time for estimating the consumed bit digital rate by using an entropy coding algorithm is shortened, the area and the power consumption of a hardware product are reduced, hardware implementation is facilitated, and the cost is reduced. According to the method, the dependency that context model values need to be updated bin by bin in each bin of entropy coding in the rate distortion optimization (RDO) process is eliminated, hardware parallel pipeline is facilitated, meanwhile, the key path of updating the context model values bin by bin is reduced, the calculation complexity is reduced, and thus the hardware power consumption is reduced.

Description

Technical Field

[0001] The present invention relates to the field of digital video compression, and in particular to a code rate estimation method suitable for hardware implementation. Background Art

[0002] The goal of video compression is to reduce the compression bit rate of the video stream as much as possible while ensuring the video quality. However, the relationship between the bit rate of the encoded output and the distortion after compression is mutually restrictive and contradictory. For example, reducing the bit rate will often increase the distortion of the video. On the contrary, if you want to obtain better video quality, you will increase the encoding bit rate of the video. Therefore, encoders often use encoding parameter optimization methods based on rate-distortion theory to balance video quality and compression bit rate.

[0003] Mode decision based on rate-distortion optimization is to calculate the Lagrangian cost including distortion and bit rate, and select the mode with the minimum cost as the final coding mode. The cost formula is as follows:

[0004] in, Indicates the distortion of different coding modes. represents the Lagrangian factor, Indicates the number of bits consumed by different encoding modes.

[0005] The bit rate in the rate-distortion optimization formula is generally calculated using the bit rate estimation method. There are two main approaches in the existing results: one is based on statistics, which fits the output bit rate according to the mathematical characteristics of the coefficient matrix without going through the specific entropy coding link; the other is based on CABAC, which designs an algorithm to optimize the intermediate link of entropy coding, or reasonably expands it in hardware to reduce the dependence on context data. [1] .literature [2] It is believed that self-information is related to the amplitude and position of coefficients in the coefficient matrix, and it is proposed to use self-information to fit the bit rate. However, a series of complex operations such as exponential functions are used when calculating the probability density function, which is not conducive to hardware implementation. [3] The bit rate estimation is accelerated by simplifying the CABAC process. It uses linear regression to replace the arithmetic coding step in the CABAC process. After this simplification, the bit rate estimation only needs binarization and context index update. However, this method will bring a large error and ultimately reduce the coding efficiency.

[0006] The rate estimation is calculated after quantization. In hardware design, the rate estimation is calculated in as few cycles as possible in order to avoid introducing extra clock cycles. In the process of rate distortion optimization (RDO), the most accurate calculation of the rate estimation is the number of bits of the actual CABAC entropy coding result. However, the complex processing flow of CABAC, the context probability model, the large amount of data dependence in arithmetic coding, and the difficulty in improving parallelism make it difficult to use the actual CABAC entropy coding as the calculation method for rate estimation on hardware. Almost all hardware manufacturers simplify the calculation process of rate estimation when designing hardware encoders.

[0007] The encoder of the SVAC3.0 reference software uses a rate-distortion optimization formula to determine different coding parameters. When calculating the bit rate consumed by different coding modes, a complete CABAC entropy coding is performed to calculate the bit rate, including binarization, building a context probability model, and binary arithmetic coding. Specifically, the complete context-adaptive binary arithmetic coding (CABAC entropy coding) process of a tree coding unit (CTU) is as follows: the tree coding unit (CTU) coding starts, first, the syntax element to be encoded is determined, and it is binarized to obtain a bin (a binary sequence value after the syntax element is binarized): the syntax element to be encoded is binarized to obtain a series of binarized bin values; then, the binarized bin is context modeled: a context probability model corresponding to the syntax element is constructed, including reloading, updating, and saving of the context probability model; finally, all bins are entropy encoded: the bin value and the context model corresponding to the syntax element are input to the binary arithmetic encoder engine for entropy encoding. If the current tree coding unit (CTU) coding is completed, the tree coding unit (CTU) coding is completed and the code stream is output. If the current tree coding unit (CTU) coding is not completed (i.e., no), the tree coding unit (CTU) coding is returned to start again.

[0008] In the process of building the context probability model, the context probability model needs to be updated. The commonly used updating method is to update the context probability model bin by bin. The bins using the same context probability model need to wait until the context probability model is updated to obtain their correct context probability model. This is a serial process. The context probability model used to encode the current bin needs to wait until the previous bin is encoded and its context probability model is updated.

[0009] Therefore, the main problems existing in the prior art and the difficulty in solving the problem are: this context probability model update method is data-dependent, and the calculation is complex and time-consuming, which introduces a long critical path and is not conducive to hardware parallel pipelining and hardware implementation.

[0010] The significance of solving the existing technical problems is to find a bit rate estimation method suitable for hardware implementation, reduce the area and power consumption of hardware products, and facilitate hardware implementation. Summary of the invention

[0011] The present invention provides a bit rate estimation method suitable for hardware implementation, optimizes the "context modeling of bins after binarization" module in the above-mentioned CABAC entropy coding process, and proposes an algorithm and a hardware pipeline structure to shorten the time of the process of estimating the bit rate consumed by using the entropy coding algorithm, reduce the area and power consumption of hardware products, and facilitate hardware implementation.

[0012] The technical solution of the present invention is as follows: The bit rate estimation method suitable for hardware implementation of the present invention comprises the following steps: S1. optimizing a context probability model; S2. separately designing a bit processing unit (BPU) for updating and saving context model values; S3. hardware-designing an entropy coding process implemented by hardware.

[0013] Optionally, in the above-mentioned bit rate estimation method suitable for hardware implementation, in step S1, an optimization algorithm is designed for the data dependency of updating the context probability model bin by bin, and the context probability model is updated after encoding a tree coding unit (CTU). The updating steps of the optimized context probability model include: 1) starting from an image, initializing the context model value (Context Model value) of each grammatical element that needs to be entropy encoded, and before performing rate-distortion optimization (RDO) on the first tree coding unit (CTU), saving the initialized Context Model value for use when performing rate-distortion optimization (RDO) on the first tree coding unit (CTU); 2) In the process of rate-distortion optimization (RDO) of the tree coding unit (CTU), the sequence number ContextID of each binarized bin value of the syntax element and its corresponding context model is saved; when the bit rate needs to be estimated, the ContextID and the binarized bin value to be encoded are obtained, and the ContextID is added to the context model value of the first syntax element to be entropy encoded. At this time, the context model value is no longer updated bin by bin, and the context model corresponding to the bin can be located by relying on the ContextID. The context model value and the binarized bin value are sent to the entropy encoder, and the consumed bit rate is obtained by querying the bit rate table; 3) for the tree coding unit ( After the rate-distortion optimization (RDO) of a tree coding unit (CTU) is completed, a complete context model consistency encoding is performed on the tree coding unit (CTU). The value of the context model will be updated bin by bin. Only after the entropy encoding of the tree coding unit (CTU) is completed, the value of the context model is saved once for the next tree coding unit (CTU) to perform rate-distortion optimization (RDO), that is, the next tree coding unit (CTU) performs RDO; 4) Repeat steps 2)~3) and perform rate-distortion optimization (RDO) on each tree coding unit (CTU) until an image is completed, that is, all the tree coding units (CTUs) of the entire image have completed rate-distortion optimization (RDO).

[0014] Optionally, in the above-mentioned rate estimation method suitable for hardware implementation, in step S2, when the context model of the tree coding unit (CTU) is continuously and consistently encoded, a bit processing unit (BPU) is separately designed to update and save the context model value.

[0015] Optionally, in the above-mentioned bit rate estimation method suitable for hardware implementation, in step S3, the bit rate estimation method suitable for hardware implementation is embedded into the entire entropy coding process to form a new entropy coding hardware architecture diagram.

[0016] Optionally, in the above-mentioned bit rate estimation method suitable for hardware implementation, the entropy coding hardware architecture diagram includes from input data to output bit stream: 1) Preparation of syntax elements and initialization of context model module including using state machine to control the sequential logic of input of each syntax element: 2) Binarization module obtains binary string bin from input syntax elements through specific encoding method, and designs to use four binarization calculation units for parallel operation, and then outputs the binarization result in parallel to binary arithmetic coding operation unit; 3) Binary arithmetic coding operation unit module, designs four parallel pipelines, and independently performs binary arithmetic coding on each bin and its corresponding ctx; 4) A separate bit processing unit (BPU) is designed to update and save context model (Context Model) value, which is conducive to hardware parallel pipeline encoding of multiple tree coding units (CTU).

[0017] According to the technical solution of the present invention, the beneficial effects produced are: The present invention aims at the SVAC3.0 coding end using the CABAC entropy coding algorithm to estimate the bit rate consumed by the coding parameters in the process of rate distortion optimization (RDO), and proposes a code rate estimation algorithm suitable for hardware implementation. The algorithm uses the context model sequence number (ContextID) of the syntax element to look up the table to obtain the context model value required for encoding each bin of the current syntax element binarization, thereby eliminating the dependency of updating the context model value bin by bin for entropy coding each bin in the process of rate distortion optimization (RDO), which is beneficial to hardware parallel pipelining. At the same time, the key path of updating the context model value bin by bin is reduced, the calculation complexity is reduced, and thus the hardware power consumption is reduced.

[0018] In order to better understand and illustrate the concept, working principle and effect of the present invention, the present invention is described in detail below through specific embodiments in conjunction with the accompanying drawings: BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art are briefly introduced below.

[0020] Figure 1 is a flow chart of a bit rate estimation method suitable for hardware implementation of the present invention; Figure 2 It is the update flow chart of the optimized context probability model; Figure 3 It is a diagram of the hardware architecture of entropy coding during the rate-distortion optimization (RDO) process; Figure 4 A hardware parallel pipeline structure diagram of a four-way parallel two-stage pipeline is designed for bin binary arithmetic coding; Figure 5 It is a hardware two-way parallel two-level pipeline structure diagram of tree coding unit (CTU) level parallelism. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical method and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific examples. These examples are only illustrative and not limiting of the present invention.

[0022] The present invention proposes a rate estimation algorithm suitable for hardware implementation for estimating the bit rate consumed by coding parameters by using the CABAC entropy coding algorithm in the rate-distortion optimization (RDO) process of the SVAC3.0 coding end.

[0023] The algorithm uses the context model number (ContextID) of the syntax element to look up the table to obtain the context model value required for encoding each bin of the current syntax element, eliminating the dependency on updating the context model value bin by bin for entropy encoding of each bin during the rate-distortion optimization (RDO) process, which is beneficial to hardware parallel pipeline. At the same time, it reduces the critical path of updating the context model value bin by bin, reduces the computational complexity, and thus reduces hardware power consumption.

[0024] After the current CTU completes the rate-distortion optimization (RDO), in order to maintain the continuity of the context model, the current CTU needs to be fully entropy-coded again. This entropy coding needs to update the context model value bin by bin and save it for the next CTU to use when performing rate-distortion optimization (RDO). The next CTU relies on the context model value provided by the previous CTU when performing rate-distortion optimization (RDO), which is not conducive to parallel pipelining between multi-core and multi-CTU. Therefore, a separate bit processing unit (BPU) is designed to update and save the context model value, which is conducive to hardware parallel pipeline encoding of multiple CTUs, greatly saving chip area and clock cycles.

[0025] The bit rate estimation method suitable for hardware implementation of the present invention comprises the following steps: S1. Optimizing the contextual probability model.

[0026] In view of the data dependency of updating the context probability model bin by bin, an optimization algorithm is designed to update the context probability model after encoding a tree coding unit (CTU), which reduces the data dependency in the hardware implementation process. The update steps of the optimized context probability model are as follows: Figure 2 As shown: 1) Starting from an image, initialize the context model value of each syntax element that needs to be entropy coded, and save the initialized context model value before performing rate distortion optimization (RDO) on the first tree coding unit (CTU) for use when performing rate distortion optimization (RDO) on the first tree coding unit (CTU); 2) During the rate-distortion optimization (RDO) process of the tree coding unit (CTU), save the data pair (bin, ContextID) bin by bin, that is, save each binarized bin value of the syntax element and the serial number ContextID of the corresponding context model for later use. When the bit rate needs to be estimated, obtain the ContextID and the binarized bin value to be encoded. Add the ContextID to the context model value of the first syntax element to be entropy encoded (generally the skip_flag syntax element), because the context model value is stored in the memory in address order, so the context model value corresponding to the binarized bin of the syntax element to be encoded can be obtained. At this time, the context model value is no longer updated bin by bin, and the context model corresponding to the bin can be located by relying on the ContextID, which is simple and efficient. Send the context model value and the binarized bin value to the entropy encoder, and obtain the consumed bit rate by querying the bit rate table, that is, complete the rate estimation of the binarized bin of the syntax element to be encoded;

[0027] 3) After the rate-distortion optimization (RDO) of the tree coding unit (CTU) is completed, in order to ensure the continuity and consistency of the context model value, the tree coding unit (CTU) needs to be fully CABAC entropy encoded again, that is, the context model (Context Model) consistency encoding. This time the context model value will be updated bin by bin, that is, the context model value will be updated bin by bin. Only after the entropy encoding of the tree coding unit (CTU) is completed, the context model (Context Model) value is saved once, that is, the context model (Context Model) value is saved for the next tree coding unit (CTU) to perform rate-distortion optimization (RDO), that is, for the next tree coding unit (CTU) to perform RDO;

[0028] 4) Repeat steps 2) to 3) and perform rate distortion optimization (RDO) on each tree coding unit (CTU) until an image is completed, that is, all tree coding units (CTUs) of the entire image have completed rate distortion optimization (RDO).

[0029] The above is the number of bits consumed by conventional encoding. Finally, add the number of bits consumed by bypass encoding to get the final estimated bit rate consumption; S2. Design a separate bit processing unit (BPU) to update and save the context model value.

[0030] When performing rate-distortion optimization (RDO) on a tree coding unit (CTU), the dependency on updating the context model bin by bin has been removed internally, which can better implement bin-level parallel pipelining. At the same time, when performing continuous and consistent coding on the context model of a tree coding unit (CTU), a separate bit processing unit (BPU) is designed to update and save the context model value, which is conducive to the hardware parallel pipelining of multiple tree coding units (CTUs).

[0031] S3. Perform hardware design for the hardware-implemented entropy coding process.

[0032] In view of the problem that the use of precise CABAC for bit rate estimation in the rate-distortion optimization (RDO) process leads to a large amount of data dependence and difficulty in improving hardware parallelism, the present invention designs a rate-distortion optimization bit rate estimation method suitable for hardware implementation and embeds it into the entire entropy coding process to form a new entropy coding hardware architecture diagram. Figure 3 This is the entropy coding hardware architecture diagram during the rate-distortion optimization (RDO) process, such as Figure 3 As shown, the code stream from input data to output includes:

[0033] 1) Prepare syntax elements and initialize context model module, including using state machine to control the sequential logic of input of each syntax element: Initialize context model at the beginning of frame -> Encode sample adaptive compensation (SAO) information -> Encode tree coding unit (CTU) block information, including coding unit (CU), prediction unit (PU), transform unit (TU) information -> Loop through all tree coding units (CTU) -> Tree coding unit (CTU) encoding ends. And prepare syntax elements to be encoded, including SAO information, CU block information, PU block information, TU block information.

[0034] 2) The binarization module obtains a binary string bin by a specific encoding method for the input syntax elements. The design uses four binarization calculation units in parallel, and then outputs the binarization results in parallel to the binary arithmetic coding operation unit.

[0035] 3) Binary arithmetic coding operation unit module. At this time, the dependency of updating the context model bin by bin has been released, that is, the ctx between different bins is independent and can be parallelized. Therefore, four parallel pipelines are also designed to perform binary arithmetic coding on each bin and its corresponding ctx independently. A four-way parallel two-stage pipeline hardware parallel pipeline structure is designed for bin binary arithmetic coding. Figure 4 shown.

[0036] I) Four pipelines, pipeline 0: binarization calculation core 0 and binary arithmetic coding calculation core 0, pipeline 1: binarization calculation core 1 and binary arithmetic coding calculation core 1, pipeline 2: binarization calculation core 2 and binary arithmetic coding calculation core 2, pipeline 3: binarization calculation core 3 and binary arithmetic coding calculation core 3; II) On the same pipeline, the binarization calculation core and the binary arithmetic coding calculation core modules are strongly related. Only the results of the previous module can be used by the next module. Therefore, there is a strong dependence on data and random execution is not possible. III) There is no data dependency between the four pipelines, so the four pipelines can be executed in parallel. At the same time, all modules are executing and the data blocks they execute are different. For example, at the same time, "binary arithmetic coding calculation core 0" and "binarization calculation core 1" are executing at the same time, and the data they process are different; IV) After the above three steps, the binary sequence value Bin can be continuously fed to the four pipelines, thereby accelerating hardware encoding and improving throughput; 4) In order to further improve the hardware throughput and maximize the degree of parallelism, a separate bit processing unit (BPU) is designed to update and save the context model value, which is conducive to hardware parallel pipeline encoding of multiple tree coding units (CTU). The hardware two-way parallel two-level pipeline structure of tree coding unit (CTU) level parallelism is as follows Figure 5 shown.

[0037] I) Two pipelines, pipeline 0: perform rate distortion optimization (RDO) on CTU0 to estimate the bit rate and estimate the bit rate consumption; BPU0 unit: perform entropy coding on CTU0 again, and update and save the context model value Bin by Bin. Pipeline 1: perform rate distortion optimization (RDO) on CTU1 to estimate the bit rate and estimate the bit rate consumption; BPU1 unit: perform entropy coding on CTU1 again, and update and save the context model value Bin by Bin;

[0038] II) On the same pipeline, the calculation core that performs rate-distortion optimization (RDO) and bitrate estimation on CTU0 and the calculation core of the BPU0 unit: entropy encoding is performed on CTU0 again, and the context model value is updated and saved Bin by Bin. These two calculation core modules are strongly associated. Only the results of the previous module can be used by the subsequent module. Therefore, there is a strong dependence on data and they cannot be executed randomly. III) There is no data dependency between the two pipelines, so the two pipelines can be executed in parallel. At the same time, all modules are executing and the data blocks they execute are different. For example, at the same time, "BPU0 unit calculation core: entropy encode CTU0 again, and update the saved context model value Bin by Bin" and "rate distortion optimization (RDO) bit rate estimation calculation core for CTU1" are executed at the same time, and the processed data are different; After the above three steps, many tree coding units (CTUs) of an image can be continuously fed to these two pipelines, thereby accelerating hardware encoding and improving throughput; The above description is the best embodiment based on the concept and working principle of the invention. The above embodiment should not be understood as limiting the protection scope of the present claims, and other implementations and combinations of implementations according to the concept of the present invention belong to the protection scope of the present invention.

[0039] References: [1] Fan Yibo. Principles of Video Codec Chip Design[M]. Beijing: Science Press, 2022. [2] ZHAO [3] LI W, REN P. Fast CABAC rate estimation for HEVC mode decision[C]. 2015 IEEE International Conference on Multimedia Big Data, Beijing, 2015: 171-175.

Claims

1. A bit rate estimation method suitable for hardware implementation, characterized in that: The following steps are involved: S1. Optimize contextual probability model; S2. A separate bit processing unit (BPU) is designed to update and save the context model value; and S3. Perform hardware design for the hardware-implemented entropy coding process.

2. The bit rate estimation method suitable for hardware implementation according to claim 1, characterized in that: In step S1, an optimization algorithm is designed for data dependency of updating the context probability model bin by bin, and the context probability model is updated after encoding a tree coding unit (CTU). The updating steps of the optimized context probability model include: 1) Starting from an image, initialize the context model value (ContextModel value) of each syntax element that needs to be entropy encoded, and before performing rate-distortion optimization (RDO) on the first tree coding unit (CTU), save the initialized Context Model value for use when performing rate-distortion optimization (RDO) on the first tree coding unit (CTU); 2) In the process of performing rate-distortion optimization (RDO) on the tree coding unit (CTU), each binarized bin value of the syntax element and the serial number ContextID of its corresponding context model are saved; when the code rate needs to be estimated, the ContextID and the binarized bin value to be encoded are obtained, and the ContextID is added to the context model value of the first syntax element to be entropy encoded. At this time, the context model value is no longer updated bin by bin, and the context model corresponding to the bin can be located by relying on the ContextID, and the context model value and the binarized bin value are sent to the entropy encoder, and the consumed bit rate is obtained by querying the code rate table; 3) After the rate-distortion optimization (RDO) is performed on the tree coding unit (CTU), a complete context model consistency encoding is performed on the tree coding unit (CTU). The value of the context model will be updated bin by bin. Only after the entropy encoding of the tree coding unit (CTU) is completed, the value of the context model is saved once for the next tree coding unit (CTU) to perform rate-distortion optimization (RDO); Repeat steps 2) to 3) to perform rate distortion optimization (RDO) on each tree coding unit (CTU) until an image is completed, that is, all tree coding units (CTUs) of the entire image have completed rate distortion optimization (RDO).

3. The bit rate estimation method suitable for hardware implementation according to claim 2, characterized in that: In step S2, when the context model of the tree coding unit (CTU) is continuously and consistently encoded, a bit processing unit (BPU) is separately designed to update and save the context model value.

4. The bit rate estimation method suitable for hardware implementation according to claim 1, characterized in that: In step S3, the bit rate estimation method suitable for hardware implementation is embedded into the entire entropy coding process to form a new entropy coding hardware architecture diagram.

5. The bit rate estimation method suitable for hardware implementation according to claim 4, characterized in that: The entropy coding hardware architecture diagram includes: The preparation of syntax elements and initialization of context model modules include using state machines to control the sequential logic of the input of each syntax element: The binarization module obtains a binary string bin by a specific encoding method for the input syntax elements. The design uses four binarization calculation units in parallel, and then outputs the binarization results in parallel to the binary arithmetic coding operation unit. The binary arithmetic coding operation unit module designs four parallel pipelines to independently perform binary arithmetic coding on each bin and its corresponding ctx; A separate bit processing unit (BPU) is designed to update and save context model values, which is conducive to hardware parallel pipeline encoding of multiple tree coding units (CTUs).

Images