A method and apparatus for bitrate control at the I-frame image block level

Abstract

This invention discloses a bitrate control method at the image block level for I-frames. Step S1: Predict the intra-frame coding mode for each image block to be encoded, obtaining candidate intra-frame coding modes and their prediction costs. Step S2: Select the minimum prediction cost to characterize the intra-frame coding complexity of the image block to be encoded; use the sum of the intra-frame coding complexities of all image blocks within the video frame as the intra-frame coding complexity of the video frame. Step S3: Calculate the target number of coding bits for each image block to be encoded within the I-frame. Step S4: Calculate the Lagrange multiplier for the image block to be encoded within the I-frame. Step S5: Perform video coding on the image block to be encoded, and then adjust the target number of coding bits for the next image block to be encoded within the I-frame. This application has low hardware overhead and low implementation cost; it has high computational accuracy and a more precise image block-level bitrate control process.

Description

Technical Field

[0001] This invention relates to a digital video coding technique, and more particularly to a bitrate control method at the I-frame image block level suitable for hardware implementation. Background Technology

[0002] Video coding is a technique that compresses redundant components in video images and represents video information using as little data as possible. Common video coding standards include HEVC (High Efficiency Video Coding, also known as H.265) and AVC (Advanced Video Coding, also known as H.264). Video coding technology uses image blocks as the most basic coding unit. For example, in HEVC, the basic coding unit is the CU (Coding Unit). A CU can be an image block of 64 pixels × 64 pixels, 32 pixels × 32 pixels, 16 pixels × 16 pixels, or 8 pixels × 8 pixels. The 64-pixel × 64-pixel image block is also called the LCU (Largest Coding Unit).

[0003] To remove information redundancy in the spatial and temporal domains, input video frames are encoded using intra-frame coding and inter-frame coding techniques. Encoded video frames are generally divided into I-frames (using only intra-frame coding) and P-frames and B-frames (using a hybrid approach). I-frames use only intra-frame coding, employing only the information of the current frame and not referencing information from other encoded frames. All image blocks within an I-frame are encoded as intra-coded blocks. P-frames and B-frames use a hybrid approach, employing both intra-frame and inter-frame coding. They not only use the information of the current frame but also reference information from one or more other encoded frames. Image blocks within P-frames and B-frames can be encoded as either intra-coded or inter-coded blocks. For each image block within a P-frame or B-frame, if intra-frame coding results in a lower rate-distortion cost, the image block will be encoded as an intra-coded block; if inter-frame coding results in a lower rate-distortion cost, the image block will be encoded as an inter-coded block. For each video frame, the type of encoded frame is determined by the image group parameters set by the user. Figure 1 It demonstrates a common image group structure, which includes I-frames, P-frames, or B-frames.

[0004] To maximize video compression, common video coding algorithms are often lossy compression algorithms, such as AVC and HEVC. This results in a difference between the reconstructed video and the original video; that is, the reconstructed video will contain distortion. For lossy compression algorithms, their coding performance needs to be measured by both the coding bitrate and the distortion introduced by the coding. In the video coding process, the coding bitrate and coding distortion are mutually restrictive and contradictory. For example, reducing the coding bitrate often increases coding distortion, while reducing coding distortion often increases the coding bitrate. Rate Distortion Optimization (RDO) aims to minimize coding distortion at a given coding bitrate, or minimize the coding bitrate at a given coding distortion. Currently, most common video coding algorithms use rate distortion optimization methods based on the Lagrange multiplier λ. Assuming D is the coding distortion, R is the coding bitrate, and J is the coding rate distortion cost (RD Cost), J = D + λ × R. Rate-distortion optimization based on the Lagrange multiplier λ can be expressed as min(J), where min() represents minimizing the value. In video coding, each coding mode has its corresponding coding distortion D and coding bit rate R, allowing the calculation of the coding rate-distortion cost J for each mode. During encoding, selecting the coding mode with the minimum rate-distortion cost J yields optimal coding performance. The process by which the encoder selects the optimal coding mode for each image block to be encoded using rate-distortion optimization is simply called mode decision for the image block to be encoded.

[0005] In real-world scenarios, the channel bandwidth (channel capacity) used to transmit compressed video is limited. If the encoding bit rate of the compressed video is too high, exceeding the channel bandwidth capacity, it will cause video transmission congestion or even packet loss. If the encoding bit rate of the compressed video is too low, the channel bandwidth will not be fully utilized, and higher video quality cannot be obtained. Therefore, it is necessary to use bitrate control technology to control the output bitrate of the video encoder to match it with the channel bandwidth capacity.

[0006] Rate control technology adjusts the encoding parameters of the video encoder to make the output bitrate of the video encoder equal to a pre-set target bitrate, while minimizing encoding distortion to improve video encoding quality. Common rate control algorithms generally achieve the task through two stages: target bit allocation and target bit control.

[0007] In common rate control algorithms, target bit allocation typically occurs at three levels: GOP (Group of Pictures, a set of time-series consecutive video frames) level, video frame level, and image block level. After target bit allocation at the GOP and video frame levels, the target number of coded bits for the current video frame to be encoded is determined. The next step is to perform target bit allocation at the image block level, allocating the target number of coded bits to each image block based on its bit allocation weight within the video frame. Then, in the target bit control stage, the corresponding Lagrange multiplier λ is calculated based on the target number of coded bits for each image block, and used in the image block encoding process.

[0008] Because video encoding algorithms involve a large amount of computation, using application-specific integrated circuits (ASICs) to accelerate the video encoding process in hardware has become a common industry practice in order to improve video encoding speed and achieve real-time encoding and transmission. These ASICs that accelerate the video encoding process in hardware are generally referred to as hardware video encoders.

[0009] In hardware video encoders, bitrate control at the image block level within an I-frame has always been a challenge. The main reason is that accurate bitrate control at the image block level requires a dedicated preprocessing stage before I-frame encoding to evaluate the intra-frame coding complexity of each image block, calculate the bit allocation weight for each block, and then determine the target number of coded bits and the corresponding Lagrange multiplier λ for each block. However, in hardware video encoders, adding a dedicated preprocessing stage is not only costly but also increases video encoding latency. Therefore, it is necessary to design a bitrate control method at the image block level for I-frames suitable for hardware video encoders. This method should accurately evaluate the intra-frame coding complexity of image blocks within an I-frame without adding a dedicated preprocessing stage, and calculate the target number of coded bits and the corresponding Lagrange multiplier λ for each block. This would make the actual number of coded bits for each block closer to the target number, resulting in more precise bitrate control of the I-frame. Summary of the Invention

[0010] The technical problem to be solved by this invention is to propose a bitrate control method at the I-frame image block level suitable for hardware video encoders.

[0011] To address the aforementioned technical problems, this invention discloses a bitrate control method at the image block level for I-frames, comprising the following steps: Step S1: Predict the intra-frame coding mode for each image block to be encoded, obtaining candidate intra-frame coding modes and their prediction costs for that image block. Step S2: Filter the prediction costs corresponding to each candidate intra-frame coding mode for the image block to be encoded, selecting the minimum prediction cost to characterize the intra-frame coding complexity of that image block; then, the sum of the intra-frame coding complexities of all image blocks within the video frame is taken as the intra-frame coding complexity of the video frame. Step S3: Calculate the target coding bit count for each image block to be encoded within the I-frame based on the ratio of the intra-frame coding complexity of the image blocks to be encoded within the I-frame to the total intra-frame coding complexity of the I-frame, and in conjunction with the target coding bit count of the I-frame. Step S4: Based on the target number of encoded bits and intra-frame coding complexity of the image block to be encoded within the I-frame to be encoded, and combined with the actual number of encoded bits, Lagrange multipliers, and intra-frame coding complexity of the corresponding image block within the previous encoded I-frame, calculate the Lagrange multiplier of the image block to be encoded within the I-frame to be encoded. Step S5: Perform video coding on the image block to be encoded within the I-frame to be encoded based on the Lagrange multiplier of the image block to be encoded, and obtain its actual number of encoded bits; then adjust the target number of encoded bits for the next image block to be encoded within the I-frame to be encoded based on the actual number of encoded bits of the encoded image block. Specifically, the intra-frame coding complexity of each image block within the I-frame to be encoded reuses the intra-frame coding complexity of the image block at the same position within the previous encoded video frame, and the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous encoded video frame.

[0012] Further, in step S1, the prediction cost corresponding to each intra-frame coding mode is calculated for each image block to be encoded, and one or more intra-frame coding modes with the lowest prediction cost are selected as intra-frame coding mode candidates; the prediction cost refers to the coding rate distortion cost.

[0013] Furthermore, step S2 calculates at least the intra-frame coding complexity at the image block level and video frame level of the preceding video frame for each I-frame.

[0014] Furthermore, step S3 is defined by the formula: Find; where T blk_curr ω represents the target number of coded bits for the current image block within the current I-frame to be encoded. blk_curr ω represents the intra-frame coding complexity of the current image block to be encoded within the current I-frame. pic T represents the intra-frame coding complexity of the current I-frame to be encoded. Pic Indicates the target number of coding bits for the current I-frame to be encoded.H This indicates the estimated number of bits required for the header information of the current I-frame to be encoded.

[0015] Furthermore, step S4 is derived from formula five: Find; where λ new f represents the Lagrange multiplier of the image block to be encoded within the I-frame to be encoded; -1 It is the inverse function of the first function f; I(n) new This represents the actual number of encoded bits for the image block within the I-frame to be encoded, replaced by the target number of encoded bits for the same image block within the I-frame; I(m) old This represents the actual number of encoded bits for the same image block within the previous encoded I-frame; g is the second function; ω n-1 ω represents the intra-frame coding complexity of image blocks at the same position within the (n-1)th frame; m λ represents the intra-frame coding complexity of image blocks at the same position within the m-th frame; old This represents the Lagrange multiplier of the same image block within the previous encoded I-frame.

[0016] Furthermore, the first function f is given by Formula 1: Formula 1 states that when different image block-level Lagrange multipliers are used to perform intra-frame coding on the same image block, the ratio of the actual number of coded bits obtained after coding is related to the ratio of the image block-level Lagrange multipliers. This relationship is represented by the first function f.

[0017] Furthermore, the second function g is derived from Formula 2: Get; ω n Formula 2 represents the intra-frame coding complexity of image blocks at the same position within the nth frame. Formula 2 states that: when using the same image block-level Lagrange multipliers to perform intra-frame coding on image blocks at the same position within different video frames, the ratio of the actual number of encoded bits obtained after coding is related to the ratio of the intra-frame coding complexity of the image blocks. This relationship is represented by the second function g.

[0018] Furthermore, based on Formula 1 and Formula 2, Formula 3 is derived: Further derivation yields Formula 4: Where, I(n) new The number of target coded bits per I-frame to be encoded is used as the replacement; ω n The intra-frame coding complexity ω of the image block to be encoded at the same position within the previous encoded video frame of the I-frame to be encoded. n-1 By making substitutions, we obtain Formula 5.

[0019] Further, step S5 is defined by the formula: Find; where T blk_next T represents the target number of coded bits for the next image block to be encoded within the current I-frame to be encoded. Pic Indicates the target number of coding bits for the current I-frame to be encoded. H Coded represents the estimated number of bits required for the header information of the current I-frame to be encoded. Pic ∑ represents the actual number of encoded bits in the current I-frame to be encoded. {all not coded blks} ω blk ω represents the sum of the intra-frame coding complexities of all uncoded image blocks within the current I-frame to be encoded. blk_next This represents the intra-frame coding complexity of the next image block to be encoded within the current I-frame.

[0020] This application also discloses an I-frame image block-level rate control device, including an intra-frame coding mode prediction module, an intra-frame coding complexity calculation module, an image block bit allocation module, an image block bit control module, and an image block coding module. The intra-frame coding mode prediction module predicts the intra-frame coding mode for each image block to be encoded, obtaining candidate intra-frame coding modes and their prediction costs for that image block. The intra-frame coding complexity calculation module filters the prediction costs corresponding to each candidate intra-frame coding mode for the image block to be encoded, selecting the minimum prediction cost to characterize the intra-frame coding complexity of that image block; then, the sum of the intra-frame coding complexities of all image blocks within the video frame is taken as the intra-frame coding complexity of the video frame. The image block bit allocation module is used to calculate the target number of encoded bits for each image block within the I-frame to be encoded before encoding, based on the ratio of the intra-frame coding complexity of the image blocks to be encoded within the I-frame to the intra-frame coding complexity of the I-frame itself, and in conjunction with the target number of encoded bits for the I-frame. The image block bit allocation module is also used to adjust the target number of encoded bits for the next image block within the I-frame to be encoded during the encoding process, based on the actual number of encoded bits of the already encoded image blocks. The image block bit control module is used to calculate the Lagrange multiplier for the image block to be encoded within the I-frame, based on the target number of encoded bits and the intra-frame coding complexity of the image blocks to be encoded within the I-frame, combined with the actual number of encoded bits, Lagrange multipliers, and intra-frame coding complexity of the corresponding image blocks within the previous encoded I-frame. The image block encoding module is used to perform video encoding on the image block to be encoded based on the Lagrange multipliers of the image blocks to be encoded within the I-frame to be encoded, and to obtain its actual number of encoded bits. Specifically, the intra-frame encoding complexity of each image block within the I-frame to be encoded reuses the intra-frame encoding complexity of the image block at the same position within the previous encoded video frame, and the intra-frame encoding complexity of the I-frame to be encoded reuses the intra-frame encoding complexity of the previous encoded video frame.

[0021] The technical advantages achieved by this invention are: no additional image preprocessing stage is required, resulting in low hardware overhead and low implementation cost; high calculation accuracy makes the actual number of encoded bits of the image block closer to the target number of encoded bits, and makes the image block-level bit rate control process of I-frame more precise. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a common image group structure.

[0023] Figure 2 This is a flowchart illustrating the image block-level bitrate control method for I-frames proposed in this application.

[0024] Figure 3This is a schematic diagram of the same image block within the m-th frame, the (n-1)-th frame, and the n-th frame.

[0025] Figure 4 This is a schematic diagram of the image block-level bitrate control device for I-frames proposed in this application.

[0026] The following are the labels in the figure: 1 is the intra-frame coding mode prediction module, 2 is the intra-frame coding complexity calculation module, 3 is the image block bit allocation module, 4 is the image block bit control module, and 5 is the image block coding module. Detailed Implementation

[0027] Please see Figure 2 The image block-level bitrate control method for I-frames suitable for hardware video encoders proposed in this application includes the following steps.

[0028] Step S1: Predict the intra-frame coding mode for each image block to be encoded, obtaining candidate intra-frame coding modes and their prediction costs. Specifically, calculate the prediction cost for each intra-frame coding mode for each image block to be encoded, and select one or more intra-frame coding modes with the lowest prediction cost as candidate intra-frame coding modes. The prediction cost refers to the coding rate distortion cost J, which has relatively low computational cost and low accuracy, J = D + λ × R. Where D is the coding distortion, generally using the SAD (Sum of Absolute Difference) algorithm or the SATD (Sum of Absolute Transformed Difference) algorithm; R is the coding bit rate, generally using a simplified bit rate estimation algorithm, such as the Exponential-Golomb coding algorithm; and generally no transformation and quantization operations are introduced during the calculation process. This step is a step in the existing video coding process, and intra-frame coding mode prediction is performed for all image blocks of I-frames, P-frames, and B-frames.

[0029] Both coding mode prediction and coding mode rate-distortion optimization employ rate-distortion optimization methods to filter coding modes for image blocks. The main difference lies in the precision of the calculation methods used for the coding rate-distortion cost. In coding mode rate-distortion optimization, a more computationally intensive and precise coding rate-distortion cost is generally used. Specifically, the SSD (Sum of Squared Differences) algorithm is typically used to calculate coding distortion, while the CABAC (Context Adaptive Binary Arithmetic Coding) or CAVLC (Context Adaptive Variable Length Coding) algorithm is generally used to calculate the coding bit rate. Furthermore, transformation and quantization operations are typically introduced during the calculation process.

[0030] Step S2: The prediction cost corresponding to each candidate intra-coding mode for the image block to be encoded is filtered, and the minimum prediction cost is selected to characterize the intra-coding complexity of the image block to be encoded. Then, the sum of the intra-coding complexities of all image blocks within the video frame is taken as the intra-coding complexity of the video frame. Preferably, this step calculates at least the intra-coding complexity at the image block level and the video frame level of the preceding video frame for each I-frame, thus serving the subsequent I-frames. This step reuses the calculation results of intra-coding mode prediction (step S1) and does not require the introduction of an additional image preprocessing stage, which is an innovation of this application.

[0031] Step S3: Based on the ratio of the intra-frame coding complexity of the image blocks to be encoded within the I-frame to the total intra-frame coding complexity of the I-frame, and combined with the target coding bit count of the I-frame, calculate the target coding bit count for each image block within the I-frame. For the I-frame to be encoded, the intra-frame coding complexity of each image block within it reuses the intra-frame coding complexity of the image block at the same position within the previous encoded video frame, and the intra-frame coding complexity of its video frame reuses the intra-frame coding complexity of the previous encoded video frame. Assuming the I-frame to be encoded is the nth frame, since encoding has not yet begun, intra-frame coding mode prediction has not started, and therefore, the intra-frame coding complexity at the image block level and the video frame level cannot be obtained. To address this issue, this application leverages the temporal correlation of consecutive video frames, assuming that consecutive video frames are similar in content. It considers the content of the (n-1)th frame to be similar to that of the nth frame. Therefore, the intra-frame coding complexity at the image block level of the nth frame reuses the intra-frame coding complexity of the image blocks at the same position in the (n-1)th frame, and the intra-frame coding complexity at the video frame level of the nth frame reuses the intra-frame coding complexity at the video frame level of the (n-1)th frame. Since the (n-1)th frame has already been encoded, the intra-frame coding complexity of its internal image blocks and the intra-frame coding complexity at the video frame level are known. Through this reuse, the target number of coded bits for each image block within the I-frame to be encoded can be calculated without special preprocessing before encoding, and the calculation accuracy can meet the requirements of the rate control algorithm. This is an innovation of this application.

[0032] As an example, step S3 is defined by the formula: Find T. blk_curr ω represents the target number of encoded bits for the current image block to be encoded. blk_curr Let ω be the intra-frame coding complexity of the current image block to be encoded. pic Let T be the intra-frame coding complexity of the current I-frame to be encoded. Pic The target number of encoded bits for the current I-frame to be encoded. H This is the number of bits required for the header information of the current I-frame to be encoded, as estimated in advance.

[0033] Step S4: Based on the target number of encoded bits and intra-frame coding complexity of the image block to be encoded within the I-frame to be encoded, and combined with the actual number of encoded bits, Lagrange multipliers, and intra-frame coding complexity of the corresponding image block within the previous encoded I-frame, calculate the Lagrange multipliers of the image block to be encoded within the I-frame to be encoded. Similar to Step S3, the intra-frame coding complexity of each image block within the I-frame to be encoded is multiplexed from the intra-frame coding complexity of the corresponding image block within the previous encoded video frame. Assuming the I-frame to be encoded is the nth frame, the intra-frame coding complexity of each image block in the nth frame is multiplexed from the intra-frame coding complexity of the corresponding image block in the (n-1)th frame.

[0034] As an example, step S4 is derived from formula five: Find λ. new f represents the Lagrange multiplier of the image block to be encoded within the I-frame to be encoded. -1 It is the inverse function of the first function f. I(n) new This indicates that the image block to be encoded within the I-frame to be encoded uses λ. new The actual number of encoded bits after encoding is given. At this point, the image block to be encoded within the I-frame has not yet been encoded, so its actual number of encoded bits cannot be obtained. Therefore, the target number of encoded bits for the image block within the I-frame is used as an approximation. I(m) old This represents the actual number of encoded bits for the same image block within the previous encoded I-frame. g is the second function. ω n-1 ω represents the intra-frame coding complexity of image blocks at the same position within the (n-1)th frame. m λ represents the intra-frame coding complexity of image blocks at the same position within the m-th frame. old This represents the Lagrange multiplier of the same image block within the previous encoded I-frame.

[0035] Please see Figure 1 Let frame m and frame n be two consecutive I-frames, with zero or more P-frames or B-frames between them. Assume frame n is the I-frame to be encoded, and all video frames preceding frame n are already encoded. For a specific image block position within a video frame, let λ... old and λ new Let I(m) represent the image block-level Lagrange multipliers at this position within two adjacent I-frames (i.e., the m-th frame and the n-th frame). old The image patch at that location within the m-th frame is represented by λ. old Let I(m) be the actual number of encoded bits after encoding. new The image patch at that location within the m-th frame is represented by λ. new Let I(n) be the actual number of encoded bits after encoding. oldThe image patch representing that location within the nth frame is represented by λ. old Let I(n) be the actual number of encoded bits after encoding. new The image patch representing that location within the nth frame is represented by λ. new The actual number of encoded bits after encoding, let ω m Let ω represent the intra-frame coding complexity of the image block at that position within the m-th frame. n Let ω represent the intra-frame coding complexity of the image patch at that position within the nth frame. n-1 This represents the intra-frame coding complexity of the image block at that position within the (n-1)th frame. Figure 3 This is a schematic diagram of the same image block within the m-th frame, the (n-1)-th frame, and the n-th frame.

[0036] The first function f is given by formula one: The physical meaning of Formula 1 is: when different image block-level Lagrange multipliers are used to intra-code the same image block, the ratio of the actual number of encoded bits obtained after encoding is related to the ratio of the image block-level Lagrange multipliers. This relationship can be represented by the first function f. The specific expression of the first function f can be obtained by fitting a function to both sides of the equal sign in Formula 1, which will not be elaborated here.

[0037] The second function g is derived from formula two: The physical meaning of Formula 2 is as follows: When using the same image block-level Lagrange multipliers to perform intra-frame coding on image blocks at the same position within different video frames, the ratio of the actual number of coded bits obtained after coding is related to the ratio of the intra-frame coding complexity of the image blocks. This relationship can be represented by the second function g. The specific expression of the second function g can be obtained by fitting a function to both sides of the equality in Formula 2, which will not be elaborated here.

[0038] In its specific implementation, this invention pre-determines the expressions for the first function f and the second function g through extensive experiments and by using function fitting. These expressions are stored in the video encoder and represent known functional relationships.

[0039] Based on the derivation of Formula 1 and Formula 2, Formula 3 can be obtained: Further derivation yields Formula 4: Among them, f -1 It is the inverse function of the first function f. Since the first function f is known, then f -1 This is also known. In Formula 4, the m-th frame is an encoded I-frame, whose encoding process has been completed. Therefore, the actual number of encoded bits I(m) corresponding to the same position image block within it is... old The image block-level Lagrange multiplier λ used in encodingold The image block-level intra-frame coding complexity ω m All of these are known quantities. The nth frame is an I-frame to be encoded, I(n). new The original expression represents the actual number of encoded bits for the image blocks within the I-frame to be encoded. However, since the encoding of the n-th frame has not yet begun, the actual number of encoded bits for the image blocks within it cannot be obtained. To derive the Lagrange multipliers required for the encoding process of the image blocks before encoding, this application approximates the actual number of encoded bits by using the target number of encoded bits for the image blocks. Since the "target number of encoded bits" for the image blocks is the expected "actual number of encoded bits" after the actual encoding process, this approximation introduces a very small error. I(n) new The number of target coded bits per I-frame is used to replace the target number of coded bits per I-frame, which is a known quantity. ω n λ represents the intra-frame coding complexity of the image block to be encoded at the same position within the nth frame. Since this image block has not yet been encoded, it is an unknown quantity. new This represents the situation where the actual number of encoded bits for the same image block at the same position within the nth frame is equal to the desired target number of encoded bits, I(n). new At this time, the required image block-level Lagrange multipliers are unknown quantities, and also quantities that need to be calculated. This is to calculate the Lagrange multipliers λ of the image blocks to be encoded within the I-frame using Formula 4. new This application utilizes the temporal correlation of consecutive video frames to reuse the intra-frame coding complexity of an image block at the same position in the (n-1)th frame, which is adjacent to the nth frame, to replace the intra-frame coding complexity of the image block to be encoded within the nth frame. Since the sampling times of the (n-1)th frame and the nth frame are very close, and the content of the two video frames is very similar, the error in replacing the intra-frame coding complexity of the image block at the same position is small, and it does not affect the Lagrange multiplier λ of the image block to be encoded. new This has a significant impact on the calculation. Thus, since the encoding of the image block at the same position in the (n-1)th frame has already been completed, its intra-frame coding complexity ω... n-1 It is a known quantity. In Formula 4, let ω n-1 Replace ω n Then we can get Formula 5.

[0040] In step S4, the Lagrange multiplier λ of the image block to be encoded within the I-frame to be encoded is derived using information from the encoded image blocks within the encoded video frame. new The process, and the replacement operation that utilizes the temporal correlation of consecutive video frames to perform intra-frame coding complexity replacement of the image blocks to be coded within the I-frame to be coded, are innovations of this invention.

[0041] Step S5: Based on the Lagrange multipliers λ of the image block to be encoded within the I-frame to be encoded new The image block to be encoded is video encoded to obtain its actual number of encoded bits. Then, the target number of encoded bits for the next image block to be encoded within the I-frame is adjusted based on the actual number of encoded bits of the encoded image block.

[0042] As an example, the latter part of step S5 is derived from the formula: Find T. blk_next T represents the target number of encoded bits for the next image block to be encoded within the current I-frame to be encoded. Pic The target number of encoded bits for the current I-frame to be encoded. H Coded is the number of bits required for the header information of the current I-frame to be encoded, as estimated in advance. Pic ω represents the actual number of encoded bits in the current I-frame to be encoded (including the actual number of encoded bits in the previous image block that was just encoded). blk ∑ represents the intra-frame coding complexity of a specific uncoded image block within the current I-frame to be encoded. {all not coded blks} ω blk ω is the sum of the intra-frame coding complexities of all uncoded image blocks within the current I-frame to be coded. blk_next This is the intra-frame coding complexity of the next image block to be encoded within the current I-frame. Similar to step S3, the intra-frame coding complexity of each image block within the I-frame to be encoded is obtained by multiplexing the intra-frame coding complexity of the corresponding image block within the previous encoded video frame.

[0043] For a single image block within an I-frame to be encoded, it actually undergoes two bit allocation processes. The first is before encoding of the I-frame, i.e., before encoding begins for any of the image blocks within that I-frame; this is called "pre-allocation," and this process is described in step S3. The second is during the encoding process of the I-frame, i.e., when some image blocks within the I-frame have been encoded while others have not; this is called "redistribution," and this process is described in step S5. The reason for this bit redistribution is that the actual number of bits after encoding of an image block may differ from the number of bits allocated during the "pre-allocation" process. Therefore, for the remaining unencoded image blocks, it is necessary to redistribute the target number of bits for the unencoded image blocks based on the actual number of bits encoded in the already encoded image blocks.

[0044] Please see Figure 4 The image block-level bitrate control device for I-frames suitable for hardware video encoders proposed in this application includes an intra-frame coding mode prediction module 1, an intra-frame coding complexity calculation module 2, an image block bit allocation module 3, an image block bit control module 4, and an image block coding module 5. Figure 4 The device shown corresponds to Figure 2 The method shown.

[0045] The intra-frame coding mode prediction module 1 is used to predict the intra-frame coding mode of each image block to be encoded, and obtain the intra-frame coding mode candidate and its prediction cost for the image block to be encoded.

[0046] The intra-frame coding complexity calculation module 2 is used to filter the prediction cost corresponding to each intra-frame coding mode candidate of the image block to be encoded, and select the minimum prediction cost to characterize the intra-frame coding complexity of the image block to be encoded; then the sum of the intra-frame coding complexities of all image blocks in the video frame is used as the intra-frame coding complexity of the video frame.

[0047] The image block bit allocation module 3 is used to calculate the target number of encoded bits for each image block within the I-frame before encoding, based on the ratio of the intra-frame coding complexity of the image blocks to be encoded within the I-frame to the intra-frame coding complexity of the I-frame itself, and in conjunction with the target number of encoded bits for the I-frame. The image block bit allocation module 3 is also used during the encoding process of the I-frame to be encoded to adjust the target number of encoded bits for the next image block within the I-frame based on the actual number of encoded bits of the already encoded image blocks.

[0048] The image block bit control module 4 is used to calculate the Lagrange multiplier of the image block to be encoded within the I-frame to be encoded based on the target number of encoded bits and the intra-frame coding complexity of the image block to be encoded within the I-frame to be encoded, combined with the actual number of encoded bits, Lagrange multipliers and intra-frame coding complexity of the co-occurring image block within the previous encoded I-frame.

[0049] The image block encoding module 5 is used to determine the Lagrange multiplier λ of the image block to be encoded within the I-frame to be encoded. new Perform video encoding on the image block to be encoded and obtain its actual number of encoded bits.

[0050] When the above modules perform calculations, the intra-frame coding complexity of each image block within the I-frame to be encoded reuses the intra-frame coding complexity of the image block at the same position within the previous encoded video frame, and the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous encoded video frame.

[0051] Chinese invention patent application CN115550656A, published on December 30, 2022, entitled "A Method and Apparatus for I-Frame Rate Control Suitable for Hardware Implementation," discloses a video frame-level rate control method suitable for hardware implementation of I-frames. Compared with CN115550656A, the main technical innovations of this application are as follows.

[0052] First, the objective of CN115550656A is to calculate the value of the video frame-level Lagrange multiplier for an I-frame. For an image frame, there is only one value for the frame-level Lagrange multiplier. The video frame-level Lagrange multiplier can be used for frame-level bitrate control of I-frames, as well as for image block-level bitrate control. When used for image block-level bitrate control, the video frame-level Lagrange multiplier is generally used as a benchmark for threshold conversion to constrain the value of the image block-level Lagrange multiplier. This application applies to image block-level bitrate control of I-frames, and its objective is to calculate the value of the image block-level Lagrange multiplier for each image block within the I-frame, for use in image block-level bitrate control of the I-frame. For example, an I-frame in HEVC contains one or more LCU blocks, each LCU block has its own Lagrange multiplier, and their values ​​can be different.

[0053] Second, both CN115550656A and this application contain formulas one through five, and they are written in the same way. However, the meanings of the variable values ​​in the formulas of the two documents are completely different. The variables in the formulas of CN115550656A are all frame-level variables, while the variables in the formulas of this application are all image block-level variables. For example, in CN115550656A, λ old and λ new All are frame-level Lagrange multipliers, λ in this application old and λ new These are all image block-level Lagrange multipliers. Therefore, this application extends the calculation of frame-level Lagrange multipliers for I-frames as described in CN115550656A to the calculation of image block-level Lagrange multipliers for I-frames.

[0054] Third, in this application, to perform image block-level bit allocation, the ratio of the intra-frame coding complexity of an image block within an I-frame to the intra-frame coding complexity of the I-frame to be encoded is used as the basis for image block bit allocation. CN115550656A, because it uses frame-level rate control, does not have an image block-level bit allocation process. Furthermore, in the image block-level bit allocation process of this application, the intra-frame coding complexity of the image block reuses the intra-frame coding complexity of the image block at the same position within the previous encoded video frame, and the intra-frame coding complexity of the video frame reuses the intra-frame coding complexity of the previous encoded video frame. Moreover, the calculation of the intra-frame coding complexity reuses the intra-frame coding mode prediction cost, eliminating the need for an additional image preprocessing stage.

[0055] Fourth, in this application, after a certain image block within the I-frame to be encoded is encoded, the target number of encoded bits for the next image block to be encoded in the I-frame needs to be adjusted based on the actual number of encoded bits for that image block. CN115550656A, because it uses frame-level rate control, does not have this process.

[0056] The I-frame image block-level rate control method proposed in this application utilizes the intra-frame coding mode prediction cost generated during the encoding process of image blocks within a video frame to characterize the intra-frame coding complexity of image blocks within the video frame, and accordingly completes the target bit allocation for image blocks within the I-frame. This method eliminates the need for additional image preprocessing stages, resulting in low hardware overhead and low implementation cost. Furthermore, this application leverages information from already encoded image blocks in encoded video frames to calculate the Lagrange multipliers of the image blocks to be encoded in the I-frame, achieving high calculation accuracy and making the actual number of encoded bits for the image blocks closer to the target number of encoded bits, thus making the I-frame rate control process more precise.

[0057] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A bitrate control method at the image block level for I-frames, characterized in that, Includes the following steps; Step S1: Predict the intra-frame coding mode for each image block to be encoded, and obtain the candidate intra-frame coding modes and their prediction costs for the image block to be encoded. Step S2: Filter the prediction costs corresponding to each candidate intra-coding mode of the image block to be encoded, and select the minimum prediction cost to characterize the intra-coding complexity of the image block to be encoded; then take the sum of the intra-coding complexities of all image blocks in the video frame as the intra-coding complexity of the video frame. Step S3: Calculate the target coding bit count of each image block within the I-frame to be encoded based on the ratio of the intra-frame coding complexity of the image block to be encoded to the intra-frame coding complexity of the I-frame to be encoded, and in conjunction with the target coding bit count of the I-frame to be encoded. Step S4: Based on the target number of coding bits and intra-frame coding complexity of the image block to be encoded within the I-frame to be encoded, and combined with the actual number of coding bits, Lagrange multipliers and intra-frame coding complexity of the co-occurring image block within the previous encoded I-frame, calculate the Lagrange multipliers of the image block to be encoded within the I-frame to be encoded. Step S4 is derived from Formula 5: Find; where, This represents the Lagrange multiplier of the image block to be encoded within the I-frame to be encoded; It is the inverse function of the first function f, which characterizes the relationship between "the ratio of the actual number of coded bits obtained after intra-frame coding of the same image block using different image block-level Lagrange multipliers" and "the ratio of the number of image block-level Lagrange multipliers". This represents the actual number of encoded bits of the image block to be encoded within the I-frame to be encoded, instead of the target number of encoded bits of the image block to be encoded within the I-frame to be encoded. denoted by , g represents the actual number of encoded bits for the same image block within the previous encoded I-frame; g is the second function characterizing the relationship between "the ratio of the actual number of encoded bits obtained after intra-coding of the same image block within different video frames using the same image block-level Lagrange multipliers and the ratio of the intra-frame coding complexity of the image block"; This represents the intra-frame coding complexity of image blocks at the same position within the (n-1)th frame; This represents the intra-frame coding complexity of image blocks at the same position within the m-th frame; The Lagrange multiplier represents the image block at the same position within the previous encoded I-frame; Step S5: Perform video encoding on the image block to be encoded according to the Lagrange multipliers of the image block to be encoded within the I-frame to be encoded, and obtain its actual number of encoded bits; then adjust the target number of encoded bits for the next image block to be encoded within the I-frame to be encoded according to the actual number of encoded bits of the encoded image block. In this process, the intra-frame coding complexity of each image block within the I-frame to be encoded reuses the intra-frame coding complexity of the image block at the same position within the previous encoded video frame, and the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous encoded video frame.

2. The image block-level bitrate control method for I-frames according to claim 1, characterized in that, In step S1, the prediction cost corresponding to each intra-frame coding mode is calculated for each image block to be encoded, and one or more intra-frame coding modes with the lowest prediction cost are selected as intra-frame coding mode candidates; the prediction cost refers to the coding rate distortion cost.

3. The image block-level bitrate control method for I-frames according to claim 1, characterized in that, Step S2 calculates at least the intra-frame coding complexity at the image block level and video frame level of the preceding video frame for each I-frame.

4. The image block-level bitrate control method for I-frames according to claim 1, characterized in that, Step S3 is defined by the formula: Find; where, This indicates the target number of coded bits for the current image block within the current I-frame to be encoded. This represents the intra-frame coding complexity of the current image block to be encoded within the current I-frame. This represents the intra-frame coding complexity of the current I-frame to be encoded. This indicates the target number of coded bits for the current I-frame to be encoded. This indicates the estimated number of bits required for the header information of the current I-frame to be encoded.

5. The image block-level bitrate control method for I-frames according to claim 1, characterized in that, The first function f is given by formula one: Obtain; This indicates that the image blocks at the same position within the m-th frame are represented by... The actual number of encoded bits after encoding; Formula 1 means that when different image block-level Lagrange multipliers are used to intra-frame encode the same image block, the ratio of the actual number of encoded bits obtained after encoding is related to the ratio of the image block-level Lagrange multipliers. This relationship is represented by the first function f.

6. The image block-level bitrate control method for I-frames according to claim 1, characterized in that, The second function g is derived from formula two: Obtain; Formula 2 represents the intra-frame coding complexity of image blocks at the same position within the nth frame. Formula 2 states that: when using the same image block-level Lagrange multipliers to perform intra-frame coding on image blocks at the same position within different video frames, the ratio of the actual number of encoded bits obtained after coding is related to the ratio of the intra-frame coding complexity of the image blocks. This relationship is represented by the second function g.

7. The image block-level bitrate control method for I-frames according to claim 5 or 6, characterized in that, Based on the derivation of Formula 1 and Formula 2, Formula 3 is obtained: Further derivation yields Formula 4: ;in, The number of target coded bits for the image block to be encoded within the I-frame to be encoded is used as the replacement. Intra-frame coding complexity of the same image block at the same position within the preceding encoded video frame of the I-frame to be encoded. By making substitutions, we obtain Formula 5.

8. The image block-level bitrate control method for I-frames according to claim 1, characterized in that, Step S5 is defined by the formula: Find; where, This indicates the target number of coded bits for the next image block to be encoded within the current I-frame to be encoded. This indicates the target number of coded bits for the current I-frame to be encoded. This indicates the estimated number of bits required for the header information of the current I-frame to be encoded. This indicates the actual number of encoded bits in the current I-frame to be encoded. This represents the sum of the intra-frame coding complexities of all uncoded image blocks within the current I-frame to be encoded. This represents the intra-frame coding complexity of the next image block to be encoded within the current I-frame.

9. A bitrate control device at the image block level for an I-frame, characterized in that, It includes an intra-frame coding mode prediction module, an intra-frame coding complexity calculation module, an image block bit allocation module, an image block bit control module, and an image block coding module; The intra-frame coding mode prediction module is used to predict the intra-frame coding mode of each image block to be encoded, and obtain the intra-frame coding mode candidate and its prediction cost for the image block to be encoded. The intra-frame coding complexity calculation module is used to filter the prediction cost corresponding to each intra-frame coding mode candidate of the image block to be encoded, and select the minimum prediction cost to characterize the intra-frame coding complexity of the image block to be encoded; then the sum of the intra-frame coding complexities of all image blocks in the video frame is used as the intra-frame coding complexity of the video frame. The image block bit allocation module is used to calculate the target number of coding bits for each image block to be encoded within the I-frame before encoding, based on the ratio of the intra-frame coding complexity of the image blocks to be encoded within the I-frame to the intra-frame coding complexity of the I-frame, and in conjunction with the target number of coding bits for the I-frame. The image block bit allocation module is also used to adjust the target number of encoded bits for the next image block to be encoded within the I-frame to be encoded, based on the actual number of encoded bits of the already encoded image blocks, during the encoding process of the I-frame to be encoded. The image block bit control module is used to calculate the Lagrange multiplier of the image block to be encoded within the I-frame to be encoded based on the target number of encoded bits and the intra-frame coding complexity of the image block to be encoded within the I-frame to be encoded, combined with the actual number of encoded bits, Lagrange multipliers and intra-frame coding complexity of the co-position image block within the previous encoded I-frame. The Lagrange multipliers of the image block to be encoded are given by Formula 5: Find; where, This represents the Lagrange multiplier of the image block to be encoded within the I-frame to be encoded; It is the inverse function of the first function f, which characterizes the relationship between "the ratio of the actual number of coded bits obtained after intra-frame coding of the same image block using different image block-level Lagrange multipliers" and "the ratio of the number of image block-level Lagrange multipliers". This represents the actual number of encoded bits of the image block to be encoded within the I-frame to be encoded, instead of the target number of encoded bits of the image block to be encoded within the I-frame to be encoded. denoted by , g represents the actual number of encoded bits for the same image block within the previous encoded I-frame; g is the second function characterizing the relationship between "the ratio of the actual number of encoded bits obtained after intra-coding of the same image block within different video frames using the same image block-level Lagrange multipliers and the ratio of the intra-frame coding complexity of the image block"; This represents the intra-frame coding complexity of image blocks at the same position within the (n-1)th frame; This represents the intra-frame coding complexity of image blocks at the same position within the m-th frame; The Lagrange multiplier represents the image block at the same position within the previous encoded I-frame; The image block encoding module is used to perform video encoding on the image block to be encoded according to the Lagrange multipliers of the image block to be encoded within the I-frame to be encoded, and to obtain its actual number of encoded bits. In this process, the intra-frame coding complexity of each image block within the I-frame to be encoded reuses the intra-frame coding complexity of the image block at the same position within the previous encoded video frame, and the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous encoded video frame.

Images