A bitrate control method and apparatus for avoiding the generation of excessively large I-frames

Abstract

This invention discloses a rate control method to avoid generating excessively large I-frames. Step S1: Perform intra-frame coding mode prediction for each image block to be encoded, obtaining all intra-frame coding mode candidates and their corresponding prediction costs for each image block. Step S2: Obtain the intra-frame coding complexity of the image block to be encoded and the video frame. Step S3: Calculate the upper limit of the number of encoded bits for an I-frame that will not cause abnormal rate control. Step S4: Calculate the lower limit of the Lagrange multiplier for the I-frame to be encoded. Step S5: Calculate the Lagrange multiplier for the I-frame to be encoded that suppresses the breathing effect. Step S6: Select the larger value between the lower limit of the Lagrange multiplier for the I-frame to be encoded and the Lagrange multiplier for the I-frame to suppress the breathing effect as the Lagrange multiplier for the I-frame to be encoded. This application can both suppress the breathing effect and avoid generating excessively large I-frames during the encoding process.

Description

Technical Field

[0001] This invention relates to a video coding technique, and more particularly to a bitrate control method. Background Technology

[0002] Video coding technology compresses redundant components in video images and represents video information using as little data as possible. Common video coding standards include AVC (Advanced Video Coding, also known as H.264) and HEVC (High Efficiency Video Coding, also known as H.265). Video coding technology uses image blocks as the most basic coding unit. For example, in HEVC, the basic coding unit is the CU (Coding Unit). A CU can be an image block of 64 pixels × 64 pixels, 32 pixels × 32 pixels, 16 pixels × 16 pixels, or 8 pixels × 8 pixels. The 64-pixel × 64-pixel image block is also called the LCU (Largest Coding Unit).

[0003] To remove information redundancy in the spatial and temporal domains, input video frames are encoded using intra-frame coding and inter-frame coding techniques. Encoded video frames are generally divided into I-frames (using only intra-frame coding) and P-frames and B-frames (using a hybrid approach). I-frames use only intra-frame coding, employing only the information of the current frame and not referencing information from other encoded frames. All image blocks within an I-frame are encoded as intra-coded blocks. P-frames and B-frames use a hybrid approach, employing both intra-frame and inter-frame coding. They not only use the information of the current frame but also reference information from one or more other encoded frames. Image blocks within P-frames and B-frames can be encoded as either intra-coded or inter-coded blocks. For each image block within a P-frame or B-frame, if intra-frame coding results in a lower rate-distortion cost, the image block will be encoded as an intra-coded block; if inter-frame coding results in a lower rate-distortion cost, the image block will be encoded as an inter-coded block. For each input video frame, the type of frame to be encoded is determined by the user-defined Group of Pictures (GOP, a set of time-series video frames). During video encoding, because P-frames and B-frames are more efficient, while I-frames help prevent error propagation, a common approach is to periodically encode I-frames and encode the majority of other video frames as P-frames or B-frames. Figure 1 It demonstrates a common image group structure, which includes I-frames, P-frames, or B-frames.

[0004] To maximize video compression, common video coding algorithms are often lossy compression algorithms, such as AVC and HEVC. This results in a difference between the reconstructed video and the original video; that is, the reconstructed video will contain distortion. For lossy compression algorithms, their coding performance needs to be measured by both the coding bitrate and the distortion introduced by the coding. In the video coding process, the coding bitrate and coding distortion are mutually restrictive and contradictory. For example, reducing the coding bitrate often increases coding distortion, while reducing coding distortion often increases the coding bitrate. Rate Distortion Optimization (RDO) aims to minimize coding distortion at a given coding bitrate, or minimize the coding bitrate at a given coding distortion. Currently, most common video coding algorithms use rate distortion optimization methods based on the Lagrange multiplier λ. Assuming D is the coding distortion, R is the coding bitrate, and J is the coding rate distortion cost (RD Cost), J = D + λ × R. Rate-distortion optimization based on the Lagrange multiplier λ can be expressed as min(J), where min() represents minimizing the value. In video coding, each coding mode has its corresponding coding distortion D and coding bit rate R, allowing the calculation of the coding rate-distortion cost J for each mode. Selecting the coding mode with the minimum rate-distortion cost J for video coding yields optimal coding performance. The process by which the encoder selects the optimal coding mode for each image block to be encoded using rate-distortion optimization is called mode decision for the image block.

[0005] In real-world scenarios, the channel bandwidth (channel capacity) used to transmit compressed video is limited. If the encoding bit rate of the compressed video is too high, exceeding the channel bandwidth capacity, it will cause video transmission congestion or even packet loss. If the encoding bit rate of the compressed video is too low, the channel bandwidth will not be fully utilized, and higher video quality cannot be achieved. Therefore, it is necessary to use bitrate control technology to control the output bitrate of the video encoder to match it with the channel bandwidth.

[0006] Rate control technology adjusts the encoding parameters of the video encoder to make the output bitrate of the video encoder equal to a pre-set target bitrate, while minimizing encoding distortion to improve video encoding quality. Common rate control algorithms typically achieve this through two stages: target bit allocation and target bit control.

[0007] In common rate control algorithms, target bit allocation is performed at three levels: GOP level, video frame level, and image block level. After target bit allocation at the GOP and video frame levels, the target number of encoded bits for the current video frame to be encoded is determined. Then, in the target bit control stage, the corresponding Lagrange multiplier λ is calculated based on the target number of encoded bits for each video frame to be encoded, and used in the video frame encoding process.

[0008] Because video encoding algorithms involve a large amount of computation, using application-specific integrated circuits (ASICs) to accelerate the video encoding process in hardware has become a common industry practice in order to improve video encoding speed and achieve real-time encoding and transmission. These ASICs that accelerate the video encoding process in hardware are generally called hardware video encoders.

[0009] When encoding video surveillance and video conferencing scenarios, periodic visual flicker is frequently observed in the encoded video stream. This flicker is particularly noticeable in static areas of the scene. This periodic visual flicker phenomenon is called the "breathing effect." The "breathing effect" is related to the periodically encoded I-frames in the video stream. Because I-frames only use intra-frame coding and do not reference information from previously encoded video frames, the distortion magnitude and pattern of the I-frame differ significantly from the P-frames or B-frames encoded before it. This distortion difference causes the visual flicker to be perceived by the human eye when viewing the decoded video stream. Since I-frames are generally encoded periodically during video encoding, this visual flicker will occur periodically, resulting in the "breathing effect." The "breathing effect" significantly impacts the viewer's subjective visual experience; therefore, it should be suppressed during encoding.

[0010] To suppress the breathing effect during video encoding, the most common method is to set the QP (Quantization Parameter) of the I-frame near the QP of the previous encoded P-frame or B-frame, i.e.: I-frame QP = QP of the previous encoded P-frame or B-frame + offset δ. Here, the offset δ is set to a small integer, typically between -2 and 2. This makes the distortion of the I-frame close to that of the previously encoded P-frame or B-frame, effectively suppressing the breathing effect. In this way, after calculating the QP of the I-frame to be encoded based on the QP of the previous encoded P-frame or B-frame and the offset δ, the Lagrange multiplier λ of the I-frame can be calculated using the relationship between QP and the Lagrange multiplier λ, and then used in the actual encoding process of the I-frame.

[0011] While the above method for encoding I-frames can effectively suppress the breathing effect, it can easily lead to the encoding of excessively large I-frames in certain video scenarios, particularly those with static temporal content and complex spatial textures. These I-frames have a significantly higher bitrate than P-frames or B-frames. This not only causes severe instantaneous bitrate fluctuations but also results in an average bitrate far exceeding the target bitrate, leading to abnormal bitrate control. In these video scenarios, due to the static temporal content, the QP (Queries Per Frame) of P-frames and B-frames encoded using inter-frame coding is relatively small. When encoding I-frames, to suppress the breathing effect, the QP is set near the QP of the previous encoded P-frame or B-frame, resulting in a smaller QP value for the I-frame. Simultaneously, due to the complex spatial textures in these video scenarios, using intra-frame coding with a smaller QP can encode excessively large I-frames, far exceeding the size of P-frames or B-frames, causing severe instantaneous bitrate fluctuations. In severe cases, this can even cause the average bitrate to far exceed the target bitrate, rendering bitrate control ineffective. In other words, in this case, setting the QP value of the I-frame to be encoded near the QP value of the previous encoded P-frame or B-frame can suppress the breathing effect, but it will produce an oversized I-frame, leading to abnormal bitrate control.

[0012] The industry hopes for a technology that, during the bitrate control process of video encoding, sets the QP of the I-frame near the QP of the previous encoded P-frame or B-frame when no extra-large I-frames will be encoded, in order to suppress the breathing effect and improve subjective visual quality; and only suppresses the size of the I-frame when extra-large I-frames will be encoded, so as to avoid abnormal bitrate control.

[0013] Currently, there are several technical solutions available to avoid generating excessively large I-frames during video encoding.

[0014] One approach is to calculate the QP of the I-frame from the QP of the previous encoded P-frame or B-frame, but use a large positive integer offset δ. This way, since the QP of the I-frame is much larger than that of the P-frame or B-frame, the encoded I-frame is smaller, preventing the number of encoded bits from far exceeding that of the P-frame or B-frame. This approach effectively avoids oversized I-frames, minimizes instantaneous bitrate fluctuations, and provides more accurate average bitrate control. However, in video encoding scenarios where oversized I-frames wouldn't normally be encoded, this approach, by indiscriminately making the I-frame's QP much larger than that of the P-frame or B-frame, results in poor I-frame encoding quality. This degrades the encoding quality of P-frames and B-frames that directly or indirectly reference this I-frame, significantly reducing the overall encoding efficiency of the video stream.

[0015] Another approach is to calculate the QP of an I-frame entirely based on the target number of encoded bits, independent of the QP of the preceding encoded P-frame or B-frame. While this approach effectively avoids excessively large I-frames, it is highly prone to situations where the QP of an I-frame differs significantly from that of the preceding encoded P-frame or B-frame. This results in a substantial difference in the subjective visual quality of the I-frame compared to the P-frame or B-frame, causing severe visual flicker and significantly reducing the overall subjective visual quality of the video stream. Summary of the Invention

[0016] The technical problem to be solved by this invention is to design a bitrate control method that, while maintaining the subjective visual quality of I-frames as consistent as possible with that of previously encoded P-frames and B-frames to suppress the breathing effect, avoids the generation of ultra-large I-frames during the encoding process.

[0017] To address the aforementioned technical problems, this invention proposes a rate control method to avoid generating excessively large I-frames, comprising the following steps: Step S1: Perform intra-frame coding mode prediction for each image block to be encoded, obtaining all intra-frame coding mode candidates and their corresponding prediction costs for each image block to be encoded. Step S2: Filter the prediction costs corresponding to each intra-frame coding mode candidate of the image block to be encoded, selecting the minimum prediction cost to characterize the intra-frame coding complexity of the image block to be encoded; then, the sum of the intra-frame coding complexities of all image blocks within the video frame is taken as the intra-frame coding complexity of the video frame. Step S3: Based on the encoding state of the current video sequence, calculate the upper limit of the number of encoded bits for an I-frame that will not cause abnormal rate control. Step S4: Based on the actual number of encoded bits, Lagrange multipliers, and intra-frame coding complexity of the previous encoded I-frame, and combining the upper limit of the number of encoded bits and the intra-frame coding complexity of the I-frame to be encoded, calculate the lower limit of the Lagrange multipliers of the I-frame to be encoded; wherein, the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous video frame. Step S5: Based on the quantization parameters of the previous encoded P-frame or B-frame, calculate the Lagrange multipliers of the I-frame to be encoded that suppress the breathing effect. Steps S1 to S4 constitute a group; this group and step S5 can be performed simultaneously or arbitrarily before each other. Step S6: Select the larger value between the lower limit of the Lagrange multipliers of the I-frame to be encoded and the Lagrange multipliers of the I-frame to be encoded that suppress the breathing effect, as the Lagrange multipliers of the I-frame to be encoded.

[0018] Further, in step S1, the prediction cost corresponding to each intra-frame coding mode is calculated for each image block to be encoded, and one or more intra-frame coding modes with the lowest prediction cost are selected as intra-frame coding mode candidates for the image block to be encoded; the prediction cost refers to the coding rate distortion cost.

[0019] Preferably, step S2 calculates at least the intra-frame coding complexity of each I-frame and the video frame preceding each I-frame.

[0020] Furthermore, in step S3, the state information of the sliding window used to smooth frame-level bit allocation during the rate control process is also used to calculate the upper limit of the number of encoded bits for an I-frame that will not cause abnormal rate control.

[0021] Furthermore, in step S3, the target number of encoded bits R of the sliding window is first calculated. sw R sw =R PicAvg ×(N coded +N SW )-R coded Among them, R PicAvg The average number of target coded bits per video frame, N, is calculated from the average bitrate of the video sequence. coded N is the number of currently encoded video frames in the video sequence. SW R is the size of the sliding window. coded It is the sum of the actual encoded bits of all currently encoded video frames in the video sequence; then, according to R... sw Calculate the upper limit R of the number of encoded bits for an I-frame that will not cause bitrate control anomalies. Imax ; Where, N I It is the number of I-frames in the sliding window; η is a constant with a value range of (0,1).

[0022] Furthermore, step S4 is derived from formula five: Find; where λ new It is the lower bound of the Lagrange multipliers of the I-frame to be encoded; f -1 It is the inverse function of the first function f; I(n) new I(m) represents the upper limit of the number of encoded bits for an I-frame that will not cause rate control anomalies. old This represents the actual number of encoded bits in the previous encoded I-frame; g is the second function; ω n-1 The value represents the intra-frame coding complexity of the preceding encoded video frame, used to replace the intra-frame coding complexity ω of the I-frame to be encoded. n ;ω m λ represents the intra-frame coding complexity of the previous encoded I-frame; old This represents the Lagrange multiplier of the previous encoded I-frame.

[0023] Furthermore, the first function f is given by Formula 1: Formula 1 states that when using different Lagrange multipliers to perform I-frame encoding on the same video frame, the ratio of the actual number of encoded bits obtained is related to the ratio of the Lagrange multipliers. This relationship is represented by the first function f. The second function g is derived from Formula 2: Formula 2 states that when using the same Lagrange multipliers to perform I-frame encoding on different video frames, the ratio of the actual number of encoded bits obtained after encoding is related to the ratio of the intra-frame encoding complexity of the video frames. This relationship is represented by the second function g.

[0024] Furthermore, based on Formula 1 and Formula 2, Formula 3 is derived: Further derivation yields Formula 4: Where, λ new It is the Lagrange multiplier of the I-frame to be encoded; I(n) new Replaced by the target number of coded bits of the I-frame to be encoded; ω n The intra-frame coding complexity ω of the previous encoded video frame is derived from the I-frame to be encoded. n-1 By substitution, we obtain Formula 5; let I(n) new λ is the upper limit of the number of encoded bits for the I-frame to be encoded that will not cause rate control anomalies. new This is the lower bound of the Lagrange multipliers of the I-frame to be encoded.

[0025] Furthermore, in step S5, the quantization parameter QP of the previous encoded P-frame or B-frame is first used to encode the I-frame to be encoded. PB The quantization parameter QP, which suppresses the breathing effect, is calculated from the offset δ of the I-frame to be encoded. I The calculation formula is: QP I =QP PB +δ; where δ is an integer offset; then, based on the quantization parameter QP of the I-frame to be encoded, which has a suppressive effect on breathing. I The Lagrange multiplier λ that suppresses the breathing effect in the I-frame to be encoded is calculated. I The calculation formula is: Exp() is an exponential function with the natural logarithm e as its base.

[0026] This application also proposes a rate control device to avoid generating excessively large I-frames, including an intra-frame coding mode prediction module, an intra-frame coding complexity calculation module, an I-frame coding bit count upper limit calculation module, an I-frame Lagrange multiplier lower limit calculation module, an I-frame Lagrange multiplier calculation module, and an I-frame Lagrange multiplier selection module. The intra-frame coding mode prediction module predicts the intra-frame coding mode for each image block to be encoded, outputting all intra-frame coding mode candidates and their corresponding prediction costs for each image block. The intra-frame coding complexity calculation module filters the prediction costs corresponding to each intra-frame coding mode candidate for the image block to be encoded, selecting the minimum prediction cost to characterize the intra-frame coding complexity of the image block; then, the sum of the intra-frame coding complexities of all image blocks within the video frame is used as the intra-frame coding complexity of the video frame. The I-frame coding bit count upper limit calculation module calculates the upper limit of the coding bit count for I-frames that will not cause abnormal rate control based on the coding state of the current video sequence. The I-frame Lagrange multiplier lower bound calculation module is used to calculate the lower bound of the Lagrange multipliers of the I-frame to be encoded based on the actual number of encoded bits, Lagrange multipliers, and intra-frame coding complexity of the previous encoded I-frame, combined with the upper bound of the number of encoded bits and the intra-frame coding complexity of the I-frame to be encoded; wherein, the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous video frame. The I-frame Lagrange multiplier calculation module is used to calculate the Lagrange multipliers of the I-frame to be encoded that suppress the breathing effect based on the quantization parameters of the previous encoded P-frame or B-frame. The I-frame Lagrange multiplier selection module is used to select the larger value between the lower bound of the Lagrange multipliers of the I-frame to be encoded and the Lagrange multipliers of the I-frame to be encoded that suppress the breathing effect, as the Lagrange multipliers of the I-frame to be encoded.

[0027] The technical effect achieved by this invention is that it proposes a bitrate control method for I-frames, which can suppress the breathing effect and avoid generating excessively large I-frames during the encoding process. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of a common image group structure.

[0029] Figure 2 This is a flowchart illustrating the bitrate control method proposed in this application to avoid generating excessively large I-frames.

[0030] Figure 3 This is a schematic diagram of the bitrate control device proposed in this application to avoid generating excessively large I-frames.

[0031] The following are the labels in the figure: 1 is the intra-frame coding mode prediction module, 2 is the intra-frame coding complexity calculation module, 3 is the upper limit calculation module for the number of I-frame coding bits, 4 is the lower limit calculation module for the I-frame Lagrange multiplier, 5 is the I-frame Lagrange multiplier calculation module, and 6 is the I-frame Lagrange multiplier selection module. Detailed Implementation

[0032] Please see Figure 2 The bitrate control method proposed in this application to avoid generating excessively large I-frames includes the following steps.

[0033] Step S1: Perform intra-frame coding mode prediction for each image block to be encoded. This involves calculating the prediction cost for each intra-frame coding mode for each image block, selecting one or more intra-frame coding modes with the lowest prediction cost as candidates for that image block, and outputting all candidate intra-frame coding modes and their corresponding prediction costs for each image block. The prediction cost refers to the low-computation, low-precision coding rate distortion cost J, J = D + λ × R. Here, D is the coding distortion, typically using the SAD (Sum of Absolute Difference) algorithm or the SATD (Sum of Absolute Transformed Difference) algorithm; R is the coding bit rate, typically using a simplified bit rate estimation algorithm, such as Exponential-Golomb coding. Furthermore, transformation and quantization operations are generally not introduced during the calculation. This step is inherent in existing video coding processes; intra-frame coding mode prediction is performed for all image blocks in I-frames, P-frames, and B-frames.

[0034] Both coding mode prediction and coding mode rate-distortion optimization employ rate-distortion optimization methods to filter coding modes for image blocks. The main difference lies in the precision of the calculation methods used for the coding rate-distortion cost. In coding mode rate-distortion optimization, a more computationally intensive and precise coding rate-distortion cost is generally used. Specifically, the SSD (Sum of Squared Differences) algorithm is typically used to calculate coding distortion, while the CABAC (Context Adaptive Binary Arithmetic Coding) or CAVLC (Context Adaptive Variable Length Coding) algorithm is generally used to calculate the coding bit rate. Furthermore, transformation and quantization operations are typically introduced during the calculation process.

[0035] Step S2: Filter the prediction costs corresponding to each intra-coding mode candidate of the image block to be encoded, and select the minimum prediction cost to characterize the intra-coding complexity of the image block to be encoded; then, the sum of the intra-coding complexities of all image blocks within the video frame is taken as the intra-coding complexity of the video frame. Preferably, this step calculates at least the intra-coding complexity of each I-frame and the video frame preceding each I-frame, thus serving the subsequent I-frames. This step reuses the calculation results of intra-coding mode prediction (step S1), does not require the introduction of an additional image preprocessing stage, and has very low hardware implementation cost.

[0036] Step S3: Based on the encoding state of the current video sequence, calculate the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control anomalies. For example, use the state information of the sliding window (SW) used to smooth frame-level bit allocation during bitrate control to calculate the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control anomalies.

[0037] During video encoding, when a video frame is encoded, there is often an error between the actual number of encoded bits and the target number of encoded bits calculated before encoding. This error can be smoothed out using a sliding window consisting of multiple subsequent consecutive video frames to reduce instantaneous bitrate fluctuations. The number of consecutive video frames contained within the sliding window is called the size of the sliding window, and the sum of the target number of encoded bits for all video frames within it is called the target number of encoded bits for the sliding window. The target number of encoded bits for the sliding window can be calculated using the following formula: R sw =R PicAvg ×(N coded +N SW )-R coded Among them, R sw R is the target number of encoded bits for the sliding window. PicAvg The average number of target coded bits per video frame, N, is calculated from the average bitrate of the video sequence. coded N is the number of currently encoded video frames in the video sequence. SW R is the size of the sliding window. coded It is the sum of the actual encoded bits of all currently encoded video frames in the video sequence. R PicAvg N coded N SW R coded This represents the current encoding state of the video sequence. It can be seen that as the encoding process progresses, R... sw The value of R changes continuously depending on the encoding state of the video sequence. This step, performed before encoding each I-frame, calculates R based on the current video sequence encoding state. sw The value, and based on R swThe value is used to calculate the upper limit of the encoded bits for I-frames that will not cause bitrate control anomalies. For example, the calculation formula is as follows: Among them, R Imax This is the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control anomalies; R sw N is the target number of encoded bits for the sliding window; I It is the number of I-frames in the sliding window; the value of η ranges from (0,1], and its specific value is related to the encoding parameters of the video sequence. It can generally be set to 0.67.

[0038] Step S4: Based on the actual number of encoded bits, Lagrange multipliers, and intra-frame coding complexity of the previous encoded I-frame, and combining the upper limit of the number of encoded bits and the intra-frame coding complexity of the I-frame to be encoded, calculate the lower limit of the Lagrange multipliers of the I-frame to be encoded. The intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous video frame.

[0039] The lower bound of the Lagrange multipliers of the I-frame to be encoded is given by Formula 5: Here is an example of how to find λ. new It is the Lagrange multiplier of the I-frame to be encoded. -1 It is the inverse function of the first function f. I(n) new Indicates that the I-frame to be encoded uses λ new The actual number of encoded bits after encoding is given. Since the I-frame to be encoded has not yet been encoded, its actual number of encoded bits cannot be obtained. Therefore, the target number of encoded bits for the I-frame to be encoded is used as an approximation. I(m) old This represents the actual number of encoded bits in the previous encoded I-frame. g is the second function. ω n-1 The value represents the intra-frame coding complexity of the preceding encoded video frame, used to replace the intra-frame coding complexity ω of the I-frame to be encoded. n ω m This represents the intra-frame coding complexity of the previous encoded I-frame. λ old Let I(n) represent the Lagrange multiplier of the previous encoded I-frame. In Equation 5, let I(n)... new λ is the upper limit of the number of encoded bits for the I-frame to be encoded that will not cause rate control anomalies. new This is the lower bound of the Lagrange multipliers of the I-frame to be encoded.

[0040] Please see Figure 1 , where the m-th frame and the n-th frame are two consecutive I-frames, and there may be zero or more P-frames or B-frames between them. λ old and λ new Let I(m) represent the values ​​of two distinct Lagrange multipliers. old Indicates that the m-th frame uses λ oldThe actual number of encoded bits after encoding. I(m) new Indicates that the m-th frame uses λ new The actual number of encoded bits after encoding. I(n) old Indicates that the nth frame uses λ old The actual number of encoded bits after encoding. I(n) new Indicates that the nth frame uses λ new The actual number of encoded bits after encoding. ω m ω represents the intra-frame coding complexity of the m-th frame. n This represents the intra-frame coding complexity of the nth frame.

[0041] The first function f is given by formula one: The physical meaning of Formula 1 is: when using different Lagrange multipliers to perform I-frame encoding on the same video frame, the ratio of the actual number of encoded bits obtained is related to the ratio of the Lagrange multipliers. This relationship is represented by the first function f. The specific expression of the first function f can be obtained by fitting a function to both sides of the equal sign in Formula 1, which will not be elaborated here.

[0042] The second function g is derived from formula two: The physical meaning of Formula 2 is as follows: When using the same Lagrange multipliers to perform I-frame encoding on different video frames, the ratio of the actual number of encoded bits obtained after encoding is related to the ratio of the intra-frame coding complexity of the video frames. This relationship is represented by the second function g. The specific expression of the second function g can be obtained by fitting a function to both sides of the equals sign in Formula 2, which will not be elaborated here.

[0043] In its specific implementation, this application pre-exists expressions for the first function f and the second function g through extensive experiments and function fitting, which are stored in the video encoder and represent known functional relationships.

[0044] Based on the derivation of Formula 1 and Formula 2, Formula 3 can be obtained: Further derivation yields Formula 4: In Formula 4, the m-th frame is an encoded I-frame whose encoding process has been completed, and its corresponding actual number of encoded bits is I(m). old The Lagrange multiplier λ used in encoding old Intra-frame coding complexity ω m All of these are known quantities. The nth frame is an I-frame to be encoded, I(n). newThe original text represents the actual number of encoded bits for the I-frame to be encoded. However, since the encoding of the nth frame has not yet begun, its actual number of encoded bits is unavailable. To derive the Lagrange multipliers required for the encoding process of the I-frame before encoding, this application approximates the actual number of encoded bits by using the target number of encoded bits for the I-frame. Since the "target number of encoded bits" for the I-frame is the expected "actual number of encoded bits" after the actual encoding process, this approximation introduces a very small error. I(n) new The number of target coded bits for the I-frame to be encoded is used as a substitute, which is a known quantity. ω n λ represents the intra-frame coding complexity of the nth frame, which is an unknown quantity since the nth frame has not yet been encoded. new This represents the situation where the actual number of encoded bits in the nth frame is equal to the desired target number of encoded bits, I(n). new When calculating the Lagrange multipliers needed, these are unknown quantities, but also quantities that need to be determined. This is done using Formula 4 to calculate the Lagrange multipliers λ of the I-frame to be encoded. new This invention utilizes the temporal correlation of consecutive video frames (consecutive video frames are similar in content) and uses the intra-frame coding complexity ω of the (n-1)th frame that is already encoded and adjacent to the nth frame. n-1 To replace the intra-frame coding complexity ω of the nth frame to be encoded n Since the sampling times of frame (n-1) and frame (n) are very close, the content of the two video frames is very similar. The error in the replacement operation within the intra-frame coding complexity is very small, and it will not affect the frame-level Lagrange multiplier λ of the uncoded I-frame. new This has a significant impact on the calculation. Thus, since the encoding of the (n-1)th frame has already been completed, its intra-frame coding complexity ω... n-1 It is a known quantity. In Formula 4, let ω n-1 Replace ω n Then we can get Formula 5.

[0045] Step S5: Calculate the Lagrange multiplier that inhibits the breathing effect in the I-frame to be encoded, based on the quantization parameters of the previous encoded P-frame or B-frame.

[0046] In this step, the quantization parameters QP of the preceding encoded P-frame or B-frame are first used to determine the I-frame to be encoded. PB The quantization parameter QP, which suppresses the breathing effect, is calculated from the offset δ of the I-frame to be encoded. I The calculation formula is: QP I =QP PB+δ. Here, δ is a small integer offset, for example, between [-2, 2]. This allows the distortion of the I-frame to be close to the distortion of the P-frames or B-frames encoded before it, effectively suppressing the breathing effect. Then, based on the quantization parameter QP of the I-frame to be encoded, which has a suppressive effect on the breathing effect... I The Lagrange multiplier λ that suppresses the breathing effect in the I-frame to be encoded is calculated. I The calculation formula is: Exp() is an exponential function with the natural logarithm e as its base.

[0047] Steps S1, S2, S3, and S4 form a group. The order of this group with step S5 is not strictly limited; they can be performed simultaneously or one of them can come first.

[0048] Step S6: Select the larger value between the lower limit of the Lagrange multiplier of the I-frame to be encoded and the Lagrange multiplier of the I-frame to be encoded that has an inhibitory effect on the breathing effect, and use it as the Lagrange multiplier of the I-frame to be encoded for the actual encoding process of the I-frame to be encoded.

[0049] Based on step S5, the Lagrange multiplier λ that suppresses the breathing effect of the I-frame to be encoded is calculated. I While this method can effectively suppress the breathing effect, in certain coding scenarios, it may result in excessively large I-frames, leading to abnormal bitrate control. Therefore, when the Lagrange multiplier λ of the I-frame to be encoded has a suppressive effect on the breathing effect... I ≥ The lower bound λ of the Lagrange multipliers that will not cause abnormal bit rate control new When choosing λ I The actual coding process used for the I-frame to be coded can effectively suppress the breathing effect. When λ I <λ new When choosing λ new The actual encoding process used for the I-frame to be encoded can effectively avoid the generation of oversized I-frames. In this way, the goal of this invention can be achieved, namely: during the bitrate control process of video encoding, when oversized I-frames will not be encoded, the QP of the I-frame is set near the QP of the previous encoded P-frame or B-frame to suppress the breathing effect and improve subjective visual quality; only when oversized I-frames will be encoded are the size of the I-frame suppressed to avoid abnormal bitrate control.

[0050] Please see Figure 3 The bit rate control device proposed in this application to avoid generating excessively large I-frames includes an intra-frame coding mode prediction module 1, an intra-frame coding complexity calculation module 2, an upper limit calculation module for the number of I-frame coding bits 3, an I-frame Lagrange multiplier lower limit calculation module 4, an I-frame Lagrange multiplier calculation module 5, and an I-frame Lagrange multiplier selection module 6. Figure 3The device shown corresponds to Figure 2 The method shown.

[0051] The intra-frame coding mode prediction module 1 is used to predict the intra-frame coding mode for each image block to be encoded, and output all intra-frame coding mode candidates and their corresponding prediction costs for each image block to be encoded.

[0052] The intra-frame coding complexity calculation module 2 is used to filter the prediction cost corresponding to each intra-frame coding mode candidate of the image block to be encoded, and select the minimum prediction cost to characterize the intra-frame coding complexity of the image block to be encoded; then the sum of the intra-frame coding complexities of all image blocks in the video frame is used as the intra-frame coding complexity of the video frame.

[0053] The I-frame encoding bit limit calculation module 3 is used to calculate the upper limit of the I-frame encoding bit count that will not cause abnormal bit rate control based on the encoding status of the current video sequence.

[0054] The I-frame Lagrange multiplier lower bound calculation module 4 is used to calculate the lower bound of the Lagrange multipliers of the I-frame to be encoded based on the actual number of encoded bits, the Lagrange multipliers, and the intra-frame coding complexity of the previous encoded I-frame, combined with the upper bound of the number of encoded bits and the intra-frame coding complexity of the I-frame to be encoded. The intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous video frame.

[0055] The I-frame Lagrange multiplier calculation module 5 is used to calculate the Lagrange multiplier of the I-frame to be encoded that has an inhibitory effect on the breathing effect based on the quantization parameters of the previous encoded P-frame or B-frame.

[0056] The I-frame Lagrange multiplier selection module 6 is used to select the larger value among the lower limit value of the Lagrange multiplier of the I-frame to be encoded and the Lagrange multiplier of the I-frame to be encoded that has an inhibitory effect on the breathing effect, as the Lagrange multiplier of the I-frame to be encoded.

[0057] The bitrate control method proposed in this application uses the intra-frame coding mode prediction cost generated during the video frame coding process to characterize the intra-frame coding complexity of the video frame. It does not require an additional image preprocessing stage, has low hardware overhead, and low implementation cost. This application also uses the information of the encoded video frames and the coding state of the video sequence to calculate the lower limit of the Lagrange multipliers of the I-frame to be encoded that will not cause abnormal bitrate control. It uses this lower limit to constrain the Lagrange multipliers of the I-frame to be encoded that have a suppressive effect on the breathing effect. On the basis of effectively suppressing the breathing effect, it avoids the generation of ultra-large I-frames, which not only improves the subjective visual quality of the video, but also ensures the normal operation of the bitrate control process.

[0058] Chinese invention patent application CN115550656A, published on December 30, 2022, entitled "A Method and Apparatus for I-Frame Rate Control Suitable for Hardware Implementation," discloses a video frame-level rate control method suitable for hardware implementation of I-frames. Compared with CN115550656A, the main technical innovations of this application are as follows.

[0059] First, CN115550656A aims to achieve precise control over the bitrate of I-frames, ensuring that the actual number of encoded bits in the I-frame to be encoded is as close as possible to the target number of encoded bits. The objective of this application is to avoid generating excessively large I-frames during the encoding process, while maintaining as much subjective visual quality as possible between the I-frame and the previously encoded P-frames and B-frames to suppress the breathing effect.

[0060] Second, both CN115550656A and this application contain formulas one through five, and they are written in the same way. However, the meanings of the variable values ​​in the formulas of the two documents are completely different. In CN115550656A, formula five is used to calculate the value of the Lagrange multiplier I(n) required when calculating the target number of coded bits for a given I-frame to be coded. new λ represents the target number of coded bits in the I-frame to be encoded. new This means that the actual number of encoded bits in the I-frame to be encoded should equal the target number of encoded bits I(n). new The Lagrange multipliers required at that time. In this application, Formula 5 is used to calculate the lower limit of the Lagrange multipliers I(n) for the I-frame to be encoded that will not cause rate control abnormalities. new λ represents the upper limit of the target number of encoded bits for the I-frame to be encoded that will not cause rate control anomalies. new This represents the lower bound of the Lagrange multipliers for an I-frame to be encoded that will not cause rate control anomalies.

[0061] Third, in this application, when calculating the lower limit of the Lagrange multiplier of the I-frame to be encoded that will not cause bitrate control abnormalities, the upper limit of the number of encoded bits of the I-frame that will not cause bitrate control abnormalities is first calculated based on the encoding state of the current video sequence (i.e., step S3). This is not present in CN115550656A.

[0062] Fourth, in this application, the Lagrange multiplier λ that suppresses the breathing effect in the I-frame to be encoded is calculated. I The process (i.e. step S5) is also not found in CN115550656A.

[0063] Fifth, in CN115550656A, the value of the Lagrange multiplier λ of the I-frame to be encoded is calculated using Formula 5. new After that, λ can be used. newThe I-frame to be encoded is then encoded. In this application, Formula 5 is used to calculate the lower bound λ of the Lagrange multipliers of the I-frame to be encoded that will not cause abnormal bitrate control. new From now on, only λ new It is used as a lower bound threshold only when the Lagrange multiplier λ of the I-frame to be encoded has a suppressive effect on the breathing effect. I Ratio λ new λ will only be used after hours. new Encode the I-frame to be encoded; otherwise, use λ. I The I-frame to be encoded is encoded (i.e., step S6).

[0064] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A bitrate control method to avoid generating excessively large I-frames, characterized in that, Includes the following steps; Step S1: Perform intra-frame coding mode prediction for each image block to be encoded to obtain all intra-frame coding mode candidates and their corresponding prediction costs for each image block to be encoded. Step S2: Filter the prediction costs corresponding to each candidate intra-coding mode of the image block to be encoded, and select the minimum prediction cost to characterize the intra-coding complexity of the image block to be encoded; then take the sum of the intra-coding complexities of all image blocks in the video frame as the intra-coding complexity of the video frame. Step S3: Based on the encoding status of the current video sequence, calculate the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control abnormalities; Step S4: Based on the actual number of encoded bits, Lagrange multipliers, and intra-frame coding complexity of the previous encoded I-frame, and combining the upper limit of the number of encoded bits and the intra-frame coding complexity of the I-frame to be encoded, calculate the lower limit of the Lagrange multipliers of the I-frame to be encoded; wherein, the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous video frame of the I-frame to be encoded. Step S5: Calculate the Lagrange multiplier that inhibits the breathing effect in the I-frame to be encoded based on the quantization parameters of the previous encoded P-frame or B-frame. Steps S1 to S4 form a group; this group and step S5 may be performed simultaneously or either precede each other. Step S6: Select the larger value among the lower limit of the Lagrange multiplier of the I-frame to be encoded and the Lagrange multiplier of the I-frame to be encoded that has an inhibitory effect on the breathing effect, as the Lagrange multiplier of the I-frame to be encoded.

2. The bitrate control method for avoiding the generation of excessively large I-frames according to claim 1, characterized in that, In step S1, the prediction cost corresponding to each intra-frame coding mode is calculated for each image block to be encoded, and one or more intra-frame coding modes with the lowest prediction cost are selected as intra-frame coding mode candidates for the image block to be encoded; the prediction cost refers to the coding rate distortion cost.

3. The bitrate control method for avoiding the generation of excessively large I-frames according to claim 1, characterized in that, Step S2 calculates at least the intra-frame coding complexity of each I-frame and the video frame preceding each I-frame.

4. The bitrate control method for avoiding the generation of excessively large I-frames according to claim 1, characterized in that, In step S3, the state information of the sliding window used to smooth frame-level bit allocation during the rate control process is also used to calculate the upper limit of the number of encoded bits for an I-frame that will not cause abnormal rate control.

5. The bitrate control method for avoiding the generation of excessively large I-frames according to claim 4, characterized in that, In step S3, the target number of encoded bits R of the sliding window is first calculated. sw R sw =R PicAvg ×(N coded +N SW )-R coded Among them, R PicAvg The average number of target coded bits per video frame, N, is calculated from the average bitrate of the video sequence. coded N is the number of currently encoded video frames in the video sequence. SW R is the size of the sliding window. coded It is the sum of the actual encoded bits of all currently encoded video frames in the video sequence; then according to R sw Calculate the upper limit R of the number of encoded bits for an I-frame that will not cause bitrate control anomalies. Imax ; Where, N I It is the number of I-frames in the sliding window; η is a constant with a value range of (0,1).

6. The bitrate control method for avoiding the generation of excessively large I-frames according to claim 4, characterized in that, Step S4 is derived from Formula 5: Find; where λ new It is the lower bound of the Lagrange multipliers of the I-frame to be encoded; f -1 It is the inverse function of the first function f; I(n) new I(m) represents the upper limit of the number of encoded bits for an I-frame that will not cause rate control anomalies. old This represents the actual number of encoded bits in the previous encoded I-frame; g is the second function; ω n-1 The value represents the intra-frame coding complexity of the preceding encoded video frame, used to replace the intra-frame coding complexity ω of the I-frame to be encoded. n ;ω m λ represents the intra-frame coding complexity of the previous encoded I-frame; old This represents the Lagrange multiplier of the previous encoded I-frame.

7. The bitrate control method for avoiding the generation of excessively large I-frames according to claim 6, characterized in that, The first function f is given by formula one: Formula 1 states that when I-frame encoding is performed on the same video frame using different Lagrange multipliers, the ratio of the actual number of encoded bits obtained after encoding is related to the ratio of the Lagrange multipliers. This relationship is represented by the first function f. The second function g is derived from formula two: Formula 2 states that when using the same Lagrange multipliers to perform I-frame encoding on different video frames, the ratio of the actual number of encoded bits obtained after encoding is related to the ratio of the intra-frame encoding complexity of the video frames. This relationship is represented by the second function g.

8. The bitrate control method for avoiding the generation of excessively large I-frames according to claim 7, characterized in that, Based on the derivation of Formula 1 and Formula 2, Formula 3 is obtained: Further derivation yields Formula 4: Where, λ new It is the Lagrange multiplier of the I-frame to be encoded; I(n) new Replaced by the target number of coded bits of the I-frame to be encoded; ω n The intra-frame coding complexity ω of the previous encoded video frame is derived from the I-frame to be encoded. n-1 By substitution, we obtain Formula 5; let I(n) new λ is the upper limit of the number of encoded bits for the I-frame to be encoded that will not cause rate control anomalies. new This is the lower bound of the Lagrange multipliers of the I-frame to be encoded.

9. The bitrate control method for avoiding the generation of excessively large I-frames according to claim 1, characterized in that, In step S5, the quantization parameter QP of the previous encoded P-frame or B-frame of the I-frame to be encoded is first used. PB The quantization parameter QP, which suppresses the breathing effect, is calculated from the offset δ of the I-frame to be encoded. I The calculation formula is: QP I =QP PB +δ; where δ is an integer offset; then, based on the quantization parameter QP of the I-frame to be encoded, which has a suppressive effect on breathing. I The Lagrange multiplier λ that suppresses the breathing effect in the I-frame to be encoded is calculated. I The calculation formula is: Exp() is an exponential function with the natural logarithm e as its base.

10. A bitrate control device for avoiding the generation of excessively large I-frames, characterized in that, It includes an intra-frame coding mode prediction module, an intra-frame coding complexity calculation module, an I-frame coding bit limit calculation module, an I-frame Lagrange multiplier lower limit calculation module, an I-frame Lagrange multiplier calculation module, and an I-frame Lagrange multiplier selection module; The intra-frame coding mode prediction module is used to perform intra-frame coding mode prediction for each image block to be encoded, and output all intra-frame coding mode candidates and their corresponding prediction costs for each image block to be encoded. The intra-frame coding complexity calculation module is used to filter the prediction cost corresponding to each intra-frame coding mode candidate of the image block to be encoded, and select the minimum prediction cost to characterize the intra-frame coding complexity of the image block to be encoded; then the sum of the intra-frame coding complexities of all image blocks in the video frame is used as the intra-frame coding complexity of the video frame. The I-frame encoding bit limit calculation module is used to calculate the upper limit of the I-frame encoding bit count that will not cause abnormal bit rate control based on the encoding status of the current video sequence. The I-frame Lagrange multiplier lower bound calculation module is used to calculate the lower bound of the Lagrange multiplier of the I-frame to be encoded based on the actual number of encoded bits, the Lagrange multiplier, and the intra-frame coding complexity of the previous encoded I-frame, combined with the upper bound of the number of encoded bits and the intra-frame coding complexity of the I-frame to be encoded; wherein, the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous video frame of the I-frame to be encoded. The I-frame Lagrange multiplier calculation module is used to calculate the Lagrange multiplier of the I-frame to be encoded that has an inhibitory effect on the breathing effect based on the quantization parameters of the previous encoded P-frame or B-frame. The I-frame Lagrange multiplier selection module is used to select the larger value among the lower limit value of the Lagrange multiplier of the I-frame to be encoded and the Lagrange multiplier of the I-frame to be encoded that has an inhibitory effect on the breathing effect, as the Lagrange multiplier of the I-frame to be encoded.

Images