Method and apparatus for inhibiting bit rate control of super-large i-frame

Abstract

The application discloses a code rate control method for inhibiting super-large I frame. According to the coding state of a current video sequence, the upper limit value of the coding bit number of the I frame which will not cause code rate control abnormality is calculated. According to the actual coding bit number, the quantization parameter and the intra-frame coding complexity of the previous coded I frame, in combination with the upper limit value of the coding bit number and the intra-frame coding complexity of the to-be-coded I frame, the lower limit value of the quantization parameter of the to-be-coded I frame is calculated. According to the quantization parameter of the previous coded P frame or B frame, the quantization parameter of the to-be-coded I frame which has inhibiting effect on the breathing effect is calculated. The larger value of the lower limit value of the quantization parameter of the to-be-coded I frame and the quantization parameter of the to-be-coded I frame which has inhibiting effect on the breathing effect is selected as the quantization parameter of the to-be-coded I frame, which is used in the actual coding process of the to-be-coded I frame. The application takes the video frame level quantization parameter as the operation core of the code rate control algorithm, which is more suitable for hardware implementation.

Description

TECHNICAL FIELD

[0001] The present application relates to a video coding technique, and in particular, to a method for rate control to suppress super-large I-frames. BACKGROUND

[0002] Video coding is a technique to represent video information using as few data as possible by compressing the redundancy in video images. Common video coding standards include HEVC (High Efficiency Video Coding, also known as H.265), AVC (Advanced Video Coding, also known as H.264), etc.

[0003] Video coding techniques take image blocks as the most basic coding unit. For example, in HEVC, the basic unit of video coding is a CU (Coding Unit). The CU can be an image block of 64 pixels x 64 pixels, 32 pixels x 32 pixels, 16 pixels x 16 pixels, or 8 pixels x 8 pixels in size. The image block of 64 pixels x 64 pixels in size is also referred to as an LCU (Largest Coding Unit).

[0004] To improve the video compression rate as much as possible, common video coding algorithms often use lossy compression algorithms, which cause the reconstructed video after coding to be different from the original video (resulting in distortion). HEVC and AVC are both lossy compression algorithms. The coding performance of a lossy compression algorithm needs to be measured by the coding bit rate (Rate) and the coding distortion (Distortion) introduced. In the video coding process, the coding bit rate R and the coding distortion D are mutually restrictive and contradictory. Reducing the coding bit rate R often increases the coding distortion D, and reducing the coding distortion D often increases the coding bit rate R. Rate-distortion optimization (RDO) is to reduce the coding distortion D as much as possible under a certain coding bit rate R, or to reduce the coding bit rate R as much as possible under a certain coding distortion D. Common video coding algorithms use a rate-distortion optimization method based on a Lagrange multiplier λ. This method is to obtain the minimum coding rate-distortion cost (RD Cost) J, J = D + λ x R.

[0005] In video encoding, each encoding mode has an encoding distortion D and an encoding bit rate R, and the encoding rate-distortion cost J of each encoding mode can be calculated. In video encoding, the encoding mode with the minimum encoding rate-distortion cost J is selected for encoding to achieve the optimal encoding performance. The process of selecting the optimal encoding mode for each to-be-encoded image block by the rate-distortion optimization method of the encoder is called mode decision of the to-be-encoded image block.

[0006] To remove the information redundancy in the spatial and temporal domains, the video encoding technology uses the intra-frame encoding technique and the inter-frame encoding technique to encode the input video frames. The encoding using only the intra-frame encoding technique is the I frame. The encoding using the intra-frame encoding technique and the inter-frame encoding technique is the P frame and the B frame. The type of the encoded frame is determined by the group of pictures (GOP) parameters set by the user. Figure 1 A common GOP structure of I frames, P frames, and B frames is shown. Since the P frame and the B frame have higher encoding efficiency, the I frame has the function of blocking error propagation, so the periodic encoding of I frames and the encoding of the remaining most video frames as P frames or B frames are generally used. When the I frame is encoded, only the information of the video frame is used for encoding, and the information of other encoded video frames is not referred to. All image blocks in the I frame are encoded as intra-frame encoding blocks. When the P frame and the B frame are encoded, not only the information of the video frame is used for encoding, but also the information of one or more other encoded video frames is referred to. The image blocks in the P frame and the B frame can be encoded as intra-frame encoding blocks or inter-frame encoding blocks, depending on which encoding rate-distortion cost J is smaller.

[0007] In real life, the channel bandwidth capacity for transmitting video is limited. If the encoding bit rate of the video is too high and exceeds the channel bandwidth capacity, video transmission congestion or even packet loss will occur. If the encoding bit rate of the video is too low, the channel bandwidth cannot be fully utilized, and higher video quality cannot be achieved. Therefore, it is necessary to use the rate control technology to control the output bit rate of the video encoder to match the channel bandwidth capacity.

[0008] The purpose of rate control is to adjust the encoding parameters of the video encoder so that the output bit rate of the video encoder is equal to the target bit rate set in advance, and the encoding distortion D is reduced as much as possible to improve the video encoding quality. In common rate control algorithms, the rate control task is generally achieved through two links of target bit allocation and target bit control.

[0009] The target bit allocation is generally performed at three levels, i.e. GOP (Group Of Picture, i.e. a set of time-continuous video frames) level, video frame level and image block level. The target bit allocation at GOP level is to obtain the target coding bits of a GOP according to the target coding bit rate of the video. The target bit allocation at video frame level is to obtain the target coding bits of each video frame in the GOP according to the target coding bits of the GOP. The target bit allocation at image block level is to obtain the target coding bits of each image block in a video frame according to the target coding bits of the video frame and the bit allocation weight of each image block in the video frame.

[0010] The target bit control is generally performed at video frame level and image block level. At video frame level, the Lagrange multiplier λ and the quantization parameter QP at video frame level are calculated according to the target coding bits of the video frame. At image block level, the Lagrange multiplier λ and the quantization parameter QP at image block level of a certain image block are calculated according to the target coding bits of the image block, the Lagrange multiplier λ and the quantization parameter QP at video frame level of the video frame to which the image block belongs, and the Lagrange multiplier λ and the quantization parameter QP at image block level of the previous coded image block, which are used in the video encoding process of the image block.

[0011] Both the Lagrange multiplier λ and the quantization parameter QP are divided into video frame level and image block level. The Lagrange multiplier λ and the quantization parameter QP at video frame level are used in the target bit allocation and the target bit control at video frame level (i.e. the frame level rate control), and the Lagrange multiplier λ and the quantization parameter QP at image block level are used in the target bit allocation and the target bit control at image block level (i.e. the block level rate control). The Lagrange multiplier λ at video frame level is also referred to as the frame level Lagrange multiplier, the frame level Lagrange multiplier, or the Lagrange multiplier of the video frame. The Lagrange multiplier λ at image block level is also referred to as the block level Lagrange multiplier, the block level Lagrange multiplier, or the Lagrange multiplier of the image block. The same applies to the quantization parameter.

[0012] Due to the large amount of computation of the video encoding algorithm, in order to improve the video encoding speed and realize real-time encoding and transmission, the industry generally uses an application specific integrated circuit (ASIC) to accelerate the video encoding process. The application specific integrated circuit for accelerating the video encoding process is generally referred to as a hardware video encoder.

[0013] When encoding video monitoring and video conferencing scenes, periodic visual flicker is often observed in the encoded video stream. The visual flicker is more obvious in the stationary area of the scene. This periodic visual flicker phenomenon is called "breathing effect". The "breathing effect" is related to the periodic I frame in the video stream. Since the I frame only uses the intra-frame encoding technique, it does not refer to the information of the previously encoded video frame, so the distortion size and distortion mode of the I frame are quite different from the previously encoded P frame or B frame. When watching the decoded video stream, this distortion difference will cause the human eye to observe the visual flicker phenomenon. In the video encoding process, the I frame is generally periodically encoded, and the visual flicker observed by the human eye will occur periodically, that is, the "breathing effect" is generated. The "breathing effect" will greatly affect the subjective visual experience of the viewer, so the "breathing effect" should be suppressed during encoding.

[0014] In order to suppress the breathing effect in the process of video encoding, the most common method is to set the QP of the I frame near the QP of the previously encoded P frame or B frame, that is, I frame QP = previously encoded P frame or B frame QP + offset δ. Wherein, let the offset δ be an integer with a small absolute value, usually between -2 and 2 (including the numbers). In this way, the distortion size of the I frame is close to the distortion size of the previously encoded P frame or B frame, effectively suppressing the breathing effect. In this way, after calculating the QP of the to-be-encoded I frame according to the QP of the previously encoded P frame or B frame and the offset δ, the Lagrange multiplier λ of the to-be-encoded I frame can be calculated according to the relationship formula between QP and Lagrange multiplier λ, which is used in the actual encoding process of the to-be-encoded I frame.

[0015] Although the above method can effectively suppress the breathing effect, for some video scenes, especially the video scenes with static content in time domain and complex texture in spatial domain, it is easy to encode an oversized I-frame, i.e. the encoding bit number of the I-frame is much larger than that of the P-frame or B-frame. This not only causes severe instantaneous code rate fluctuation, but also makes the actual average code rate much larger than the preset target code rate, resulting in abnormal code rate control. For such video scenes, since the content is static in time domain, the QP of the P-frame and B-frame encoded by using the inter-frame coding technique is small. When the I-frame is encoded, in order to suppress the breathing effect, the QP of the I-frame is set near the QP of the previous encoded P-frame or B-frame, so the QP of the I-frame is also a small value. Meanwhile, since the video scene has complex texture in spatial domain, when the I-frame is encoded by using the intra-frame coding technique, the oversized I-frame is encoded by using the small QP, which is much larger than the P-frame or B-frame, causing severe instantaneous code rate fluctuation, and even making the actual average code rate much larger than the preset target code rate, resulting in invalid code rate control. In other words, in this case, although the QP value of the to-be-encoded I-frame is set near the QP value of the previous encoded P-frame or B-frame, the breathing effect can be suppressed, but the oversized I-frame is generated, resulting in abnormal code rate control.

[0016] It is desired in the industry that there is a technology that, in the code rate control process of video encoding, when the oversized I-frame is not encoded, the QP of the I-frame is set near the QP of the previous encoded P-frame or B-frame to suppress the breathing effect and improve the subjective visual quality; only when the oversized I-frame is encoded, the size of the I-frame is suppressed to avoid abnormal code rate control.

[0017] At present, there are some technical solutions to avoid the generation of oversized I-frame in the video encoding process.

[0018] One solution is that the QP of the I-frame is still calculated from the QP of the previous encoded P-frame or B-frame, but the offset δ is selected as a large positive integer. In this way, since the QP of the I-frame is much larger than that of the P-frame or B-frame, the size of the encoded I-frame is small, and the encoding bit number of the I-frame is not much larger than that of the P-frame or B-frame. By using this solution for encoding, the occurrence of oversized I-frame can be effectively avoided, the instantaneous code rate fluctuation is small, and the average code rate control is accurate. However, when this solution is used, for the video encoding scene that does not originally encode the oversized I-frame, since the QP of the I-frame is made much larger than that of the P-frame or B-frame without distinction, the coding quality of the I-frame is poor, the coding quality of the P-frame and B-frame directly or indirectly referencing the I-frame is reduced, and the coding efficiency of the entire video stream is greatly reduced.

[0019] Another solution is that the QP of the I frame is irrelevant to the QP of the previous coded P frame or B frame, but is calculated completely by the target coding bit number of the I frame. With this solution, the appearance of the super large I frame can also be effectively avoided, but the QP of the I frame is easily greatly different from the QP of the previous coded P frame or B frame, resulting in a large difference between the subjective visual quality of the I frame and the subjective visual quality of the P frame or B frame, causing serious visual flicker and greatly reducing the subjective visual quality of the entire video stream. SUMMARY

[0020] The technical problem to be solved by the present application is how to make the code rate control of video coding keep the subjective visual quality of the I frame consistent with the subjective visual quality of the previous coded P frame or B frame as much as possible to suppress the breathing effect, and avoid the appearance of the super large I frame in the coding process.

[0021] To solve the above technical problem, the present application proposes a code rate control method for suppressing super large I frame, comprising the following steps. Step S1: intra-frame coding mode prediction is performed on the to-be-coded image block, that is, the prediction cost corresponding to each intra-frame coding mode is calculated for the to-be-coded image block, only one or more intra-frame coding modes with the lowest prediction cost are reserved as intra-frame coding mode candidates, and the intra-frame coding mode prediction cost corresponding to each intra-frame coding mode candidate is also reserved. Step S2: the smallest intra-frame coding mode prediction cost is selected from the intra-frame coding mode prediction costs corresponding to all the intra-frame coding mode candidates of the to-be-coded image block to represent the intra-frame coding complexity of the to-be-coded image block, and the sum of the intra-frame coding complexities of all the image blocks in the video frame is taken as the intra-frame coding complexity of the video frame. Step S3: according to the coding state of the current video sequence, the upper limit value of the coding bit number of the I frame that will not cause abnormal code rate control is calculated. The steps S1 and S2 constitute a group, and the group is performed before or at the same time as the step S3, or in any order. Step S4: according to the actual coding bit number, the quantization parameter and the intra-frame coding complexity of the previous coded I frame, in combination with the upper limit value of the coding bit number and the intra-frame coding complexity of the to-be-coded I frame, the lower limit value of the quantization parameter of the to-be-coded I frame is calculated; wherein the intra-frame coding complexity of the to-be-coded I frame reuses the intra-frame coding complexity of the previous coded video frame of the to-be-coded I frame. Step S5: according to the quantization parameter of the previous coded P frame or B frame, the quantization parameter of the to-be-coded I frame that has a suppressing effect on the breathing effect is calculated. The step S5 is performed before or at the same time as the previous four steps, or in any order. Step S6: the larger one of the lower limit value of the quantization parameter of the to-be-coded I frame and the quantization parameter of the to-be-coded I frame that has a suppressing effect on the breathing effect is selected as the quantization parameter of the to-be-coded I frame, which is used in the actual coding process of the to-be-coded I frame.

[0022] Preferably, in the step S2, only the intra-frame coding complexity of each I frame and the intra-frame coding complexity of the previous video frame of each I frame are calculated.

[0023] Furthermore, in step S3, the state information of the sliding window used to smooth the target bit allocation at the video frame level during the bitrate control process is used to calculate the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control abnormalities.

[0024] Furthermore, in step S3, the target number of encoded bits R of the sliding window is first calculated. sw R sw =R PicAvg ×(N coded +N Sw )-R coded Among them, R PicAvg The average number of target coded bits per video frame, N, is calculated from the average bitrate of the video sequence. coded N is the number of currently encoded video frames in the video sequence. SW R is the size of the sliding window. coded It is the sum of the actual encoded bits of all currently encoded video frames in the video sequence; then, according to R... sw Calculate the upper limit R of the number of encoded bits for an I-frame that will not cause bitrate control anomalies. Imax ; Where, N I η is the number of I-frames in the sliding window; η is a constant, 0 < η ≤ 1.

[0025] Furthermore, step S4 is derived from formula five: Find; where QP new f is the video frame-level quantization parameter of the I-frame to be encoded; -1 It is the inverse function of the first function f; I(n) new I(m) represents the target number of encoded bits for the I-frame to be encoded. old This represents the actual number of encoded bits in the previous encoded I-frame; g is the second function; ω n-1 The value represents the intra-frame coding complexity of the preceding encoded video frame, used to replace the intra-frame coding complexity ω of the I-frame to be encoded. n ;ω m QP represents the intra-frame coding complexity of the previous encoded I-frame. old This represents the video frame-level quantization parameter of the previous encoded I-frame; in Formula 5, let I(n)... new The QP is the upper limit of the number of encoded bits for the I-frame to be encoded that will not cause rate control anomalies. new This refers to the lower limit of the quantization parameters for the I-frame to be encoded that will not cause abnormal bitrate control.

[0026] Furthermore, the first function f is given by Formula 1: obtained by the same video frame level quantization parameter, and the ratio of the actual encoding bit number obtained by the different video frames to the difference of the video frame level quantization parameter used by the different video frames.

[0027] Further, the second function g is represented by Equation Two: obtained by the same video frame level quantization parameter, and the ratio of the actual encoding bit number obtained by the different video frames to the difference of the video frame level quantization parameter used by the different video frames.

[0028] Further, based on Equation One and Equation Two, Equation Three is obtained: Further, Equation Four is obtained by further derivation: wherein ω n represents the intra-frame encoding complexity of the nth frame, and the intra-frame encoding complexity ω n-1 of the (n-1)th frame adjacent to the nth frame is used to replace, and Equation Five is obtained.

[0029] Further, in the step S5, the calculation formula used is: QP I = QP PB + δ; wherein QP I represents the quantization parameter of the to-be-encoded I frame having inhibition effect on the breathing effect, QP PB represents the quantization parameter of the previous encoded P frame or B frame of the to-be-encoded I frame, and δ represents an integer offset.

[0030] The application further provides a code rate control device for inhibiting super-large I frames, comprising an intra-frame coding mode prediction module, an intra-frame coding complexity calculation module, an I frame coding bit number upper limit value calculation module, an I frame quantization parameter lower limit value calculation module, an I frame quantization parameter second calculation module and an I frame quantization parameter selection module. The intra-frame coding mode prediction module is used for performing intra-frame coding mode prediction on a to-be-coded image block to obtain all intra-frame coding mode candidates of the to-be-coded image block and corresponding intra-frame coding mode prediction costs. The intra-frame coding complexity calculation module is used for selecting the smallest intra-frame coding mode prediction cost to represent the intra-frame coding complexity of the to-be-coded image block, and taking the sum of intra-frame coding complexities of all image blocks in a video frame as the intra-frame coding complexity of the video frame. The I frame coding bit number upper limit value calculation module is used for calculating the upper limit value of the coding bit number of the I frame which will not cause code rate control abnormality according to the coding state of the current video sequence. The I frame quantization parameter lower limit value calculation module is used for calculating the lower limit value of the quantization parameter of the to-be-coded I frame according to the actual coding bit number, the quantization parameter and the intra-frame coding complexity of the previous coded I frame, in combination with the upper limit value of the coding bit number and the intra-frame coding complexity of the to-be-coded I frame, wherein the intra-frame coding complexity of the to-be-coded I frame reuses the intra-frame coding complexity of the previous coded video frame of the to-be-coded I frame. The I frame quantization parameter second calculation module is used for calculating the quantization parameter of the to-be-coded I frame which has inhibiting effect on the breathing effect according to the quantization parameter of the previous coded P frame or B frame. The I frame quantization parameter selection module is used for selecting the greater value between the lower limit value of the quantization parameter of the to-be-coded I frame and the quantization parameter of the to-be-coded I frame which has inhibiting effect on the breathing effect as the quantization parameter of the to-be-coded I frame.

[0031] The application has the following technical effects: (1) taking the video frame level quantization parameter as the operation core of the code rate control algorithm, the operation of the quantization parameter is mainly addition and subtraction, which is more suitable for hardware implementation; (2) no image pre-processing stage is needed, the hardware cost is small, and the implementation cost is low; (3) the lower limit value of the quantization parameter of the to-be-coded I frame which will not cause code rate control abnormality is calculated by using the information of the coded video frame and the coding state of the video sequence, and the quantization parameter of the to-be-coded I frame which has inhibiting effect on the breathing effect is constrained by the lower limit value, so that the generation of super-large I frames is avoided on the basis of effectively inhibiting the breathing effect. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 Fig. 1 is a schematic diagram of a common image group structure of I frames, P frames and B frames.

[0033] Figure 2 Fig. 2 is a flowchart of the code rate control method for inhibiting super-large I frames provided by the application.

[0034] Figure 3It is the structure schematic view of the code rate control device for inhibiting super large I frame proposed in the application.

[0035] The figure mark explanation: 1 is the intra-frame coding mode prediction module, 2 is the intra-frame coding complexity calculation module, 3 is the I frame coding bit number upper limit value calculation module, 4 is the I frame quantization parameter lower limit value calculation module, 5 is the I frame quantization parameter second calculation module, 6 is the I frame quantization parameter selection module. DETAILED DESCRIPTION

[0036] Please refer to Figure 2 The code rate control method for inhibiting super large I frame proposed in the application comprises the following steps.

[0037] Step S1: intra-frame coding mode prediction is performed on the to-be-coded image block, that is, the prediction cost corresponding to each intra-frame coding mode is calculated for the to-be-coded image block, only one or more intra-frame coding modes with the lowest prediction cost are reserved as intra-frame coding mode candidates, and the intra-frame coding mode prediction cost corresponding to each intra-frame coding mode candidate is also reserved. This step is a step in the existing video coding process, and intra-frame coding mode prediction is performed on all image blocks of I frames, P frames and B frames.

[0038] In a hardware video encoder, in order to reduce cost and power consumption, when a rate-distortion optimization method is used to select a mode for a to-be-coded image block, the mode selection process is generally divided into an encoding mode prediction stage and an encoding mode rate-distortion optimization stage. The encoding mode prediction stage is completed by an intra-frame or inter-frame coding mode prediction module, and the encoding mode rate-distortion optimization stage is completed by an intra-frame or inter-frame coding mode rate-distortion optimization module.

[0039] In essence, both the coding mode prediction stage and the coding mode rate-distortion optimization stage are to screen the coding mode of the to-be-coded image block by using the rate-distortion optimization method. The difference between the two stages mainly lies in the accuracy of the calculation method of the coding rate-distortion cost J. In the coding mode prediction stage, a coding rate-distortion cost J calculation method with less calculation amount and lower accuracy is generally used; the coding distortion D is generally calculated by using the SAD (Sum of Absolute Difference) algorithm or the SATD (Sum of Absolute Transformed Difference) algorithm; the coding bit rate R is generally calculated by using a simplified bit rate estimation algorithm, such as the exponential Golomb algorithm; and the coding mode prediction stage generally does not introduce transform and quantization operations. In the coding mode rate-distortion optimization stage, a coding rate-distortion cost J calculation method with more calculation amount and higher accuracy is generally used; the coding distortion D is generally calculated by using the SSD (Sum of Squared Differences) algorithm; the coding bit rate R is generally calculated by using the CABAC (Context Adaptive Binary Arithmatic Coding) algorithm or the CAVLC (Context Adaptive Variable Length Coding) algorithm; and the coding mode rate-distortion optimization stage generally introduces transform and quantization operations. Conventionally, the small calculation amount and low-precision coding rate-distortion cost J calculated in the coding mode prediction stage is referred to as the prediction cost of the coding mode.

[0040] When encoding the to-be-coded image block, the coding mode prediction stage performs preliminary screening on a large number of coding modes, and selects one or several coding mode candidates with lower prediction cost by calculating the prediction cost of each coding mode. The subsequent coding mode rate-distortion optimization stage then performs accurate calculation on the prediction cost of the coding mode candidates to obtain the accurate coding rate-distortion cost J of the candidate coding mode, and finally the mode selection module selects the coding mode with the minimum coding rate-distortion cost J as the optimal coding mode of the to-be-coded image block.

[0041] Step S2: Among the intra-coding mode prediction costs corresponding to all intra-coding mode candidates for the image block to be encoded, the intra-coding mode prediction cost with the smallest value is selected to characterize the intra-coding complexity of the image block to be encoded. Then, the sum of the intra-coding complexities of all image blocks within the video frame is taken as the intra-coding complexity of the video frame. Preferably, this step only calculates the intra-coding complexity of each I-frame and the video frame preceding each I-frame, thereby serving the subsequent I-frames. This step reuses the calculation results of intra-coding mode prediction (step S1), does not require the introduction of an additional image preprocessing stage, and has very low hardware implementation cost.

[0042] Step S3: Based on the encoding state of the current video sequence, calculate the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control anomalies. For example, use the state information of the sliding window (SW) used to smooth the target bit allocation at the video frame level during bitrate control to calculate the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control anomalies.

[0043] During video encoding, when a video frame is encoded, the actual number of encoded bits often differs from the target number of encoded bits calculated before encoding. This error can be smoothed out within a sliding window consisting of multiple subsequent consecutive video frames to reduce instantaneous bitrate fluctuations. The number of consecutive video frames included in this sliding window is called the size of the sliding window, and the sum of the target number of encoded bits for all video frames within the sliding window is called the target number of encoded bits for the sliding window. The target number of encoded bits for the sliding window can be calculated using the following formula: R sw =R PicAvg ×(N coded +N SW )-R coded Among them, R sw R is the target number of encoded bits for the sliding window. PicAvg The average number of target coded bits per video frame, N, is calculated from the average bitrate of the video sequence. coded N is the number of currently encoded video frames in the video sequence. Sw R is the size of the sliding window. coded It is the sum of the actual encoded bits of all currently encoded video frames in the video sequence. R PicAvg N coded N Sw R coded This represents the current encoding state of the video sequence. It can be seen that as the encoding process progresses, R... sw The value of R changes continuously depending on the encoding state of the video sequence. This step calculates R based on the encoding state of the current video sequence before encoding each I-frame. sw The value, and based on Rsw The value calculation of the upper limit value of the encoded bit number of the I frame that does not cause the abnormality of the code rate control. For example, the following calculation formula is used: wherein, R Imax is the upper limit value of the encoded bit number of the I frame that does not cause the abnormality of the code rate control; R sw is the target encoded bit number of the sliding window; N I is the number of the I frames in the sliding window; and η is in the range of 0 < η ≤ 1, and the specific value is related to the encoding parameters of the video sequence, and can be set as 0.67.

[0044] The steps S1 and S2 constitute a group. The order of the group and the step S3 is not strictly limited, or they are performed simultaneously, or any one is prior.

[0045] Step S4: According to the actual encoded bit number, the quantization parameter and the intra-frame encoding complexity of the previous encoded I frame, and in combination with the upper limit value of the encoded bit number and the intra-frame encoding complexity of the to-be-encoded I frame, the lower limit value of the quantization parameter of the to-be-encoded I frame is calculated. The intra-frame encoding complexity of the to-be-encoded I frame is reused with the intra-frame encoding complexity of the previous encoded video frame of the to-be-encoded I frame. Since the upper limit value of the encoded bit number of the to-be-encoded I frame that does not cause the abnormality of the code rate control is combined in the calculation process, the lower limit value of the quantization parameter of the to-be-encoded I frame that does not cause the abnormality of the code rate control is obtained. The step S4 must be performed after the steps S1, S2 and S3.

[0046] For example, the lower limit value of the quantization parameter of the to-be-encoded I frame is obtained by the formula five: wherein, QP new is the video frame-level quantization parameter of the to-be-encoded I frame. f -1 is the inverse function of the first function f. I(n) new is the target encoded bit number of the to-be-encoded I frame. I(m) old represents the actual encoded bit number of the previous encoded I frame. g is the second function. ω n-1 represents the intra-frame encoding complexity of the previous encoded video frame of the to-be-encoded I frame, which is used to replace the intra-frame encoding complexity ω n of the to-be-encoded I frame. ω m represents the intra-frame encoding complexity of the previous encoded I frame. QP old represents the video frame-level quantization parameter of the previous encoded I frame. In the formula five, I(n) new is the upper limit value of the encoded bit number of the to-be-encoded I frame that does not cause the abnormality of the code rate control, and the calculated QP new is the lower limit value of the quantization parameter of the to-be-encoded I frame. In this step, the video frame-level quantization parameter QP newThe process, and the replacement operation that utilizes the temporal correlation of consecutive video frames to perform intra-frame coding complexity replacement on the I-frame to be coded, are innovations of this invention.

[0047] Please see Figure 1 Frame m and frame n are two consecutive I-frames, and there may be zero or more P-frames or B-frames between them. Let I(m) be the frame number. old This indicates that the m-th frame uses QP. old The actual number of encoded bits after encoding is denoted by I(m). new This indicates that the m-th frame uses QP. new The actual number of encoded bits after encoding is denoted by I(n). old This indicates that the nth frame uses QP. old The actual number of encoded bits after encoding is denoted by I(n). new This indicates that the nth frame uses QP. new The actual number of encoded bits after encoding, where QP old and QP new This represents two different video frame-level quantization parameters. ω is used to represent these parameters. m Let ω represent the intra-frame coding complexity of the m-th frame. n This represents the intra-frame coding complexity of the nth frame.

[0048] The first function f is given by formula one: The physical meaning of Formula 1 is that when I-frame encoding the same video frame using different video frame-level quantization parameters, the ratio of the actual number of encoded bits obtained is related to the difference in the video frame-level quantization parameters. This relationship can be represented by the first function f. The first function f characterizes the relationship between the ratio of the different actual number of encoded bits obtained for the same video frame using different video frame-level quantization parameters during I-frame encoding and the difference in the different video frame-level quantization parameters used in that video frame. The specific expression of the first function f can be obtained by calculating the actual parameters of the encoded I-frame and fitting Formula 1 to the function; it will not be elaborated here.

[0049] The second function g is derived from formula two: The physical meaning of Formula Two is that the ratio of the actual coding bit numbers of different video frames after I-frame coding using the same video frame level quantization parameter is related to the ratio of the intra-frame coding complexities of the video frames, and this relationship can be represented by a second function g. The second function g represents the relationship between the ratio of the actual coding bit numbers of different video frames and the ratio of the intra-frame coding complexities of the video frames when the same video frame level quantization parameter is used for I-frame coding. The specific expression of the second function g can be obtained by performing operations on the actual parameters of the coded I-frames and function fitting on Formula Two, which will not be described herein.

[0050] The first function f and the second function g in Formulas One and Two are known function relationships, which are obtained by I-frame coding experiments on a large number of different video streams in advance and are saved in the video encoder in the form of function fitting.

[0051] Based on Formulas One and Two, Formula Three can be obtained: Further derivation can obtain Formula Four: wherein the function f -1 is the inverse function of the first function f, and since the first function f is known, the function f -1 is also known. In Formula Four, the mth frame is a coded I-frame, and its coding process has been completed, so its corresponding actual coding bit number I(m) old , the video frame level quantization parameter QP old used during coding, and the intra-frame coding complexity ω m are all known quantities. The nth frame is a to-be-coded I-frame, and I(n) new represents the expected target coding bit number and is a known quantity; ω n represents the intra-frame coding complexity of the nth frame, and since the nth frame has not been coded, it is an unknown quantity; QP new represents the video frame level quantization parameter needed to be used when the actual coding bit number of the nth frame is the expected target coding bit number I(n) new , and is an unknown quantity that needs to be obtained. In order to calculate the video frame level quantization parameter QP new of the to-be-coded I-frame by using Formula Four, the present application uses the intra-frame coding complexity ω n-1 of the coded (n-1)th frame adjacent to the nth frame to replace the intra-frame coding complexity ω nSince the sampling times of frame n-1 and frame n are very close, the content of the two video frames is very similar. The error in the replacement operation of the intra-frame coding complexity is very small, and it will not affect the video frame-level quantization parameter QP of the uncoded I-frame. new This has a significant impact on the calculation. Thus, since the encoding of the (n-1)th frame has already been completed, its intra-frame coding complexity ω... n-1 It is a known quantity. In Formula 4, let ω n-1 Replace ω n This leads to Formula 5. In Formula 5, let I(n)... new The QP is the upper limit of the number of encoded bits for the I-frame to be encoded that will not cause rate control anomalies. new This refers to the lower limit of the quantization parameters for the I-frame to be encoded that will not cause abnormal bitrate control.

[0052] Step S5: Calculate the quantization parameters of the I-frame to be encoded that have an inhibitory effect on the breathing effect based on the quantization parameters of the previous encoded P-frame or B-frame.

[0053] The calculation formula used in this step is, for example: QP I =QP PB +δ. Where QP I QP represents the quantization parameter that suppresses the breathing effect in the I-frame to be encoded. PB This represents the quantization parameter of the preceding encoded P-frame or B-frame to be encoded, where δ represents a small integer offset, such as -2≤δ≤2. This allows the distortion of the I-frame to be close to that of the preceding encoded P-frame or B-frame, effectively suppressing the breathing effect.

[0054] The order of step S5 with the previous four steps is not strictly limited; they can be performed simultaneously or arbitrarily placed before each other.

[0055] Step S6: Select the larger value among the lower limit of the quantization parameters of the I-frame to be encoded and the quantization parameters of the I-frame to be encoded that have the effect of suppressing the breathing effect, and use them as the quantization parameters of the I-frame to be encoded for the actual encoding process of the I-frame to be encoded.

[0056] The quantization parameter QP of the I-frame to be encoded, which has an inhibitory effect on the breathing effect, is calculated based on step S5. I While it can effectively suppress the breathing effect, in certain coding scenarios, it may result in excessively large I-frames, leading to abnormal bitrate control. Therefore, the quantization parameter QP, which has a suppressive effect on the breathing effect of the I-frame to be encoded, is crucial. I ≥ The lower limit of the quantization parameter QP that will not cause bitrate control anomalies new When selecting QP I The actual coding process used for the I-frame to be coded can effectively suppress the breathing effect. When QPI < QP new QP new The actual encoding process for the I frame to be encoded can effectively avoid the generation of the super-large I frame. In this way, the desired goal of the present application can be achieved, that is, in the code rate control process of video encoding, when a super-large I frame is not encoded, the QP of the I frame is set near the QP of the previous encoded P frame or B frame to suppress the breathing effect and improve the subjective visual quality; only when a super-large I frame is encoded, the size of the I frame is suppressed to avoid abnormal code rate control.

[0057] Referring to Figure 3 The code rate control device for suppressing the super-large I frame provided by the present application comprises an intra-frame encoding mode prediction module 1, an intra-frame encoding complexity calculation module 2, an I frame encoding bit number upper limit value calculation module 3, an I frame quantization parameter lower limit value calculation module 4, an I frame quantization parameter second calculation module 5, and an I frame quantization parameter selection module 6. Figure 3 The device corresponds to Figure 2 The method.

[0058] The intra-frame encoding mode prediction module 1 is used for intra-frame encoding mode prediction of the to-be-encoded image block to obtain all intra-frame encoding mode candidates of the to-be-encoded image block and the corresponding intra-frame encoding mode prediction cost.

[0059] The intra-frame encoding complexity calculation module 2 is used for selecting the smallest intra-frame encoding mode prediction cost to represent the intra-frame encoding complexity of the to-be-encoded image block; and then the sum of the intra-frame encoding complexities of all image blocks in the video frame is taken as the intra-frame encoding complexity of the video frame.

[0060] The I frame encoding bit number upper limit value calculation module 3 is used for calculating the upper limit value of the encoding bit number of the I frame that will not cause abnormal code rate control according to the encoding state of the current video sequence.

[0061] The I frame quantization parameter lower limit value calculation module 4 is used for calculating the lower limit value of the quantization parameter of the to-be-encoded I frame according to the actual encoding bit number, the quantization parameter and the intra-frame encoding complexity of the previous encoded I frame, in combination with the upper limit value of the encoding bit number and the intra-frame encoding complexity of the to-be-encoded I frame. The intra-frame encoding complexity of the to-be-encoded I frame is multiplexed with the intra-frame encoding complexity of the previous video frame of the to-be-encoded I frame.

[0062] The I frame quantization parameter second calculation module 5 is used for calculating the quantization parameter of the to-be-encoded I frame that has a suppressing effect on the breathing effect according to the quantization parameter of the previous encoded P frame or B frame.

[0063] The I frame quantization parameter selection module 6 is used for selecting the lower limit value of the quantization parameter of the I frame to be encoded, and the larger value in the quantization parameter of the I frame to be encoded that has an inhibitory effect on the breathing effect as the quantization parameter of the I frame to be encoded.

[0064] The present application proposes a code rate control method for inhibiting super-large I frames. The method uses the intra-frame coding mode prediction cost generated in the video frame coding process to represent the intra-frame coding complexity of the video frame, without increasing the image preprocessing stage, with small hardware overhead and low implementation cost. The method uses the information of the coded video frame and the coding state of the video sequence to calculate the lower limit value of the quantization parameter of the I frame to be encoded that will not cause abnormal code rate control, and uses it to constrain the quantization parameter of the I frame to be encoded that has an inhibitory effect on the breathing effect. On the basis of effectively inhibiting the breathing effect, the generation of super-large I frames is avoided, which not only improves the subjective visual quality of the video, but also ensures the normality of the code rate control process.

[0065] The present application is similar to the technical target of the Chinese invention patent application "Code rate control method and device for avoiding the generation of super-large I frames" with patent application number 202310216093.0 and patent application date March 7, 2023, both of which are based on effectively inhibiting the breathing effect to avoid the generation of super-large I frames. However, the technical solutions adopted by the two are significantly different. In 202310216093.0, the core of all formula derivation and operation is based on the video frame level Lagrange multiplier. In the present application, the core of all formula derivation and operation is based on the video frame level quantization parameter. If the video frame level Lagrange multiplier is used as the operation core, the operation on the Lagrange multiplier will introduce more multiplication and division operations. Due to the wide range of values, high precision requirements and large bit width occupation of the Lagrange multiplier, the bit width of the multiplier and divider will be large when implemented in hardware, and the storage space cost and access bandwidth cost generated by storing and accessing the Lagrange multiplier will also be high. After changing the operation core to the video frame level quantization parameter in the present application, the operation on the quantization parameter is mainly addition and subtraction operation, avoiding a large number of multiplication and division operations. And because the quantization parameter has a small value range and small bit width occupation, the storage space cost and access bandwidth cost generated by storing and accessing the quantization parameter are also small when implemented in hardware. Therefore, compared with 202310216093.0, the operation cost, storage cost and bandwidth cost of the present application are lower, and it is more suitable for hardware implementation.

[0066] The above is only a preferred embodiment of the present application and is not intended to limit the present application. Those skilled in the art can make various changes and modifications to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A bitrate control method for suppressing excessively large I-frames, characterized in that, Includes the following steps; Step S1: Perform intra-frame coding mode prediction on the image block to be coded, that is, calculate the prediction cost corresponding to each intra-frame coding mode for the image block to be coded, and retain only one or more intra-frame coding modes with the lowest prediction cost as intra-frame coding mode candidates, while retaining the intra-frame coding mode prediction cost corresponding to each intra-frame coding mode candidate. Step S2: Among the intra-coding mode prediction costs corresponding to all intra-coding mode candidates of the image block to be encoded, select the intra-coding mode prediction cost with the smallest value to characterize the intra-coding complexity of the image block to be encoded, and take the sum of the intra-coding complexities of all image blocks in the video frame as the intra-coding complexity of the video frame. Step S3: Based on the encoding state of the current video sequence, calculate the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control abnormalities; specifically, use the state information of the sliding window used to smooth the target bit allocation at the video frame level during bitrate control to calculate the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control abnormalities. Steps S1 and S2 form a group, and this group can be performed in the order of step S3, either simultaneously or either first or last. Step S4: Based on the actual number of encoded bits, quantization parameters, and intra-frame coding complexity of the previous encoded I-frame, and combined with the upper limit of the number of encoded bits and the intra-frame coding complexity of the I-frame to be encoded, calculate the lower limit of the quantization parameters of the I-frame to be encoded; wherein, the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous encoded video frame of the I-frame to be encoded. Step S5: Calculate the quantization parameters of the I-frame to be encoded that have an inhibitory effect on the breathing effect based on the quantization parameters of the previous encoded P-frame or B-frame. The order of step S5 with the previous four steps can be either simultaneous or any of them can precede it. Step S6: Select the larger value among the lower limit of the quantization parameters of the I-frame to be encoded and the quantization parameters of the I-frame to be encoded that have the effect of suppressing the breathing effect, and use them as the quantization parameters of the I-frame to be encoded for the actual encoding process of the I-frame to be encoded.

2. The bitrate control method for suppressing excessively large I-frames according to claim 1, characterized in that, In step S2, the intra-frame coding complexity is calculated only for each I-frame and the video frame preceding each I-frame.

3. The bitrate control method for suppressing excessively large I-frames according to claim 1, characterized in that, In step S3, the target number of encoded bits for the sliding window is first calculated. , ;in, It is the average target coded bits per video frame, calculated from the average bitrate of the video sequence. It is the number of frames currently encoded in the video sequence. It is the size of the sliding window. It is the sum of the actual encoded bits of all currently encoded video frames in the video sequence; then according to Calculate the upper limit of the number of encoded bits for an I-frame that will not cause bitrate control anomalies. ; ;in, η is the number of I-frames in the sliding window; η is a constant, 0 < η ≤ 1.

4. The bitrate control method for suppressing excessively large I-frames according to claim 1, characterized in that, Step S4 is derived from Formula 5: Find; where, These are the video frame-level quantization parameters of the I-frame to be encoded; It is the inverse function of the first function f; The target number of encoded bits for the I-frame to be encoded; This represents the actual number of encoded bits in the previous encoded I-frame; g is the second function; This represents the intra-frame coding complexity of the preceding encoded video frame, used to replace the intra-frame coding complexity of the I-frame to be encoded. ; This represents the intra-frame coding complexity of the previous encoded I-frame; This represents the video frame-level quantization parameter of the previous encoded I-frame; in Formula 5, let... The upper limit of the number of encoded bits for the I-frame to be encoded that will not cause rate control anomalies is calculated. This refers to the lower limit of the quantization parameters for the I-frame to be encoded that will not cause abnormal bitrate control.

5. The bitrate control method for suppressing excessively large I-frames according to claim 4, characterized in that, The first function f is given by formula one: The first function f represents the relationship between the ratio of the different actual encoded bits obtained by the same video frame when using different video frame-level quantization parameters during I-frame encoding and the difference between the different video frame-level quantization parameters used in the video frame.

6. The bitrate control method for suppressing excessively large I-frames according to claim 4, characterized in that, The second function g is derived from formula two: The second function g represents the relationship between the ratio of the actual number of encoded bits obtained by different video frames when using the same video frame-level quantization parameters during I-frame encoding and the ratio of their respective intra-frame coding complexity.

7. The bitrate control method for suppressing excessively large I-frames according to claim 5 or 6, characterized in that, Based on the derivation of Formula 1 and Formula 2, Formula 3 is obtained: Further derivation yields Formula 4: ;in, The intra-frame coding complexity of frame n is represented by the intra-frame coding complexity of the (n-1)th frame that is adjacent to frame n. By replacing it with [something else], we get Formula 5.

8. The bitrate control method for suppressing excessively large I-frames according to claim 1, characterized in that, The calculation formula used in step S5 is as follows: ;in, The quantization parameter represents the ability of the I-frame to be encoded to suppress the breathing effect. This represents the quantization parameter of the preceding encoded P-frame or B-frame to be encoded, where δ represents an integer offset.

9. A bitrate control device for suppressing excessively large I-frames, characterized in that, It includes an intra-frame coding mode prediction module, an intra-frame coding complexity calculation module, an I-frame coding bit number upper limit calculation module, an I-frame quantization parameter lower limit calculation module, an I-frame quantization parameter second calculation module, and an I-frame quantization parameter selection module. The intra-coding mode prediction module is used to perform intra-coding mode prediction on the image block to be encoded, and obtain all intra-coding mode candidates of the image block to be encoded and their corresponding intra-coding mode prediction costs. The intra-frame coding complexity calculation module is used to select the minimum intra-frame coding mode prediction cost to characterize the intra-frame coding complexity of the image block to be encoded, and to take the sum of the intra-frame coding complexities of all image blocks in the video frame as the intra-frame coding complexity of the video frame. The I-frame encoding bit limit calculation module is used to calculate the upper limit of the encoding bit count of an I-frame that will not cause bitrate control abnormalities based on the encoding state of the current video sequence. Specifically, it uses the state information of the sliding window used to smooth the target bit allocation at the video frame level during bitrate control to calculate the upper limit of the encoding bit count of an I-frame that will not cause bitrate control abnormalities. The I-frame quantization parameter lower limit calculation module is used to calculate the lower limit of the quantization parameters of the I-frame to be encoded based on the actual number of encoded bits, quantization parameters, and intra-frame coding complexity of the previous encoded I-frame, combined with the upper limit of the number of encoded bits and the intra-frame coding complexity of the I-frame to be encoded; wherein, the intra-frame coding complexity of the I-frame to be encoded reuses the intra-frame coding complexity of the previous encoded video frame of the I-frame to be encoded. The second calculation module for I-frame quantization parameters is used to calculate the quantization parameters of the I-frame to be encoded that have an inhibitory effect on the breathing effect based on the quantization parameters of the previous encoded P-frame or B-frame. The I-frame quantization parameter selection module is used to select the larger value among the lower limit of the quantization parameters of the I-frame to be encoded and the quantization parameters of the I-frame to be encoded that have an inhibitory effect on the breathing effect, as the quantization parameters of the I-frame to be encoded.

Images