Bit rate control allocation method and storage medium based on video content characteristics

By analyzing the texture complexity and motion characteristics of video frames in the video encoder, building new bit allocation weights and optimizing the bit allocation of CTU layer, the problem of inaccurate bit rate control in the existing technology is solved, and the video encoding quality is improved and Rate distortion performance.

CN113973205BActive Publication Date: 2024-11-22XINYI ZHIPU (BEIJING) TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111226979.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-21
Publication Date
2024-11-22
Estimated Expiration
2041-10-21

AI Technical Summary

Technical Problem

The existing video encoder bit rate control algorithm fails to fully consider the impact of the target bits of the frame layer on the bit allocation of the CTU layer, as well as the texture characteristics and time-domain prediction information of the video content, resulting in unreasonable bit allocation and poor rate distortion performance.

Method used

By analyzing the texture complexity and motion characteristics of video frames, calculating information entropy and energy proportional factors, building a new frame layer and CTU layer bit allocation weight, optimizing bit allocation, combining Hadamard transformation to detect energy distribution areas, and performing more accurate bits distribute.

Benefits of technology

The bit rate control accuracy and rate distortion performance of the video encoder are improved, making bit allocation more reasonable and improving video encoding quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113973205B_ABST
    Figure CN113973205B_ABST
Patent Text Reader

Abstract

The present invention claims protection for a method and storage medium for bit allocation based on video coding rate control, the method comprising: for the frame layer, firstly counting the statistical characteristics of the video frame to be encoded, calculating the information entropy of the frame to be encoded, calculating the parameter adjustment factor of the frame to be encoded according to the frame layer fixed weight in the rate control algorithm, and then constructing a new frame layer bit allocation weight according to the information entropy and the parameter adjustment factor to guide the target bit allocation of the frame layer; for the CTU layer, evaluating the texture complexity and motion characteristics of the CTU of the frame to be encoded, calculating the energy proportion factor of each CTU in the frame and the predicted residual energy proportion factor, calculating the feature quantization weighting coefficient, and then constructing a new CTU layer bit allocation weight to guide the target bit allocation of the CTU layer. The present invention can achieve more reasonable bit allocation of the frame layer and the CTU layer, which can not only improve the rate control accuracy, but also improve the video encoding quality, and can be used in video encoders such as H.265 / HEVC and H.266 / VVC.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of video coding, and more specifically, relates to a code rate control bit allocation method based on video content characteristics. Background Art

[0002] Video is the main carrier of information in people's lives and work. With the growing demand for consumer videos with high resolution and high color saturation, high-definition, ultra-high-definition and high dynamic range videos have been widely used. Digital video has shifted from the current 720P to 1080P, and even 4K×2K. Currently, 8K×4K video technology is also under development. The frame rate of digital video has increased from 30fps to 60fps and 120fps, and is even moving towards the goal of 240fps. The widespread application of high-definition and high-frame-rate videos has led to an explosive growth in video data traffic. For example, the data volume of RGB color high-definition video with a frame rate of 30fps, a resolution of 1920×1080, and 8 bits per hour is as high as 4.89TB. The transmission and storage of such a huge amount of data has brought challenges to the current communication network and digital media storage technology. Video coding technology is one of the effective measures to improve network transmission efficiency and reduce data storage pressure.

[0003] Since the 1990s, video compression technology has continued to become a hot area of ​​research and application at home and abroad. In order to meet people's growing requirements for video quality and adapt to different communication network environments, the International Organization for Standardization / International Electrotechnical Commission ISO / IEC, the International Telecommunication Union ITU-T and the China Digital Audio and Video Coding Technology Standard Working Group have successively formulated multiple video coding standards such as H.261, H.263, H.264 / AVC, MPEG-1, MPEG-4, MEPG-7, AVS, etc. The formulation of these international and domestic video coding standards has greatly promoted the promotion of video technology and also enabled the rapid development of digital video coding technology. In 2010, the Video Coding Experts Group (VCEG) of ITU-T and the Moving Picture Experts Group of ISO / IEC jointly established the Joint Collaborative Team on Video Coding (JCT-VC) to study and formulate a new generation of video coding standards - High Efficiency Video Coding (HEVC). In 2013, JCT-VC launched the first version of HEVC and continued to optimize and upgrade it in the following years. HEVC includes the latest video coding technologies, such as multi-angle intra-frame prediction, high-precision motion compensation, motion estimation merging, variable-size discrete cosine transform, mode-dependent discrete sine transform, semantic-based entropy coding, and adaptive loop filtering.

[0004] Rate Control (RC) can encode the video under a given target bit rate and dynamically adjust the quantization parameter QP (Quantization Parameter) through the state of the buffer. When the QP value is small, more edge details in the image can be retained, the video quality is high, but the output bit rate is high; if the QP value is large, the encoding output bit rate can be reduced, but the video quality will be reduced. Rate control makes the actual bit rate required by the video encoder as close as possible to the target bit rate, which can be better transmitted in communication and optimize the quality of the compressed video. Therefore, the rate control algorithm is an indispensable and important part of the video encoder.

[0005] Similar to other rate control schemes, the algorithm of the R-λ model is mainly divided into two parts: one is bit allocation, and the other is the calculation of the Lagrange multiplier λ and the quantization parameter QP. The R-λ model establishes an exponential relationship between the bit rate and the Lagrange multiplier, and its model is:

[0006]

[0007] Where D is the coding distortion; C and K are model parameters related to the video sequence; α and β are model parameters related to the video content characteristics; the relationship between λ and RD curve is shown in the attached figure. Figure 2 As shown in the figure, in a geometric sense, the rate-distortion optimization problem is to select the optimal working point on the RD curve, and λ is the slope of the tangent of the curve, so it is a value that corresponds to the optimal working point one by one; R represents the coding bit, the unit is bpp (bit per pixel), that is, the bit consumed per pixel. If the target bit of a frame or a CTU is T and the number of pixels is N, the bpp calculation formula is:

[0008]

[0009] Among them, α and β are model parameters, which are updated every time a CTU or frame is encoded. Then the QP can be determined by an empirical formula.

[0010] QP=4.2005lnλ+13.7122 (3)

[0011] Bit allocation will be implemented at three levels, including GOP level, picture level and basic coding unit level. First, calculate the target number of bits per frame:

[0012]

[0013] Assume the number of encoded pictures is N coded , the number of bits used by these pictures is R coded , the number of pictures in the current GOP is N GOP , SW is the size of the sliding window for smooth bit allocation, which is used to make the bit consumption change and the quality of the encoded picture smoother. Here it is set to 40, and the bit allocation at the GOP level is:

[0014]

[0015] T GOP =T AvgPic ×N GOP (6)

[0016] If the SW frame image can be exactly the same as each frame consuming T AvgPic bits, the above formula can be changed to:

[0017]

[0018] The first part in equation (7) represents the target bit rate, and the second part represents the buffer status.

[0019] Then there is the bit allocation at the picture level. The number of bits used in the current GOP is CodedGOP , ω is the bit allocation weight of each picture, then the target bit rate of the current picture is:

[0020]

[0021] The bit allocation at the CTU layer, in the proposal, is considered that a basic unit contains a CTU, where the target number of bits is determined by the following formula:

[0022]

[0023] Among them, Bit header It is an estimate of the number of bits of all header information, and is estimated from the actual number of bits of header information of the previous coded pictures at the same layer.

[0024] From the above, it can be seen that the existing rate control algorithm does not fully consider the impact of the target bit of the frame layer on the bit allocation of the CTU layer and the encoding characteristics of the video encoder itself, nor does it fully consider the texture characteristics of each frame, time domain prediction information and video content characteristics, resulting in unreasonable bit allocation and poor rate-distortion performance. Summary of the invention

[0025] The present invention aims to solve the above problems of the prior art. A rate control bit allocation method based on video content characteristics and a storage medium are proposed to improve the accuracy and rate distortion performance of video coding rate control. The technical solution of the present invention is as follows:

[0026] The method for allocating bit rates for rate control based on video content characteristics comprises the following steps:

[0027] The bit analysis steps at the frame level are as follows: First, the texture complexity of the input video frame is analyzed, the information entropy of the frame to be encoded is calculated, and a fixed weight value is assigned according to the bits in the bit rate control algorithm. Calculate the parameter adjustment factor of the frame to be encoded, and then construct a new frame layer bit allocation weight based on the information entropy and the parameter adjustment factor to guide the target bit allocation of the frame layer;

[0028] The bit analysis steps of the CTU layer are as follows: first, the texture complexity and motion characteristics of the CTU of the frame to be encoded are evaluated, the SATD (transformed sum of absolute differences) value of each CTU and the SATD value of its prediction residual are calculated, the average energy and average prediction residual energy of the current frame to be encoded are counted, and the energy share factor and prediction residual energy share factor of each CTU in the frame are obtained. Then, the feature quantization weighting coefficient is calculated, and a new CTU layer bit allocation weight is constructed to guide the target bit allocation of the CTU layer.

[0029] Furthermore, the information entropy EI of the frame to be encoded is calculated by using formula (1):

[0030]

[0031] Among them, p(χ) is the proportion of pixels with gray value χ in the image, and N is the number of gray levels of the image.

[0032] The parameter adjustment factor of the frame to be encoded is calculated by accumulating the fixed weights and information entropy of all frames in a GOP (Group of Pictures) to ensure that the fixed weights and information entropy of the frame layer are at the same order of magnitude, and using formula (2) to calculate the parameter adjustment factor Af between the two.

[0033]

[0034] Furthermore, the construction of a new frame layer bit allocation weight refers to using formula (3) and formula (4) to calculate the new frame layer bit allocation weight ω′ of the frame to be encoded: pic and the total bit weight ω total .

[0035]

[0036]

[0037] Among them, EI i is the information entropy of the current frame to be encoded, ∑ NotCodedPictures ω i The sum of weights is assigned to the bits of all uncoded pictures in the current group of pictures GOP.

[0038] Furthermore, the target bit allocation of the frame layer is obtained by a new frame layer bit allocation weight ω′ pic and the total bit weight ω total Perform bit allocation on the frame layer; formula (5) is the target bit allocation formula of the frame layer.

[0039]

[0040] Among them, T GOP is the total number of bits allocated to the current group of pictures GOP; Coded GOP The number of bits consumed by the current group of pictures GOP.

[0041] Further, the SATD value of each CTU and the SATD value of its prediction residual are calculated, specifically including:

[0042] SATD is the absolute error sum obtained after Hadamard transformation of the coding block or prediction residual, and its calculation formula is as follows:

[0043]

[0044] Where X is the N×N prediction residual matrix, and H is the normalized N×N Hadamard matrix.

[0045] Divide CTU into 8×8 sub-blocks and use H 8×8 The Hadamard matrix of is used to calculate the SATD value of each sub-block using formula (6), and then the SATD value D′1 of each CTU is calculated using formula (7).

[0046]

[0047] Where m is the number of CTU sub-blocks; SATD i is the SATD value of the i-th sub-block of CTU; height and width are the height and width of CTU respectively.

[0048] The pixel values ​​of the CTU of the current frame are subtracted from the pixel values ​​of the CTU at the same position of the previous encoded frame to obtain the prediction residual CTU; the prediction residual CTU is divided into 8×8 sub-blocks, and H 8×8 The Hadamard matrix of is used to calculate the SATD value of each prediction residual sub-block using formula (6), and the SATD value R1′ of each CTU prediction residual is calculated using formula (8).

[0049]

[0050] Where m is the number of CTU sub-blocks; SATD i ′ is the SATD value of the i-th prediction residual sub-block of CTU; height and width are the height and width of CTU respectively.

[0051] Furthermore, the statistical average energy and average prediction residual energy of the current frame to be encoded are calculated by formula (9) and formula (10);

[0052] Since the SATD value can reflect the energy of the coding block or residual block in the frequency domain, the average energy D2′ and the average predicted residual energy R2′ of the current frame to be encoded can be obtained by calculating the average SATD of the 8×8 size sub-block and 8×8 predicted residual sub-block of each frame.

[0053]

[0054]

[0055] Where n is the number of sub-blocks in each frame; SATD j is the SATD value of the j-th sub-block; H and W are the height and width of the current frame respectively.

[0056] Furthermore, the energy proportion factor η1 and the prediction residual energy proportion factor η2 of each CTU in the frame are calculated by formula (11) and formula (12).

[0057]

[0058]

[0059] The calculation of the feature quantization weighted coefficient refers to performing feature quantization on the energy values ​​and prediction residual values ​​of all CTUs in a frame, and using the coefficients normalized by a linear function as the weights of η1 and η2; Formula (13) to Formula (15) are calculation formulas for the feature quantization weighted coefficient.

[0060]

[0061]

[0062]

[0063] in, a represents the weight of η1, the weight of η2, and the characteristic quantization weighting coefficient of η1 respectively; D′ MIN , D′ MAX are the minimum and maximum CTU energy in a frame respectively; R′ MIN , R′ MAX They are the minimum and maximum values ​​of the predicted residual CTU energy in a frame.

[0064] Furthermore, the construction of a new CTU layer bit allocation weight refers to jointly weighting the energy share factor of the CTU layer and the prediction residual energy share factor according to the texture complexity of the image, the motion area and the encoding characteristics of the encoder to obtain a new complexity NC, and its calculation formula is shown in formula (16):

[0065] NC=a×η1+(1-a)×η2 (16)

[0066] Furthermore, the target bit allocation of the guiding CTU layer refers to performing initial target bit allocation on the CTU layer according to the weight NC of the current CTU and the number of remaining target bits of the current frame through formula (17); and then calculating the target bit T of the initial allocation of the current CTU according to the buffer state using formula (18). CTU Perform dynamic adjustment to obtain the final target bit number T CurrCTU .

[0067]

[0068] T CurrCTU =TCTU -(totalWeight-B left ) / realInfluenceCTU+0.5 (18)

[0069] T in formula (17) CTU Indicates the target bits initially allocated to the current CTU; T CurrPic Indicates the target bit of the current frame; Coded Pic Indicates the number of bits consumed by the current Pic; NotCodedCTUs indicates the total number of uncoded CTUs in the current frame; AllCTUs indicates the total number of CTUs in the current frame; NC i Indicates the bit allocation weight of the i-th CTU; Bit header is the estimated number of bits of the frame header information; ω CurrCTU Indicates the adaptive bit allocation weight of the original platform CTU; NC CurrCTU represents the bit allocation weight of each CTU of the proposed algorithm; b is a weighting coefficient, whose value is greater than 0 and less than 1; for the selection of b value, by encoding multiple test video sequences, the influence of different b values ​​on RD performance and bit rate control accuracy is statistically calculated, and the best b value is selected; B in formula (18) left Indicates the actual remaining bits of the current frame; totalWeight indicates the sum of bits required by the remaining CTUs including the current CTU; realInfluenceCTU indicates the actual smoothing window size.

[0070] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for allocating bit rates for rate control based on video content features as described in any one of the above is implemented.

[0071] The advantages and beneficial effects of the present invention are as follows:

[0072] The present invention aims at the problem that the current video encoder rate control algorithm does not fully consider the impact of the target bit of the frame layer on the bit allocation of the CTU layer, and does not fully consider the texture characteristics, time domain prediction information and video content characteristics of each frame, resulting in low accuracy of the target bit of the frame layer and unreasonable allocation of the CTU target bit. A rate control bit allocation method that can be used for video encoders such as H.265 / HEVC and H.266 / VVC is proposed. First, for the frame layer, the statistical characteristics of the entire source are considered, the information entropy of the encoded source is added to the R-lambda model, and a bit allocation optimization model is constructed to guide the frame layer bit allocation, and a bit allocation weight considering the current encoded frame is obtained, which improves the bit accuracy of the frame layer; then for the CTU layer, the Hadamard transform is used to detect the energy distribution area and the motion area, and the energy value and the predicted residual value of the image are constructed into a new complexity, so as to reasonably adjust and accurately allocate the target bit of the CTU layer. The present invention takes into account the image complexity and the encoding characteristics unique to the video encoder, and is used to more reasonably allocate the bits of the frame layer and the CTU layer. The video encoder of the present invention can increase the accuracy of bit rate control and improve rate-distortion performance, and can therefore be used in video encoders such as H.265 / HEVC and H.266 / VVC.

[0073] The present invention proposes for the first time to use the fixed weights and information entropy of all frames in a group of pictures GOP as parameter adjustment factors for frame layer bit allocation, and proposes calculation formulas (1) to (2) for the parameter adjustment factors of the frame to be encoded, frame layer bit allocation weight calculation formulas (3) to (4) and frame layer target bit allocation calculation formula (5). For the first time, the energy proportion factor and prediction residual energy proportion factor of each CTU in the encoded frame are proposed, and the feature quantization weighting coefficient is defined to construct a new CTU layer bit allocation weight, and the energy proportion factor calculation formulas (7), (9), (11) of the CTU to be encoded and the prediction residual energy proportion factor calculation formulas (8), (10), (12) of the CTU are proposed, as well as the feature quantization weighting coefficient formulas (13) to (15), the CTU bit allocation weight calculation formula (16) and the CTU target bit allocation calculation formulas (17) to (18). The information entropy and parameter adjustment factors proposed in the present invention fully consider the characteristics of each frame content itself, and take into account the bit requirements of the current coded frame on the frame layer and the impact of the subsequent CTU layer to be encoded, so that the target bit allocation of the frame layer is more reasonable, which can not only make the actual bit rate closer to the target bit rate, but also improve the video encoding quality to a certain extent. The energy proportion factor of each CTU in the frame, the prediction residual energy proportion factor, the feature quantization weighting coefficient and a new CTU layer bit allocation weight proposed in the present invention not only consider the texture characteristics, time domain prediction information and video content characteristics of each frame, but also can effectively distinguish the motion area, and combine the actual complexity of the CTU for a good fit, that is, a linear relationship. Making the bit allocation more reasonable can not only improve the accuracy of bit rate control, but also improve the quality of video encoding.

[0074] At present, the effects of some existing H.265 / HEVC and H.266 / VVC bit rate control algorithms are not ideal, so the invention in this article has strong innovation and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a flow chart of a method for allocating bit rates for rate control based on video content characteristics according to a preferred embodiment of the present invention;

[0076] Figure 2 Schematic diagram of the relationship between the Lagrange multiplier λ and the RD curve. DETAILED DESCRIPTION

[0077] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.

[0078] The technical solution of the present invention to solve the above technical problems is:

[0079] like Figure 1 FIG. 1 is a flow chart of a method for allocating bit rates for rate control based on video content characteristics provided by the present invention, which specifically includes the following steps:

[0080] 101. Calculate the information entropy EI of the frame to be encoded and obtain the frame layer fixed weight in the bit rate control algorithm

[0081] 102. Calculate a parameter adjustment factor Af of a frame to be encoded;

[0082] 103. Construct a new frame layer bit allocation weight according to the information entropy of the frame to be encoded and the parameter adjustment factor, so as to guide the target bit allocation of the frame layer;

[0083] 104. Calculate the SATD (transformed sum of absolute differences) value of each CTU and the SATD value of its prediction residual;

[0084] 105. Count the average energy and average prediction residual energy of the current frame to be encoded;

[0085] 106. Calculate the energy proportion factor and prediction residual energy proportion factor of each CTU in the frame;

[0086] 107. Calculate the feature quantization weighting coefficient;

[0087] 108. Construct a new CTU layer bit allocation weight to guide the target bit allocation of the CTU layer.

[0088] Preferably, in step 101, calculating the information entropy of the frame to be encoded refers to using the following formula for calculation; obtaining the frame layer fixed weight refers to the fixed weight value of the bit allocation of each frame in the video bit rate control algorithm

[0089]

[0090] Among them, p(χ) is the proportion of pixels with gray value χ in the image, and N is the number of gray levels of the image.

[0091] Preferably, in step 102, the parameter adjustment factor of the frame to be encoded is calculated, which means that in order to ensure that the fixed weight and information entropy of the frame layer are at the same order of magnitude, the fixed weights and information entropy of all frames in a group of images GOP are accumulated, and the parameter adjustment factor Af between the two is calculated using formula (2).

[0092]

[0093] Preferably, in step 103, constructing a new frame layer bit allocation weight means using formula (3) and formula (4) to calculate the new frame layer bit allocation weight ω′ of the frame to be encoded.pic and the total bit weight ω total .

[0094]

[0095]

[0096] Among them, EI i is the information entropy of the current frame to be encoded, The sum of weights is assigned to the bits of all uncoded pictures in the current group of pictures GOP.

[0097] Preferably, in step 103, the target bit allocation for guiding the frame layer is a new frame layer bit allocation weight ω′ pic and the total bit weight ω total Perform high-precision bit allocation on the frame layer. The formula for target bit allocation on the frame layer is shown in equation (5).

[0098]

[0099] Among them, T GOP is the total number of bits allocated to the current GOP; Coded GOP The number of bits consumed by the current GOP.

[0100] Preferably, in step 104, the SATD value of each CTU and the SATD value of its prediction residual are calculated by the following steps and formulas.

[0101] SATD (Sum of Absolute Transformed Difference) is the absolute error sum obtained after the coding block or prediction residual is transformed by Hadamard transform. The calculation formula is as follows:

[0102]

[0103] Where X is the N×N prediction residual matrix, and H is the normalized N×N Hadamard matrix.

[0104] Divide CTU into 8×8 sub-blocks; use H 8×8 The Hadamard matrix of is used to calculate the SATD value of each sub-block using formula (6), and then the SATD value D′1 of each CTU is calculated using formula (7).

[0105]

[0106] Where m is the number of CTU sub-blocks; SATD iis the SATD value of the i-th sub-block of CTU; height and width are the height and width of CTU respectively.

[0107] The pixel values ​​of the CTU of the current frame are subtracted from the pixel values ​​of the CTU at the same position in the previous encoded frame to obtain the prediction residual CTU. The prediction residual CTU is divided into 8×8 sub-blocks and H 8×8 The Hadamard matrix of is used to calculate the SATD value of each prediction residual sub-block using formula (6), and the SATD value R1′ of each CTU prediction residual is calculated using formula (8).

[0108]

[0109] Where m is the number of CTU sub-blocks, SATD′ i is the SATD value of the i-th prediction residual sub-block of CTU; height and width are the height and width of CTU respectively.

[0110] Preferably, in step 105, the statistics can reflect the average energy and average prediction residual energy of the current frame to be encoded, which are calculated by formula (9) and formula (10).

[0111] Since the SATD value can reflect the energy of the coding block or residual block in the frequency domain, the average energy D′2 and the average predicted residual energy R2′ of the current frame to be encoded can be obtained by calculating the SATD average value of the 8×8 size sub-block and the 8×8 predicted residual sub-block in each frame.

[0112]

[0113]

[0114] Where n is the number of sub-blocks in each frame; SATD j is the SATD value of the j-th sub-block; H and W are the height and width of the current frame respectively.

[0115] Preferably, in step 106, the energy proportion factor and the prediction residual energy proportion factor of each CTU in the frame are calculated by formula (11) and formula (12).

[0116]

[0117]

[0118] Preferably, in step 107, calculating the feature quantization weighting coefficient means that the energy values ​​and prediction residual values ​​of all CTUs in a frame are feature quantized, and the coefficients normalized by a linear function are used as the weights of η1 and η2. Formula (13) to Formula (15) are calculation formulas for the feature quantization weighting coefficient.

[0119]

[0120]

[0121]

[0122] in, a represents the weight of η1, the weight of η2, and the characteristic quantization weighting coefficient of η1 respectively; D′ MIN , D′ MAX are the minimum and maximum CTU energy in a frame respectively; R′ MIN , R′ MAX They are the minimum and maximum values ​​of the predicted residual CTU energy in a frame.

[0123] Preferably, in step 108, constructing a new CTU layer bit allocation weight means weighting the energy share factor of the CTU layer and the prediction residual energy share factor according to the texture complexity of the image, the motion area and the encoding characteristics of the encoder to obtain a new complexity NC, and its calculation formula is shown in formula (16).

[0124] NC=a×η1+(1-a)×η2 (16)

[0125] Preferably, in step 108, guiding the target bit allocation of the CTU layer means that the initial target bit allocation of the CTU layer is performed according to the weight NC of the current CTU and the number of remaining bits of the current frame through formula (17). Then, according to the buffer state, the target bit T of the current CTU is initially allocated using formula (18). CTU Perform dynamic adjustment to obtain the final target bit number T CurrCTU .

[0126]

[0127] T CurrCTU =T CTU -(totalWeight-B left ) / realInfluenceCTU+0.5 (18)

[0128] T in formula (17) CTU Indicates the target bits initially allocated to the current CTU; T CurrPicIndicates the target bit of the current frame; Coded Pic Indicates the number of bits consumed by the current Pic; NotCodedCTUs indicates the total number of uncoded CTUs in the current frame; AllCTUs indicates the total number of CTUs in the current frame; NC i Indicates the bit allocation weight of the i-th CTU; Bit header is the estimated number of bits of the frame header information; ω CurrCTU Indicates the adaptive bit allocation weight of the original platform CTU; NC CurrCTU represents the bit allocation weight of each CTU of the proposed algorithm; b is a weighting coefficient, whose value is greater than 0 and less than 1. In order to determine the value of b, the coding performance test can be performed on multiple test video sequences, and the impact of different b values ​​on RD performance and bit rate control accuracy can be statistically analyzed, and finally the value of b can be determined. B in formula (18) left Indicates the actual remaining bits of the current frame; totalWeight indicates the sum of bits required by the remaining CTUs including the current CTU; realInfluenceCTU indicates the actual smoothing window size.

[0129] A storage medium stores a computer program therein, wherein when the computer program is read by a processor, any of the above methods is executed.

[0130] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0131] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0132] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0133] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. Bit rate control bit allocation method based on video content characteristics, It is characterized in that The following steps are involved: Frame layer bit analysis steps: First, analyze the texture complexity of the input video frame, calculate the information entropy of the frame to be encoded, and assign a fixed weight value according to the bit of each frame in the bit rate control algorithm Calculate the parameter adjustment factor of the frame to be encoded, and then construct a new frame layer bit allocation weight based on the information entropy and the parameter adjustment factor to guide the target bit allocation of the frame layer; The bit analysis steps of the CTU layer: First, the texture complexity and motion characteristics of the CTU of the frame to be encoded are evaluated, the transformed sum of absolute differences (SATD) value of each CTU and the SATD value of its prediction residual are calculated, the average energy and average prediction residual energy of the current frame to be encoded are counted, and the energy proportion factor and prediction residual energy proportion factor of each CTU in the frame are obtained. Then, the feature quantization weighting coefficient is calculated, and a new CTU layer bit allocation weight is constructed to guide the target bit allocation of the CTU layer. The calculation of the information entropy EI of the frame to be encoded refers to the calculation using formula (1); Where p(χ) is the proportion of pixels with gray value χ in the image, and N is the number of gray levels of the image; The parameter adjustment factor of the frame to be encoded is calculated by accumulating the fixed weights and information entropy of all frames in a GOP image group in order to ensure that the fixed weight and information entropy of the frame layer are at the same order of magnitude, and using formula (2) to calculate the parameter adjustment factor Af between the two; The construction of a new frame layer bit allocation weight refers to using formula (3) and formula (4) to calculate the new frame layer bit allocation weight ω′ of the frame to be encoded. pic and the total bit weight ω total ; Among them, EI i is the information entropy of the current frame to be encoded; ∑ NotCodedPictures ω i Assign the sum of weights to the bits of all uncoded pictures in the current group of pictures GOP; The target bit allocation of the frame layer is obtained by the new frame layer bit allocation weight ω′ pic and the total bit weight ω total The frame layer is allocated bits. Formula (5) is the frame layer target bit allocation formula; Among them, T GOP is the total number of bits allocated to the current group of pictures GOP; Coded GOP is the number of bits consumed by the current group of pictures GOP; The energy proportion factor η of each CTU in the frame 1 And the prediction residual energy ratio factor η 2 It is calculated by formula (11) and formula (12); The calculation of the feature quantization weighted coefficient refers to performing feature quantization on the energy values ​​and prediction residual values ​​of all CTUs in a frame, and taking the coefficient normalized by the linear function as η 1 and η 2 The weight of the proportion; Formula (13) to Formula (15) are the calculation formulas for the feature quantization weighting coefficient; in, a represents η 1 The weight of the proportion, η 2 The weight of the proportion, η 1 The characteristic quantization weighting coefficient of MIN , D′ MAX are the minimum and maximum CTU energy in a frame respectively; R′ MIN , R′ MAX They are the minimum and maximum values ​​of the predicted residual CTU energy in a frame; The construction of a new CTU layer bit allocation weight refers to jointly weighting the energy share factor and the prediction residual energy share factor of the CTU layer according to the texture complexity of the image, the motion area and the encoding characteristics of the encoder to obtain a new complexity NC, and its calculation formula is shown in formula (16): NC=a×η 1 +(1-a)×η 2 (16) The target bit allocation of the guiding CTU layer refers to performing initial target bit allocation on the CTU layer according to the weight NC of the current CTU and the number of remaining target bits of the current frame through formula (17); and then using formula (18) to allocate the target bit T of the initial allocation of the current CTU according to the buffer state. CTU Perform dynamic adjustment to obtain the final target bit number T CurrCTU ; T CurrCTU =T CTU -(totalWeight-B left ) / realInfluenceCTU+0.5 (18) T in formula (17) CTU Indicates the target bits initially allocated to the current CTU; T CurrPic Indicates the target bit of the current frame; Coded Pic Indicates the number of bits consumed by the current Pic; NotCodedCTUs indicates the total number of uncoded CTUs in the current frame; AllCTUs indicates the total number of CTUs in the current frame; NC i Indicates the bit allocation weight of the i-th CTU; Bit header is the estimated number of bits of the frame header information; ω CurrCTU Indicates the adaptive bit allocation weight of the original platform CTU; NC CurrCTU represents the bit allocation weight of each CTU of the proposed algorithm; b is a weighting coefficient, whose value is greater than 0 and less than 1; for the selection of b value, by encoding multiple test video sequences, the influence of different b values ​​on RD performance and bit rate control accuracy is statistically calculated, and the best b value is selected; B in formula (18) left Indicates the actual remaining bits of the current frame; totalWeight indicates the sum of bits required by the remaining CTUs including the current CTU; realInfluenceCTU indicates the actual smoothing window size.

2. The method for allocating bit rates for rate control based on video content characteristics according to claim 1, It is characterized in that The calculation of the SATD value of each CTU and the SATD value of its prediction residual specifically includes: SATD is the absolute difference sum obtained after Hadamard transformation of the coding block or prediction residual, and its calculation formula is as follows: Where X is the N×N prediction residual square matrix, and H is the normalized N×N Hadamard matrix; Divide CTU into 8×8 sub-blocks and use H 8×8 The Hadamard matrix of the block is used to calculate the SATD value of each sub-block using formula (6), and then the SATD value D′ of each CTU is calculated using formula (7) 1 ; Where m is the number of CTU sub-blocks; SATD i is the SATD value of the i-th sub-block of CTU; height and width are the height and width of CTU respectively; The pixel values ​​of the CTU of the current frame are subtracted from the pixel values ​​of the CTU at the same position in the previous encoded frame to obtain the prediction residual CTU; the prediction residual CTU is divided into 8×8 sub-blocks, and H 8×8 The Hadamard matrix of is used to calculate the SATD value of each prediction residual sub-block using formula (6), and the SATD value R of each CTU prediction residual is calculated using formula (8) 1 ′; Where m is the number of CTU sub-blocks, SATD′ i is the SATD value of the i-th prediction residual sub-block of CTU; height and width are the height and width of CTU respectively.

3. The method for allocating bit rates for rate control based on video content characteristics according to claim 2, It is characterized in that The statistical average energy and average prediction residual energy of the current frame to be encoded are calculated by formula (9) and formula (10); Since the SATD value can reflect the energy of the coding block or residual block in the frequency domain, the average energy D of the current frame to be coded is 2 ′ and the average prediction residual energy R 2 ′ can be obtained by calculating the average value of SATD of 8×8 size sub-block and 8×8 prediction residual sub-block in each frame; Where n is the number of sub-blocks in each frame; SATD j is the SATD value of the j-th sub-block; H and W are the height and width of the current frame respectively.

4. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for allocating bit rates for rate control based on video content characteristics as claimed in any one of claims 1 to 3 is implemented.