Code rate control method and device, electronic equipment and readable storage medium
By calculating the SATD values of image frames and stripes, pre-allocating the code rate and performing code rate control, the problem of unstable code rate in video transmission is solved, network congestion is reduced, and user experience is improved.
Patent Information
- Application Number
- CN202411056577.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-12-03
AI Technical Summary
In the prior art, constant code rate encoding during video transmission cannot ensure the stability of the transmission code rate, resulting in an increase in the probability of network congestion and cannot meet the needs of real-time encoding.
By calculating the striped SATD value of the image frame and the SATD value of the image frame, the pre-allocated code rate is calculated, and the code rate is controlled according to the pre-allocated code rate, ensuring the stability of the transmission code rate and reducing the probability of network congestion.
It ensures the stability of the bit rate in video transmission, avoids network congestion caused by the bit rate impact, and brings users a smoother and smoother gaming experience.
Smart Images

Figure CN119071544A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of video processing technology, and specifically relates to a bit rate control method, device, electronic device and readable storage medium. Background Art
[0002] In related technologies, in order to ensure the stability of the transmission bit rate, real-time encoding during video transmission (such as cloud game videos, etc.) generally adopts constant bit rate encoding (CBR). However, the so-called CBR in actual video transmission is generally the average bit rate (Average Bit Rate, ABR), that is, the bit rate is controlled at the rated bit rate per unit time. In this case, when the transmitted video picture changes suddenly, a sudden increase in the single-frame bit rate will still occur, thereby increasing the probability of network congestion and failing to ensure the stability of the transmission bit rate. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide a bit rate control method, device, electronic device and readable storage medium to solve the problem in the related art that the stability of the transmission bit rate cannot be ensured.
[0004] In order to solve the above technical problems, this application is implemented as follows:
[0005] In a first aspect, a rate control method is provided, which is applied to an electronic device, and includes:
[0006] Determining SATD values of strips of an image frame;
[0007] Calculating a pre-allocated bit rate of the stripe according to the SATD value of the stripe and the SATD value of the image frame;
[0008] The code rate control is performed according to the pre-allocated code rate.
[0009] In a second aspect, a rate control device is provided, which is applied to an electronic device, including:
[0010] A determination module, used for determining the SATD value of a strip of an image frame;
[0011] A calculation module, used for calculating the pre-allocated bit rate of the strip according to the SATD value of the strip and the SATD value of the image frame;
[0012] A control module is used to perform bit rate control according to the pre-allocated bit rate.
[0013] In a third aspect, an electronic device is provided, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.
[0014] In a fourth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.
[0015] In a fifth aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in the first aspect.
[0016] Through the scheme in the embodiment of the present application, the SATD value of the strip can be used to approximately reflect the bit rate trend of the strip, so that when the pre-allocated bit rate of the strip is calculated according to the SATD value of the strip and the SATD value of the image frame, and the bit rate is controlled according to the pre-allocated bit rate, the stability of the transmission bit rate can be ensured and the probability of network congestion can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 is a flow chart of a rate control method provided by an embodiment of the present application;
[0018] Figure 2 is a schematic diagram of dividing strips in an embodiment of the present application;
[0019] Figure 3 is a schematic diagram of a zigzag scanning sequence in an embodiment of the present application;
[0020] Figure 4 It is a structural schematic diagram of a bit rate control device provided in an embodiment of the present application;
[0021] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0023] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0024] Optionally, applicable scenarios of the embodiments of the present application include but are not limited to cloud gaming scenarios, etc.
[0025] The following, in conjunction with the accompanying drawings, describes in detail the bit rate control method, device, electronic device, and readable storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios.
[0026] See also Figure 1 , Figure 1 is a flow chart of a rate control method provided in an embodiment of the present application, the method is applied to electronic devices such as Figure 1 As shown, the method comprises the following steps:
[0027] Step 11: Determine the absolute transform residual and SATD value of the strips of the image frame;
[0028] Step 12: Calculate the pre-allocated bit rate of the stripe according to the SATD value of the stripe and the SATD value of the image frame;
[0029] Step 13: Perform bit rate control according to the pre-allocated bit rate.
[0030] In an embodiment of the present application, the image frame may be any frame of an image in a real-time transmitted video (such as a cloud game video, etc.).
[0031] The strip can be any strip of the corresponding image frame. A strip can be understood as a sub-image, for example, a sub-image of size W*h in the corresponding image frame, where W represents the image width and h represents the strip height. The sizes of W and h can be determined based on actual needs; Figure 2 As shown. Figure 2 In , the image is divided into n strips, namely strip 1, strip 2, ..., strip n, and the size of each strip is W*h.
[0032] The Sum of Absolute Transformed Difference (SATD) of the strip, i.e., the SATD value, can be calculated after performing a Hadamard transform on the pixel residual. The Hadamard transform is usually used to calculate the SATD of the residual, i.e., performing a Hadamard transform on the residual signal, and then calculating the sum of the absolute values of the transformed coefficients. The Hadamard transform is a transform implemented using a Hadamard matrix as a transform matrix, for example, the Hadamard matrix can be selected as a symmetric orthogonal matrix.
[0033] The SATD value of the image frame is the total SATD value of all strips of the image frame.
[0034] In an optional implementation, the SATD value of the image frame may be equal to the weighted sum of the low-frequency luminance SATD value, the low-frequency chrominance SATD value, the high-frequency luminance SATD value and the high-frequency chrominance SATD value of the image frame, and the relevant weighting coefficient may be set based on actual conditions.
[0035] Through the scheme in the embodiment of the present application, the SATD value of the strip can be used to approximately reflect the bit rate trend of the strip, so that when the pre-allocated bit rate of the strip is calculated according to the SATD value of the strip and the SATD value of the image frame, and the bit rate is controlled according to the pre-allocated bit rate, the stability of the transmission bit rate can be ensured and the probability of network congestion can be reduced. Furthermore, when the scheme in the present application is applied to the cloud game video transmission scenario, the network congestion caused by the bit rate impact can be avoided, thereby giving users a smoother and more fluent gaming experience.
[0036] Optionally, in order to calculate the strip SATD, the strip may be divided into x n×m image blocks, wherein n is the number of pixels included in each column of pixels in the image block, and m is the number of pixels included in each row of pixels in the image block, and n and m are integers greater than 1; for example, n=m=4. The above process of determining the SATD value of the strip of the image frame may include:
[0037] Subtracting a reference pixel value of each pixel from a pixel value of each pixel in the strip to obtain a residual matrix of the strip;
[0038] Determine, according to the residual matrix of the strip, a residual matrix corresponding to each of the x image blocks; wherein, since the strip is divided into x image blocks, the residual matrix of the strip may be divided to determine the residual matrix corresponding to each of the x image blocks;
[0039] Performing Hadamard transform on the residual matrix corresponding to each image block to obtain the Hadamard coefficient corresponding to each pixel in each image block, and calculating the sum of the absolute values of the Hadamard coefficients corresponding to each pixel to obtain the SATD value of each image block; the Hadamard transform may adopt an existing Hadamard transform method, which is not limited to this;
[0040] The SATD value of the strip is determined according to the SATD value of each image block.
[0041] Optionally, since the strip is divided into x image blocks, after obtaining the SATD value of each of the x image blocks, the SATD values of the x image blocks may be directly summed to obtain the SATD value of the strip. In addition, the SATD values of the x image blocks may be weighted summed to obtain the SATD value of the strip.
[0042] Optionally, the reference pixel value may be a preset pixel value, or may be a pixel value corresponding to the last row of the upper strip of the current strip, so as to achieve vertical pixel prediction.
[0043] Optionally, if the strip is any strip except the first strip among the multiple strips of the image frame, the reference pixel value of each pixel in the strip may be equal to the pixel value of the pixel in the last row of pixels of the previous strip that belongs to the same column as the pixel; or, if the strip is the first strip among the multiple strips, since the first strip has no upper strip, that is, the reference pixel value cannot be determined based on the upper strip, the reference pixel value of each pixel may be obtained based on the pixel value of the adjacent column of the image block where the pixel is located; for example, for a certain pixel in the first strip, the horizontal average value (DC) prediction may be used, and the average value of the adjacent column of pixels of the image block where the pixel is located may be used as the reference pixel value of the pixel. For example, for a 4x4 image block, the average value of the left column of pixels may be used as the reference pixel value of the pixel in the image block.
[0044] In the embodiment of the present application, for the low-frequency sub-band, considering the characteristics of entropy coding such as CAVLC (Context-Adaptive Variable-Length Coding, context-based adaptive variable length coding), in order to ensure the coding effect, the SATD value obtained by adaptive variable length coding can be used to replace the original SATD value. The above-mentioned process of calculating the sum of the absolute values of the Hadamard coefficients corresponding to each pixel to obtain the SATD value of each image block may include:
[0045] In the reverse order of the zigzag scanning order, the following operations are performed on the Hadamard coefficients corresponding to each pixel in each image block, and the finally obtained vlc_sum is determined as the SATD value of each image block:
[0046] A i The value after right shifting vlc_cnt is added to vlc_sum;
[0047] Among them, the a i represents the Hadamard coefficient corresponding to pixel i, if it is a binary number; the vlc_cnt is the number of binary digits, and the initialization value of the vlc_cnt is 0; the vlc_sum is the accumulated value, and the initialization value of the vlc_sum is 0; when a i When the value of vlc_cnt is greater than the third threshold, the value of vlc_cnt is increased by the first value. The third threshold can be set based on actual needs; the first value can be set based on actual needs, such as being equal to 1.
[0048] For example, if the image block is a 4x4 image block, the zigzag scanning order may be as follows: Figure 3 As shown. At this time, the SATD value of the 4x4 image block can be determined according to the following formula:
[0049] vlc_cnt = 0;
[0050] for(a i : zigzag scan coefficient) / / such as Figure 3 a1, a2, ..., a16 shown, represent Hadamard coefficients;
[0051] {
[0052] vlc_sum+=a i >>vlc_cnt / / This means that a i The value after right shifting vlc_cnt is added to vlc_sum;
[0053] if(a i >vlc_thre(vlc_cnt)) / / This vlc_thre(vlc_cnt) represents the third threshold;
[0054] vlc_cnt++ / / This means vlc_cnt adds 1;
[0055] }
[0056] In the embodiment of the present application, the bit rate trend of the SATD approximation response strip can be used to pre-allocate the bit rate. The calculation of the pre-allocated bit rate of the strip based on the SATD value of the strip and the SATD value of the image frame may include any of the following:
[0057] (1) calculating a pre-allocated bit rate of the slice according to the SATD value of the slice, the SATD value of the image frame, and the number of bits of the image frame;
[0058] (2) Calculating the pre-allocated bit rate of the stripe according to the SATD value of the stripe, the SATD value of the image frame, and the difference between the number of bits of the image frame and the reserved number of bits; the reserved number of bits can be determined based on actual needs to avoid the impact of bit rate fluctuations on encoding.
[0059] For example, for (1) above, the pre-allocated bit rate of the i-th stripe can be calculated using the following formula:
[0060]
[0061] Among them, SATD i Represents the SATD value of the i-th strip, SATD sum Indicates the SATD value of the current image frame, and total_bits indicates the number of bits of the current image frame. For example, total_bits can be equal to the pixel depth (Bits per Pixel, BPP) multiplied by the number of pixels of the entire frame image, indicating the number of bits of a frame image. It is understandable that in addition to the above formula, the above formula can also be appropriately deformed (such as adding a coefficient, etc.) to calculate the pre-allocated bit rate of the i-th stripe, which should fall within the scope of protection of this application.
[0062] For example, for (2) above, the pre-allocated bit rate of the i-th stripe can be calculated using the following formula:
[0063]
[0064] Among them, SATD i Represents the SATD value of the i-th strip, SATD sum represents the SATD value of the current image frame; total_bits represents the number of bits of the current image frame; B represents the number of reserved bits, which may be equal to the number of bits of a stripe. It is understandable that, in addition to the above formula, the above formula may also be appropriately modified (such as adding a coefficient, etc.) to calculate the pre-allocated bit rate of the i-th stripe, which shall fall within the scope of protection of this application.
[0065] Optionally, in the embodiment of the present application, the bit rate control may be performed based on proportional integral derivative (PID). The bit rate control according to the pre-allocated bit rate may include:
[0066] According to the pre-allocated bit rate and the actual bit rate of the previous stripe of the stripe, a change amount of a quantizer parameter (QP) of the stripe compared with the QP of the previous stripe of the stripe is determined; for example, according to the pre-allocated bit rate of the stripe (i.e., the current stripe) and the actual bit rate / target bit rate of the previous stripe, a ΔQP of the current stripe can be obtained by a PID bit rate control formula, and the QP value of the previous stripe plus the ΔQP is the QP value of the current stripe; and the QP value of the first stripe of the image frame is usually initialized by the encoder and set by the user;
[0067] When the change amount is less than or equal to a first threshold, the bit rate control is performed on the slice according to the QP of the slice; or, when the change amount is greater than the first threshold, the change amount is set to the first threshold, and the sum of the change amount and the QP of the previous slice of the slice is determined as the QP of the slice, and the bit rate control is performed on the slice according to the QP of the slice. The first threshold can be set based on actual needs, such as being equal to 1.
[0068] In an optional implementation manner, the PID code rate control formula may be as follows:
[0069]
[0070] Wherein, R represents the actual bit rate of the previous stripe, R° is the pre-allocated bit rate of the current stripe; a and b represent proportional coefficients, for example: a=0.5, b=5.0.
[0071] It should be pointed out that QP reflects the compression of image space details. If QP is small, most of the image details will be retained; if QP increases, the bit rate will be reduced, some image details will be lost, but image distortion will be enhanced and image quality will decrease. QP and bit rate are inversely proportional, and this inverse relationship will be more obvious as the complexity of the video source increases. The ΔQP here refers to the absolute value. When ΔQP is less than -1, it will be forced to be set to -1.
[0072] Therefore, when bit rate control is performed based on PID, bit rate fluctuations can be smoothed, and the ΔQP of each strip is limited to be no greater than a first threshold (for example, 1). If ΔQP is greater than the first threshold (for example, 1), ΔQP will be forcibly set to the first threshold (for example, 1), thereby controlling the QP deviation and ensuring a smooth transmission bit rate and stable clarity.
[0073] In an optional implementation, considering the existence of QP bias in brightness, chroma, high and low frequencies, when calculating the SATD value of the image frame, weighting can be performed according to the QP bias, and relevant parameters are determined according to a preset quantization bias value, specifically 2^(-△QP / 8), and:
[0074] SATD sum =1×SATD low_luma +2 -1 / 4 ×SATD low_chroma +2 -7 / 4 ×SATD high_luma +2 -2 ×SATD high_chroma
[0075] Among them, SATD low_luma is the low-frequency brightness SATD value, SATD low_chroma is the low-frequency chromaticity SATD value, SATD high_luma is the high-frequency brightness SATD value, SATD high_chroma is the high-frequency chromaticity SATD value; 1, 2 -1 / 4 , 2 -7 / 4 and 2 -2 They represent weighting coefficients respectively.
[0076] In the embodiment of the present application, in order to improve the bit rate control effect, a strict frame-level bit rate control method can be adopted. In this case, when calculating the pre-allocated bit rate of the stripe, the pre-allocated bit rate of the stripe is calculated according to the SATD value of the stripe, the SATD value of the image frame, and the difference between the number of bits of the image frame and the reserved number of bits. The above-mentioned bit rate control according to the pre-allocated bit rate includes:
[0077] When the first condition is met, the QP of the slice is adjusted, for example, the QP of the slice is increased by 1, and the bit rate control of the slice is performed according to the adjusted QP of the slice; or, when the first condition is not met, the bit rate control of the slice is performed according to the QP of the slice;
[0078] Optionally, the first condition includes but is not limited to at least one of the following:
[0079] (1) The difference between the actual coding rate of the slice determined according to the QP of the slice and the pre-allocated coding rate of the slice is greater than or equal to a second threshold; the second threshold can be set based on actual needs, for example, equal to the number of bits of a slice; by adjusting the slice QP after the difference between the actual coding rate and the pre-allocated coding rate is greater than or equal to the second threshold, excessive recoding times can be avoided, thereby ensuring the efficiency of code rate control;
[0080] (2) The number of bits occupied by the pre-allocated code rate of the stripe and the actual coding rates of all stripes before the stripe in the image frame exceeds the number of available bits, that is, the number of bits occupied by the stripe and all stripes before it exceeds the number of available bits. The number of available bits can be understood as a standard for measuring code control, that is, the number of bits occupied in the coding cannot exceed the number of available bits. For example, the number of available bits can be stored in a buffer, and the size of the buffer is the number of available bits.
[0081] In this way, by setting the first condition, strict frame-level bit rate control can be achieved, thereby ensuring both a stable encoding output bit rate and undistorted video quality.
[0082] Optionally, in the process of encoding the strip of the image frame, the number of available bits can be reduced by a second value; wherein, after encoding the image frame, the number of available bits is less than or equal to the number of reserved bits, that is, after encoding the image frame, the number of available bits is less than or equal to the number of reserved bits. In this way, when encoding of a frame of image is completed, the number of available bits is less than or equal to the number of reserved bits, which can ensure that the actual number of encoded bits does not exceed the target number of encoded bits for a frame of image.
[0083] Optionally, the second value may be set based on actual needs, such as being determined based on the number of stripes to be encoded (i.e., the number of stripes remaining to be encoded), the number of available bits, and the number of reserved bits. For example, if the reduction operation on the number of available bits is started when the number of stripes remaining to be encoded is Y (Y is greater than 1), and the number of available bits is reduced by the second value after encoding each stripe, the value of the second value may be greater than or equal to (the number of available bits - the number of reserved bits) / Y; wherein, if the number of available bits is reduced to be equal to the number of reserved bits, the second value may be set to be equal to (the number of available bits - the number of reserved bits) / Y.
[0084] In an optional implementation, the strict frame-level rate control process may include:
[0085] S1: Calculate strip_bits, where strip_bits is the number of bits in a strip;
[0086] For example: strip_bits = bpp × number of strip pixels.
[0087] S2: Using formula Calculate the pre-allocated bit rate for the i-th stripe B represents the number of reserved bits, such as strip_bits, which is used to reserve the number of available bits after the bit rate fluctuates; rc_buffer_size
[0088] S3: During the process of encoding the strip, reduce the number of available bits rc_buffer_size, and after the encoding is completed, reduce rc_buffer_size to be equal to the number of reserved bits, such as strip_bits.
[0089] For example, if the number of remaining coding strips in an image frame is 64, the reserved number of bits is equal to strip_bits, the available number of bits rc_buffer_size = 3×strip_bits, and the available number of bits is reduced to equal to the reserved number of bits after encoding is completed, the second value can be set to 1 / 32×strip_bits, that is, the second value is set to equal to (3×strip_bits-strip_bits) / 64, and when encoding each remaining strip, rc_buffer_size is reduced by 1 / 32×strip_bits, such as rc_buffer_size(i+1)=rc_buffer_size(i)-1 / 32×strip_bits, so that after encoding 64 strips, rc_buffer_size is reduced to equal to the reserved number of bits strip_bits.
[0090] S4: When the current stripe is encoded, if the following situations occur, adjust the QP to increase by 1 (which will reduce the current stripe bit rate), and perform bit rate control based on the adjusted QP: 1) The actual coding rate of the current stripe is strip_bits more than the target bit rate; 2) The cumulative number of bits occupied by the encoded stripe (that is, the cumulative number of bits occupied by the current stripe and all previous stripes together) exceeds the available bit number rc_buffer_size.
[0091] S5: After the encoding of the entire frame is completed, the part that is less than the target bit rate is filled with zeros to ensure normal encoding.
[0092] Through testing, the solution in this application can achieve constant bit rate encoding, and can improve encoding performance compared to the existing technology. When the strict frame-level bit rate control method in this solution is applied to the cloud game video transmission scenario, network congestion caused by bit rate impact can be avoided, thereby giving users a smoother and more fluent gaming experience.
[0093] It should be noted that the execution subject of the rate control method provided in the embodiment of the present application may be a rate control device, or a control module in the rate control device for executing the rate control method. In the embodiment of the present application, the rate control device provided in the embodiment of the present application is described by taking the rate control device executing the rate control method as an example.
[0094] See also Figure 4 , Figure 4is a schematic diagram of the structure of a bit rate control device provided in an embodiment of the present application, and the device is applied to electronic devices such as Figure 4 As shown, the bit rate control device 40 includes:
[0095] A determination module 41, used to determine the SATD value of the strip of the image frame;
[0096] A calculation module 42, configured to calculate a pre-allocated bit rate of the strip according to the SATD value of the strip and the SATD value of the image frame;
[0097] The control module 43 is used to perform bit rate control according to the pre-allocated bit rate.
[0098] Optionally, the calculation module 42 is specifically configured to perform any of the following:
[0099] Calculating a pre-allocated bit rate of the slice according to the SATD value of the slice, the SATD value of the image frame, and the number of bits of the image frame;
[0100] The pre-allocated bit rate of the slice is calculated according to the SATD value of the slice, the SATD value of the image frame, and the difference between the number of bits of the image frame and the reserved number of bits.
[0101] Optionally, the control module 43 includes:
[0102] A first determining unit, configured to determine a change amount of a quantization parameter QP of the slice compared with a QP of a previous slice of the slice according to the pre-allocated bit rate and an actual bit rate of a previous slice of the slice;
[0103] A control unit is configured to perform rate control on the slice according to the QP of the slice when the change amount is less than or equal to a first threshold; or, when the change amount is greater than the first threshold, set the change amount to the first threshold, determine the sum of the change amount and the QP of a previous slice of the slice as the QP of the slice, and perform rate control on the slice according to the QP of the slice.
[0104] Optionally, if the pre-allocated bit rate of the stripe is calculated according to the SATD value of the stripe, the SATD value of the image frame, and the difference between the number of bits of the image frame and the reserved number of bits, the control module 43 is specifically configured to: when the first condition is met, adjust the QP of the stripe, and perform bit rate control on the stripe according to the adjusted QP of the stripe; or, when the first condition is not met, perform bit rate control on the stripe according to the QP of the stripe;
[0105] The first condition includes at least one of the following:
[0106] A difference between an actual coding rate of the slice determined according to the QP of the slice and a pre-allocated coding rate of the slice is greater than a second threshold;
[0107] The number of bits occupied by the slice based on the pre-allocated code rate of the slice and the actual code rates of all slices before the slice in the image frame exceeds the number of available bits.
[0108] Optionally, the bit rate control device 40 further includes:
[0109] A processing module is used to reduce the number of available bits by a second value during encoding of the strip of the image frame; wherein, after encoding the image frame, the number of available bits is less than or equal to the number of reserved bits.
[0110] Optionally, the strip is divided into x n×m image blocks, where n is the number of pixels included in each column of pixels in the image block, m is the number of pixels included in each row of pixels in the image block, and n and m are integers greater than 1;
[0111] The determination module 41 includes:
[0112] A first calculation unit, configured to obtain a residual matrix of the strip by subtracting a reference pixel value of each pixel from a pixel value of each pixel in the strip;
[0113] A second determining unit, configured to determine a residual matrix corresponding to each of the x image blocks according to the residual matrix of the strip;
[0114] A second calculation unit is used to perform Hadamard transform on the residual matrix corresponding to each image block to obtain the Hadamard coefficient corresponding to each pixel in each image block, and calculate the sum of the absolute values of the Hadamard coefficient corresponding to each pixel to obtain the SATD value of each image block;
[0115] The third determining unit is configured to determine the SATD value of the strip according to the SATD value of each image block.
[0116] Optionally, the image frame includes multiple strips, and if the strip is any strip among the multiple strips except the first strip, the reference pixel value of each pixel is equal to the pixel value of the pixel in the last row of pixels of the previous strip that belongs to the same column as the each pixel; or, if the strip is the first strip among the multiple strips, the reference pixel value of each pixel is obtained based on the pixel value of the pixel in the adjacent column of the image block where the each pixel is located.
[0117] Optionally, the second computing unit is further used for:
[0118] In the reverse order of the zigzag scanning order, the following operations are performed on the Hadamard coefficients corresponding to each pixel in each image block, and the finally obtained vlc_sum is determined as the SATD value of each image block:
[0119] A i The value after right shifting vlc_cnt is added to vlc_sum;
[0120] Among them, the a i represents the Hadamard coefficient corresponding to pixel i; the vlc_cnt is a binary digit, and the initialization value of the vlc_cnt is 0; the vlc_sum is the accumulated value, and the initialization value of the vlc_sum is 0; when a i When it is greater than a third threshold, the value of vlc_cnt is increased by a first value.
[0121] Optionally, the SATD value of the image frame is equal to a weighted sum of a low-frequency luminance SATD value, a low-frequency chrominance SATD value, a high-frequency luminance SATD value, and a high-frequency chrominance SATD value of the image frame.
[0122] The bit rate control device 40 of the embodiment of the present application can achieve the above Figure 1 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.
[0123] Optional, such as Figure 5 As shown, the embodiment of the present application also provides an electronic device 50, including a processor 51, a memory 52, and a program or instruction stored in the memory 52 and executable on the processor 51. When the program or instruction is executed by the processor 51, each process of the above-mentioned bit rate control method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0124] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned bit rate control method embodiment can be implemented and the same technical effect can be achieved. To avoid repetition, they are not described here.
[0125] The embodiment of the present application also provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned bit rate control method embodiment can be implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0126] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0127] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0128] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0129] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0130] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A rate control method, It is characterized in that include: determining absolute transform residuals and SATD values for strips of the image frame; Calculating a pre-allocated bit rate of the stripe according to the SATD value of the stripe and the SATD value of the image frame; The code rate control is performed according to the pre-allocated code rate.
2. The method according to claim 1, It is characterized in that The calculating the pre-allocated bit rate of the stripe according to the SATD value of the stripe and the SATD value of the image frame comprises any one of the following: Calculating a pre-allocated bit rate of the slice according to the SATD value of the slice, the SATD value of the image frame, and the number of bits of the image frame; The pre-allocated bit rate of the slice is calculated according to the SATD value of the slice, the SATD value of the image frame, and the difference between the number of bits of the image frame and the reserved number of bits.
3. The method according to claim 1 or 2, It is characterized in that The performing bit rate control according to the pre-allocated bit rate includes: Determine, according to the pre-allocated bit rate and an actual bit rate of a previous stripe of the stripe, a change amount of a quantization parameter QP of the stripe compared with a QP of a previous stripe of the stripe; When the change amount is less than or equal to a first threshold, rate control is performed on the slice according to the QP of the slice; or, when the change amount is greater than the first threshold, the change amount is set to the first threshold, and the sum of the change amount and the QP of the previous slice of the slice is determined as the QP of the slice, and rate control is performed on the slice according to the QP of the slice.
4. The method according to claim 2, It is characterized in that When the pre-allocated bit rate of the slice is calculated according to the SATD value of the slice, the SATD value of the image frame, and the difference between the number of bits of the image frame and the reserved number of bits, the bit rate control according to the pre-allocated bit rate includes: When the first condition is met, the QP of the slice is adjusted, and the bit rate control of the slice is performed according to the adjusted QP of the slice; or, when the first condition is not met, the bit rate control of the slice is performed according to the QP of the slice; The first condition includes at least one of the following: A difference between an actual coding rate of the slice determined according to the QP of the slice and a pre-allocated coding rate of the slice is greater than a second threshold; The cumulative number of bits occupied based on the pre-allocated code rate of the slice and actual coding rates of all slices before the slice in the image frame exceeds the number of available bits.
5. The method according to claim 4, It is characterized in that The method further comprises: In the process of encoding the slice of the image frame, the number of available bits is reduced by a second value; wherein, after the image frame is encoded, the number of available bits is less than or equal to the number of reserved bits.
6. The method according to claim 1, It is characterized in that The strip is divided into x n×m image blocks, wherein n is the number of pixels included in each column of pixels in the image block, m is the number of pixels included in each row of pixels in the image block, and n and m are integers greater than 1; The determining of the absolute transformation residual and SATD value of the strip of the image frame comprises: Subtracting a reference pixel value of each pixel from a pixel value of each pixel in the strip to obtain a residual matrix of the strip; Determine, according to the residual matrix of the strip, a residual matrix corresponding to each image block in the x image blocks; Performing Hadamard transform on the residual matrix corresponding to each image block to obtain the Hadamard coefficient corresponding to each pixel in each image block, and calculating the sum of the absolute values of the Hadamard coefficients corresponding to each pixel to obtain the SATD value of each image block; The SATD value of the strip is determined according to the SATD value of each image block.
7. The method according to claim 6, It is characterized in that The image frame includes a plurality of strips, and if the strip is any strip of the plurality of strips except the first strip, the reference pixel value of each pixel is equal to the pixel value of a pixel in the last row of pixels of a strip previous to the strip that belongs to the same column as the each pixel; or, If the stripe is the first stripe of the multiple stripes, the reference pixel value of each pixel is obtained based on the pixel value of the adjacent column pixel of the image block where the each pixel is located.
8. The method according to claim 6, It is characterized in that Calculating the sum of the absolute values of the Hadamard coefficients corresponding to each pixel to obtain the SATD value of each image block includes: In the reverse order of the zigzag scanning order, the following operations are performed on the Hadamard coefficients corresponding to each pixel in each image block, and the finally obtained vlc_sum is determined as the SATD value of each image block: A i The value after right shifting vlc_cnt is added to vlc_sum; Among them, the a i represents the Hadamard coefficient corresponding to pixel i; the vlc_cnt is a binary digit, and the initialization value of the vlc_cnt is 0; the vlc_sum is the accumulated value, and the initialization value of the vlc_sum is 0; when a i When it is greater than a third threshold, the value of vlc_cnt is increased by a first value.
9. The method according to claim 1, It is characterized in that The SATD value of the image frame is equal to a weighted sum of a low-frequency luminance SATD value, a low-frequency chrominance SATD value, a high-frequency luminance SATD value, and a high-frequency chrominance SATD value of the image frame.
10. A bit rate control device, It is characterized in that include: A determination module, used for determining the SATD value of a strip of an image frame; A calculation module, used for calculating the pre-allocated bit rate of the strip according to the SATD value of the strip and the SATD value of the image frame; A control module is used to perform bit rate control according to the pre-allocated bit rate.
11. An electronic device, It is characterized in that The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 9.
12. A readable storage medium, It is characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.
13. A computer program product, It is characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9.
Citation Information
Cited By
Shallow compression multi-stripe code rate control method and device and electronic equipment
CN119484832A
