Image frame code rate determination method and device, computer equipment and chip

Abstract

The invention relates to an image frame code rate determination method and device, computer equipment and a chip. The method comprises the following steps: acquiring first image parameters of a plurality of coding blocks in a historical image frame, determining second image parameters of each coding block according to the first image parameters, and determining a target code rate of each coding block according to each second image parameter and an original code rate of the historical image frame. By adopting the method, the display effect of the image frame can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, computer device, and chip for determining the bit rate of an image frame. Background Technology

[0002] With the continuous development of image processing technology and the continuous improvement of software and hardware, users have higher and higher demands for image processing. Users expect to achieve better image display effects on image display devices. Currently, image quality is usually measured by parameters such as mean square error, sum of squared errors, and peak signal-to-noise ratio, so as to achieve better image display effects through parameter control.

[0003] However, image display effects involve many and complex parameters, and how to better improve image display effects has become a major concern. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, and chip for determining the bit rate of image frames that can improve the display effect of image frames, in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for determining the bitrate of an image frame, comprising: acquiring first image parameters of multiple coded blocks in a historical image frame, the first image parameters being used to characterize the permissible degree of brightness distortion and / or texture distortion; for each coded block, determining second image parameters of the coded block based on the first image parameters, the second image parameters being used to characterize the permissible degree of image distortion; and determining a target bitrate for each coded block based on each second image parameter and the original bitrate of the historical image frame, the target bitrate being used to encode the current image frame.

[0006] In one embodiment, obtaining the first image parameters of multiple coded blocks in a historical image frame includes: determining the luminance distortion parameter corresponding to each coded block based on the pixel value of each coded block; determining the texture distortion parameter corresponding to each coded block based on the pixel value and the luminance distortion parameter; and determining the luminance distortion parameter and the texture distortion parameter as the first image parameters.

[0007] In one embodiment, determining the luminance distortion parameter corresponding to each coding block based on the pixel value of each coding block includes: for each coding block, determining the average background luminance based on the pixel value and luminance operator corresponding to the coding block; determining the target parameter calculation function based on the average background luminance, and substituting the average background luminance into the target parameter calculation function to calculate the luminance distortion parameter.

[0008] In one embodiment, determining the texture distortion parameters corresponding to each coding block based on each pixel value and the brightness distortion parameter includes: for each coding block, calculating multiple gradients of the coding block based on the pixel value corresponding to the coding block and each gradient operator, wherein the image pixel directions corresponding to each gradient operator are different; and determining the texture distortion parameters based on the brightness distortion parameters and each gradient.

[0009] In one embodiment, determining the second image parameter of the coded block based on the first image parameter includes: selecting the image parameter with the largest value from the brightness distortion parameter and the texture distortion parameter; and determining the ratio of the reference distortion degree to the image parameter with the largest value as the second image parameter.

[0010] In one embodiment, the target bitrate of each coding block is determined based on each second image parameter and the original bitrate of historical image frames, including: summing the second image parameters corresponding to each coding block to obtain the image parameter sum value; for each coding block, calculating the ratio of the second image parameters to the image parameter sum value, and multiplying the ratio by the original bitrate to obtain the target bitrate.

[0011] In one embodiment, the method further includes: determining the quantization parameters of each target coding block of the current image frame according to the target bitrate, wherein the current image frame and the historical image frames are sequentially continuous, and the positions of each target coding block and each coding block correspond one-to-one; and performing image encoding on the current image frame according to each quantization parameter to obtain an image bitstream.

[0012] Secondly, this application also provides a bitrate determination device for an image frame, comprising: a first parameter acquisition module, configured to acquire first image parameters of multiple coded blocks in a historical image frame, the first image parameters being used to characterize the permissible degree of brightness distortion and / or texture distortion; a second parameter acquisition module, configured to determine second image parameters of each coded block based on the first image parameters, the second image parameters being used to characterize the permissible degree of image distortion; and a bitrate determination module, configured to determine a target bitrate for each coded block based on each second image parameter and the original bitrate of the historical image frame, the target bitrate being used to encode the current image frame.

[0013] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method provided in the first aspect.

[0014] Fourthly, this application also provides a chip, including a processor and a communication interface, wherein the processor is configured to enable the chip to implement the steps of the method provided in the first aspect above.

[0015] Fifthly, this application also provides a chip module, including a communication module, a power module, a storage module, and a chip, wherein: the power module is used to provide electrical energy to the chip module; the storage module is used to store data and instructions; the communication module is used for internal communication within the chip module, or for communication between the chip module and external devices; and the chip is used to perform the steps of the method provided in the first aspect above.

[0016] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0017] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0018] The aforementioned image frame bitrate determination method, apparatus, computer equipment, and chip first acquire the first image parameters of multiple coded blocks of historical image frames, i.e., the previous image frame of the current image frame. The first image parameters characterize the allowable degree of brightness distortion and / or texture distortion. For each coded block, the second image parameters of the coded block are determined based on the first image parameters. The second image parameters characterize the allowable degree of image distortion. Based on each second image parameter and the original bitrate of the historical image frame, the target bitrate of each coded block is determined. Each target bitrate is used to encode the current image frame. By incorporating JND (Just Noticeable Difference) into the image encoding process, encoding guided by the user's visual perception is achieved, thereby making the display effect of the current image frame more in line with the user's visual experience and improving the image display effect. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an application environment diagram of the image frame bitrate determination method in one embodiment;

[0021] Figure 2 This is a flowchart illustrating a method for determining the bitrate of an image frame in one embodiment.

[0022] Figure 3 This is a flowchart illustrating step 201 in one embodiment;

[0023] Figure 4 This is a flowchart illustrating step 301 in one embodiment;

[0024] Figure 5 This is a flowchart illustrating the steps for determining texture distortion parameters in one embodiment.

[0025] Figure 6 This is a flowchart illustrating step 202 in one embodiment;

[0026] Figure 7 This is a flowchart illustrating step 203 in one embodiment;

[0027] Figure 8 This is a flowchart illustrating the encoding steps for the current image frame in one embodiment;

[0028] Figure 9 This is a flowchart illustrating the method for determining the bitrate of an image frame in another embodiment;

[0029] Figure 10 This is a structural block diagram of an image frame bitrate determination device in one embodiment;

[0030] Figure 11 This is an internal structural diagram of a computer device in one embodiment;

[0031] Figure 12 This is an internal structure diagram of a chip module in one embodiment. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0034] The image frame bitrate determination method provided in this application embodiment can be applied to, for example, Figure 1 The application environment shown includes at least encoding device 101 and terminal device 102.

[0035] The encoding device 101 acquires image frames sent by the terminal device 102 and obtains first image parameters of multiple encoding blocks in historical image frames. For each encoding block, it determines second image parameters based on the first image parameters, and determines the target bitrate of each encoding block based on the second image parameters and the original bitrate of the historical image frames. Each target bitrate is used to encode the current image frame. The encoding device 101 can be a hardware device with video / image encoding capabilities, such as an encoder, or a server with encoding capabilities. This server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Chips or chip modules can be deployed in the encoding device 101 to achieve the above functions.

[0036] Terminal device 102 can be a device with image transmission capabilities, capable of sending image frames to encoding device 101. Furthermore, terminal device 102 can also capture video / images and transmit the captured video / images to encoding device 101 in real time. Terminal device 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, cameras, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Encoding device 101 communicates with terminal device 102 via a network.

[0037] In practical video coding scenarios, bitrate control aims to achieve optimal video quality within a given bitrate limit. Currently, objective evaluation methods for measuring video quality mainly include mean square error, sum of squared errors, and peak signal-to-noise ratio. However, these methods do not fully reflect the characteristics of the human visual system. High or low quality parameters do not necessarily represent high or low visual quality. Later, the SSIM (Structural Similarity Index Measure) was introduced to guide video coding. However, research has shown that the SSIM evaluation metric still cannot fully characterize the characteristics of the human visual system. To better provide the human eye with a better perceived quality, the JND (Joint Distance) perceptual model was proposed. The JND model can calculate the human visual threshold. Pixel changes below this threshold will not attract the human eye's attention. The lower the JND threshold, the easier it is for the human eye to detect distortion, and vice versa. Therefore, the JND model can well characterize the main features of the human visual system.

[0038] This application integrates the JND model into a hybrid video coding framework, leveraging the characteristics of JND for video coding. This effectively reduces visual redundancy and the required bitrate for video coding without significantly degrading visual perception quality. The specific principle is as follows: The human visual system was modeled through a series of experiments under specific conditions, including regions of interest (ROIs), saliency detection, and JND. The JND visual perception model effectively quantifies visual redundancy. Combined with JND's perceptual video coding bitrate control, it can improve the visual perception quality of the human eye within a certain target bitrate range. The JND visual perception model is mainly divided into pixel-domain JND and transform-domain JND models. The pixel-domain JND model measures visual redundancy in the pixel domain; it is simple and computationally inexpensive. The transform-domain JND model measures visual redundancy in the transform domain; it is complex and computationally intensive, but the results are more accurate.

[0039] In this application, to balance coding efficiency and complexity, we use the pixel-domain JND model for video coding. The pixel-domain JND model mainly includes texture masking, brightness masking, and other models. Through the interaction of these models, the JND value of the pixel can be obtained. If the pixel distortion is less than the JND value, the human eye will not perceive this distortion. Conversely, if the pixel distortion is greater than or equal to the JND value, the human eye will perceive this distortion. This application can compress the original bitrate while ensuring the image display effect, thereby improving the image coding effect.

[0040] In one exemplary embodiment, such as Figure 2 As shown, a method for determining the bitrate of an image frame is provided, which can be applied to... Figure 1Encoding devices in, or those applied to Figure 1 The following steps are used as an example of a chip / chip module with data processing capabilities, including steps 201 to 203.

[0041] Step 201: Obtain the first image parameters of multiple coded blocks in the historical image frame.

[0042] In this application, the first image parameter refers to the JND parameter of the image frame. This image parameter is used to characterize the minimum level of distortion that the human eye can generally perceive. The first image parameter may include a luminance distortion parameter and / or a texture distortion parameter. The luminance distortion parameter can be used to characterize the permissible level of luminance distortion, or to characterize the maximum permissible level of luminance distortion. The texture distortion parameter can be used to characterize the permissible level of texture distortion, or to characterize the maximum permissible level of luminance distortion. The first image parameter is used to characterize the permissible level of luminance distortion and / or texture distortion.

[0043] Among them, an image frame refers to an image frame sent by a terminal device to an encoding device. The terminal device can send a video to the encoding device at one time, and the video includes multiple consecutive image frames; or, the terminal device can send image frames to the encoding device in real time, and the terminal device receives the image frames in chronological order.

[0044] In the case where the terminal device sends video to the encoding device all at once, the historical image frame refers to the previous image frame in the time sequence of the currently processed image frame; in the case where the terminal device sends image frames to the encoding device in real time, the historical image frame refers to the image frame at the previous moment of the currently processed image frame.

[0045] During implementation, the encoding device determines the first image parameters for each encoding block based on the pixel values ​​of each encoding block in historical image frames. During execution, the encoding device may determine the luminance distortion parameters for each encoding block, or the texture distortion parameters for each encoding block, or both luminance distortion parameters and texture distortion parameters for each encoding block.

[0046] Step 202: For each coded block, determine the second image parameters of the coded block based on the first image parameters.

[0047] In this application, the second image parameter is an image parameter determined based on the first image parameter that best represents the permissible image distortion level of the coded block. The second image parameter is used to characterize the permissible image distortion level.

[0048] During implementation, for each coding block, the coding device determines the second image parameters of the coding block based on the first image parameters; during execution, the coding device may determine the second image parameters of the coding block based on the brightness distortion parameters and / or texture distortion parameters.

[0049] Step 203: Determine the target bitrate of each coding block based on the original bitrates of each second image parameter and historical image frames.

[0050] In this application, for the two scenarios of transmitting image frames described above, the original bitrate is fixed. The bitrate represents the amount of data transmitted per unit time. In this application, the target bitrate for each coding block can be different. Each target bitrate is used to encode the current image frame. The target bitrate is less than the original bitrate.

[0051] During implementation, the encoding device corrects the original bitrate of historical image frames based on the second image parameters of each coded block, obtaining the target bitrate for each coded block. Then, it encodes the image regions corresponding to each coded block in the current image frame based on the target bitrate of each coded block. This achieves the allocation of the current image frame's bitrate according to historical image frames, using the JND factor of each coded block in the historical image frames as weights. This reduces the bitrate and improves the transmission efficiency of image frames without affecting the user's visual experience.

[0052] In the above-mentioned method for determining the bitrate of an image frame, the first image parameters of multiple coding blocks of historical image frames (i.e., the previous image frame of the current image frame) are first obtained. The first image parameters represent the allowable degree of brightness distortion and / or texture distortion. For each coding block, the second image parameters of the coding block are determined based on the first image parameters. The second image parameters represent the allowable degree of image distortion. Based on each second image parameter and the original bitrate of the historical image frame, the target bitrate of each coding block is determined. Each target bitrate is used to encode the current image frame. By adding JND to the image encoding process, encoding guided by the user's visual perception is achieved, thereby making the display effect of the current image frame more in line with the user's visual experience and improving the image display effect.

[0053] Based on the above exemplary embodiment, the following provides a method for determining the bitrate of an image frame in one or more exemplary embodiments, which is applied to... Figure 1 Taking the encoding device in the example, the following content is specifically included.

[0054] In the process of obtaining the first image parameters, the first image parameters may include brightness distortion parameters and texture distortion parameters, and the brightness distortion parameters and texture distortion parameters are jointly determined as the first image parameters; in one optional embodiment provided by this application, such as Figure 3 As shown, step 201 includes steps 301 to 302:

[0055] Step 301: Determine the luminance distortion parameters corresponding to each coding block based on the pixel values ​​of each coding block.

[0056] During implementation, the encoding device first acquires the pixel values ​​of each encoding block in the historical image frame, and determines the brightness distortion parameter corresponding to each encoding block based on each pixel value and the brightness operator. The brightness distortion parameter can be determined based on the average background brightness, which is used to characterize the minimum brightness distortion perceived by the user under normal circumstances, or to characterize the minimum allowable brightness distortion.

[0057] Step 302: Determine the texture distortion parameters corresponding to each coding block based on each pixel value and the brightness distortion parameters, and determine each brightness distortion parameter and each texture distortion parameter as the first image parameters.

[0058] During implementation, the encoding device calculates the texture distortion parameter of each encoding block based on the pixel value and the brightness distortion parameter of each encoding block, and determines the brightness distortion parameter and texture distortion parameter together as the second image parameter; wherein, the texture distortion parameter can be determined based on the texture mask value, which is used to characterize the minimum distortion perception of the texture by the user under normal circumstances, or to characterize the minimum allowable texture distortion.

[0059] One optional implementation provided in this application is to use both brightness distortion parameters and texture distortion parameters as the first image parameters, so that the encoding device can determine the second image parameters from multiple dimensions, thereby improving the display effect of the image frame.

[0060] In determining the luminance distortion parameters, a luminance operator can be pre-set, and the luminance distortion parameters can be calculated based on the luminance operator and the luminance distortion calculation function; in one optional implementation provided in this application, such as Figure 4 As shown, step 301 includes steps 401 to 402:

[0061] Step 401: For each coded block, determine the average background brightness based on the pixel value and brightness operator corresponding to the coded block.

[0062] During implementation, for each coding block, the average background brightness is first calculated based on the pixel values ​​of the coding block, and then the target parameter calculation function is selected based on the average background brightness.

[0063] During execution, the pixel values ​​of the encoded block are first substituted into the calculation function for the average background brightness to obtain the average background brightness of the encoded block; for example, see formula (1):

[0064] Formula (1);

[0065] in, This represents the average background brightness operator. Indicates coordinates as The pixel values, the average background brightness operator can be: .

[0066] Step 402: Determine the target parameter calculation function based on the average background brightness, and substitute the average background brightness into the target parameter calculation function to calculate the brightness distortion parameter.

[0067] During implementation, based on the average background brightness value, a matching target parameter calculation function is selected from multiple parameter calculation functions. The average background brightness is then substituted into the target parameter calculation function for calculation to obtain the brightness distortion parameter, namely: the JND parameter.

[0068] For example, calculate the pixel-domain JND value of each coding block in the previous frame, then calculate the JND factor based on the JND value and allocate the target bit rate to the coding block. The calculation process of the brightness masking model corresponding to the image pixels in the pixel-domain JND model is shown in formula (2):

[0069] Formula (2)

[0070] in, Indicates coordinates as The average background brightness corresponding to the pixel is calculated. If the average background brightness is less than or equal to 127, the first formula is selected; if it is greater than 127, the second formula is selected.

[0071] One optional implementation provided in this application calculates the luminance distortion parameters by means of average background luminance, thereby improving the reliability of the luminance distortion parameters and ensuring the image display effect.

[0072] In determining texture distortion parameters, these parameters can be calculated based on the pixel values ​​of the coded blocks, brightness distortion parameters, and map operators in each pixel direction. One optional implementation provided in this application is as follows: Figure 5 As shown, the calculation process for texture distortion parameters includes steps 501 to 502:

[0073] Step 501: For each coding block, calculate multiple gradients of the coding block based on the pixel values ​​corresponding to the coding block and each gradient operator.

[0074] During implementation, for each coding block, multiple gradients corresponding to the coding block and gradient operators are calculated based on multiple gradient operators and pixel values. The gradient can represent the maximum absolute value of the gradient in that direction. The image pixel directions corresponding to each gradient operator are different. Optionally, the gradient operators include four directions.

[0075] For example, This represents the maximum absolute value of the gradient from the image pixel in four directions. The calculation process is shown in formulas (3) and (4):

[0076] Formula (3);

[0077] Formula (4);

[0078] Among them, there are 4 gradient operators The directions are as follows:

[0079] .

[0080] Step 502: Determine the texture distortion parameters based on the brightness distortion parameters and each gradient.

[0081] During implementation, the encoding device first calculates the texture brightness based on the brightness distortion parameters, and then determines the texture distortion parameters according to each gradient and the texture brightness.

[0082] For example, the calculation process of the texture masking model can be found in formulas (5) to (8):

[0083] Formula (5);

[0084] Formula (6);

[0085] Formula (7);

[0086] in, A function representing the average background brightness.

[0087] One optional implementation provided in this application calculates texture distortion parameters by average background brightness, thereby determining the minimum level of texture distortion perceived by the user. This allows image encoding to be performed without affecting the user's visual perception during the subsequent bitrate determination process.

[0088] In determining the second image parameter, the image parameter with the largest value can be selected from the brightness distortion parameter and the texture distortion parameter, and the second image parameter can be determined based on the largest image parameter; in one optional implementation provided by this application, such as Figure 6 As shown, step 202 includes steps 601 to 602:

[0089] Step 601: Select the image parameter with the largest value from the brightness distortion parameter and the texture distortion parameter.

[0090] During implementation, the encoding device selects the image parameter with the largest value from the brightness distortion parameter and the texture distortion parameter.

[0091] For example, refer to formula (8):

[0092] Formula (8);

[0093] in, This is a brightness distortion parameter. This is the texture distortion parameter.

[0094] Step 602: The ratio of the reference distortion to the image parameter with the largest value is determined as the second image parameter.

[0095] During implementation, a pre-set reference distortion is determined. The reference distortion is used as the numerator, and the image parameter with the largest value is used as the denominator to calculate the ratio. This ratio is then determined as the second image parameter. Optionally, the reference distortion is 1, which represents the unit of change of JND.

[0096] For example, see formula (9):

[0097] Formula (9);

[0098] in, The image parameter with the largest value.

[0099] One optional implementation provided in this application selects the image parameter with the largest value among the brightness distortion parameter and the texture distortion parameter, and selects the parameter that has the greatest impact on the user's visual perception for bitrate calculation. This saves computing power without affecting the user's visual experience, thereby improving the encoding effect and image display effect.

[0100] In determining the target bitrate, image parameters from historical image frames can be used as weights to determine the bitrate allocation for the current image frame. Since a larger second image parameter indicates greater sensitivity to distortion in the coded block at that location, the bitrate for the portion with a larger second image parameter can be increased, while the bitrate for the portion with a smaller second image parameter can be decreased. In one optional implementation provided by this application, such as... Figure 7 As shown, step 203 includes steps 701 to 702:

[0101] Step 701: Summate the second image parameters corresponding to each coding block to obtain the image parameter sum value.

[0102] During implementation, the encoding device sums the second image parameters corresponding to each encoding block to obtain the image parameter sum value; during execution, the target bitrate allocation of the current frame encoding block is guided by the target bitrate of the current frame and the JND factor of the encoding block of the previous frame.

[0103] Step 702: For each coding block, calculate the ratio of the second image parameter to the image parameter and value, and multiply the ratio by the original bit rate to obtain the target bit rate.

[0104] During implementation, for each coding block, the ratio of the second image parameter of that coding block to the image parameter and value is calculated, and the ratio is multiplied by the original bitrate to obtain the target bitrate, which is then used as a weight for bitrate allocation.

[0105] For example, see formula (10):

[0106] Formula (10);

[0107] in This indicates the raw bitrate of the current frame. The second image parameter represents the coded block at the same position in the previous frame where the current coded block is located. This represents the sum of the second image parameters of all coding units in the previous frame.

[0108] One optional implementation provided in this application performs bitrate allocation processing by calculating the ratio between the second image parameters and the second image parameters. The weight of bitrate allocation is determined based on historical image frames, which improves the usability of bitrate allocation, enhances image display effect, and saves video bitrate.

[0109] In practical scenarios, after obtaining the target bitrate, the current image frame can be encoded based on the target bitrate to obtain the image bitstream; in one optional implementation provided in this application, such as Figure 8 As shown, the method further includes steps 801 to 802:

[0110] Step 801: Determine the quantization parameters of each target coding block of the current image frame based on the target bit rate.

[0111] During implementation, the encoding device determines the initial quantization parameters of each target coding block in the current image frame based on the target bitrate, calculates additional quantization parameters based on the initial quantization parameters and the current fullness, and determines the final quantization parameters based on the sum of the initial and additional quantization parameters. The current image frame and historical image frames are sequentially continuous, and each target coding block corresponds one-to-one with the position of each coding block.

[0112] For example, first calculate the initial QP required for encoding using the R-lambda model, see formulas (11)-(12):

[0113] Formula (11);

[0114] Formula (12);

[0115] in, , This represents the parameters of the R-lambda model. The target bitrate.

[0116] Step 802: Encode the current image frame according to each quantization parameter to obtain the image bitstream.

[0117] During implementation, each target coding block of the current image frame is encoded and compressed according to each quantization parameter to obtain the image bitstream of the current image frame.

[0118] One optional implementation method provided in this application determines the quantization parameters by a target bitrate, and then encodes the current image frame using the quantization parameters to obtain an image bitstream. This achieves bitrate compression without affecting the user's visual experience, thereby improving the encoding effect of the image frame while ensuring the display effect of the image frame.

[0119] In one embodiment, see Figure 9 The document illustrates a flowchart of a method for determining the bitrate of an image frame according to an embodiment of this application. This method can be applied to... Figure 1 In the encoding device shown. For example... Figure 9 As shown, the method for determining the bitrate of this image frame may include the following steps:

[0120] Step 901: Obtain the pixel values ​​of multiple coded blocks of the historical image frame.

[0121] Step 902: For each coded block, determine the average background brightness based on the pixel value and brightness operator corresponding to the coded block.

[0122] Step 903: Determine the target parameter calculation function based on the average background brightness, and substitute the average background brightness into the target parameter calculation function to calculate the brightness distortion parameter.

[0123] Step 904: Calculate multiple gradients of the coding block based on the pixel values ​​corresponding to the coding block and each gradient operator.

[0124] Step 905: Determine the texture distortion parameters based on the brightness distortion parameters and each gradient.

[0125] Step 906: Among the brightness distortion parameter and the texture distortion parameter, select the image parameter with the largest value.

[0126] Step 907: The ratio of the reference distortion to the image parameter with the largest value is determined as the second image parameter.

[0127] Step 908: Summing up the second image parameters corresponding to each coding block to obtain the image parameter sum value.

[0128] Step 909: For each coding block, calculate the ratio of the second image parameter to the image parameter and value, and multiply the ratio by the original bit rate to obtain the target bit rate.

[0129] It should be noted that any one or more of steps 901 to 909 can be combined to form a new implementation method according to the needs of implementation and deployment. Furthermore, any one or more technical features in the technical solution composed of steps 901 to 909 can also be combined to form a new implementation method according to the actual deployment needs, or technical features in one or more optional implementations provided by one or more of the above embodiments can be combined to form a new implementation method. These will not be elaborated on here.

[0130] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0131] Based on the same inventive concept, this application also provides an image frame bitrate determination apparatus for implementing the image frame bitrate determination method described above. This apparatus can be applied to or integrated into a chip or chip module, for example. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more image frame bitrate determination apparatus embodiments provided below can be found in the limitations of the image frame bitrate determination method described above, and will not be repeated here.

[0132] In one exemplary embodiment, such as Figure 10As shown, an image frame bitrate determination device is provided, comprising: a first parameter acquisition module 1001, a second parameter acquisition module 1002, and a bitrate determination module 1003, wherein: the first parameter acquisition module 1001 is used to acquire first image parameters of multiple coding blocks in historical image frames, the first image parameters being used to characterize the allowable degree of brightness distortion and / or texture distortion; the second parameter acquisition module 1002 is used to determine second image parameters of each coding block based on the first image parameters, the second image parameters being used to characterize the allowable degree of image distortion; the bitrate determination module 1003 is used to determine the target bitrate of each coding block based on each second image parameter and the original bitrate of the historical image frames, the target bitrate being used to encode the current image frame.

[0133] In one embodiment, the first parameter acquisition module 1001 includes a first determining unit and a second determining unit, wherein: the first determining unit is used to determine the luminance distortion parameter corresponding to each coding block according to the pixel value of each coding block; the second determining unit is used to determine the texture distortion parameter corresponding to each coding block according to each pixel value and the luminance distortion parameter, and determine each luminance distortion parameter and each texture distortion parameter as the first image parameter.

[0134] In one embodiment, the first determining unit includes a calculation function determining unit and a luminance distortion parameter calculating unit, wherein: the calculation function determining unit is used to determine the average background luminance for each coding block based on the pixel value and luminance operator corresponding to the coding block; the luminance distortion parameter calculating unit is used to determine the target parameter calculation function based on the average background luminance, and substitute the average background luminance into the target parameter calculation function to calculate the luminance distortion parameter.

[0135] In one embodiment, the second determining unit includes a gradient calculation unit and a texture distortion parameter determining unit, wherein: the gradient calculation unit is used to calculate multiple gradients of the coding block for each coding block based on the pixel value corresponding to the coding block and each gradient operator, and the image pixel direction corresponding to each gradient operator is different; the texture distortion parameter determining unit is used to determine the texture distortion parameter based on the brightness distortion parameter and each gradient.

[0136] In one embodiment, the second parameter acquisition module 1002 includes a parameter filtering unit and a second parameter determination unit, wherein: the parameter filtering unit is used to filter out the image parameter with the largest value among the brightness distortion parameter and the texture distortion parameter; the second parameter determination unit is used to determine the ratio of the reference distortion degree to the image parameter with the largest value as the second image parameter.

[0137] In one embodiment, the bitrate determination module 1003 includes a summation unit and a bitrate calculation unit, wherein: the summation unit is used to sum the second image parameters corresponding to each coding block to obtain the image parameter sum value; the bitrate calculation unit is used to calculate the ratio of the second image parameter to the image parameter sum value for each coding block, and multiply the ratio by the original bitrate to obtain the target bitrate.

[0138] In one embodiment, the apparatus further includes a quantization parameter determination unit and an image encoding unit, wherein: the quantization parameter determination unit is used to determine the quantization parameters of each target coding block of the current image frame according to the target bit rate, the current image frame and the historical image frames are sequentially continuous, and the positions of each target coding block and each coding block correspond one-to-one; the image encoding unit is used to perform image encoding on the current image frame according to each quantization parameter to obtain an image bitstream.

[0139] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits; for various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0140] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores bitrate determination data for image frames. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a bitrate determination method for image frames.

[0141] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring first image parameters of a plurality of coded blocks in a historical image frame, the first image parameters being used to characterize the permissible degree of brightness distortion and / or texture distortion; for each coded block, determining second image parameters of the coded block based on the first image parameters, the second image parameters being used to characterize the permissible degree of image distortion; and determining a target bitrate for each coded block based on each second image parameter and the original bitrate of the historical image frame, the target bitrate being used to encode the current image frame.

[0143] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the luminance distortion parameter corresponding to each coding block based on the pixel value of each coding block; determining the texture distortion parameter corresponding to each coding block based on the pixel value and the luminance distortion parameter, and determining the luminance distortion parameter and the texture distortion parameter as the first image parameter.

[0144] In one embodiment, when the processor executes the computer program, it further performs the following steps: for each coded block, determining the average background brightness based on the pixel value and brightness operator corresponding to the coded block; determining the target parameter calculation function based on the average background brightness, and substituting the average background brightness into the target parameter calculation function to calculate the brightness distortion parameter.

[0145] In one embodiment, when the processor executes the computer program, it further performs the following steps: for each coded block, calculates multiple gradients of the coded block based on the pixel values ​​corresponding to the coded block and each gradient operator, wherein the image pixel directions corresponding to each gradient operator are different; and determines the texture distortion parameters based on the brightness distortion parameters and each gradient.

[0146] In one embodiment, when the processor executes the computer program, it further performs the following steps: selecting the image parameter with the largest value from the brightness distortion parameter and the texture distortion parameter; and determining the ratio of the reference distortion degree to the image parameter with the largest value as the second image parameter.

[0147] In one embodiment, when the processor executes the computer program, it further performs the following steps: summing the second image parameters corresponding to each coding block to obtain the image parameter sum value; for each coding block, calculating the ratio of the second image parameter to the image parameter sum value, and multiplying the ratio by the original bit rate to obtain the target bit rate.

[0148] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the quantization parameters of each target coding block of the current image frame according to the target bitrate, wherein the current image frame and the historical image frames are sequentially continuous, and the positions of each target coding block and each coding block correspond one-to-one; and performing image encoding on the current image frame according to each quantization parameter to obtain an image bitstream.

[0149] Based on the same inventive concept, embodiments of this application also provide a chip, including a processor and a communication interface; the communication interface is used to receive or send data; the processor is configured to cause the chip to perform the following steps: acquiring first image parameters of multiple coded blocks in a historical image frame, the first image parameters being used to characterize the allowable degree of brightness distortion and / or texture distortion; for each coded block, determining second image parameters of the coded block based on the first image parameters, the second image parameters being used to characterize the allowable degree of image distortion; determining a target bitrate for each coded block based on each second image parameter and the original bitrate of the historical image frame, each target bitrate being used to encode the current image frame.

[0150] In one embodiment, the processor is configured to cause the chip to perform the following steps: determining a luminance distortion parameter corresponding to each coding block based on the pixel value of each coding block; determining a texture distortion parameter corresponding to each coding block based on the pixel value and the luminance distortion parameter; and determining the luminance distortion parameter and the texture distortion parameter as a first image parameter.

[0151] In one embodiment, the processor is configured to cause the chip to perform the following steps: for each coded block, determine the average background brightness based on the pixel value and brightness operator corresponding to the coded block; determine a target parameter calculation function based on the average background brightness, and substitute the average background brightness into the target parameter calculation function to calculate the brightness distortion parameter.

[0152] In one embodiment, the processor is configured to cause the chip to perform the following steps: for each coding block, calculating multiple gradients of the coding block based on the pixel values ​​corresponding to the coding block and each gradient operator, wherein each gradient operator corresponds to a different image pixel orientation; and determining texture distortion parameters based on brightness distortion parameters and each gradient.

[0153] In one embodiment, the processor is configured to cause the chip to perform the following steps: selecting the image parameter with the largest value from the brightness distortion parameter and the texture distortion parameter; and determining the ratio of a reference distortion degree to the image parameter with the largest value as a second image parameter.

[0154] In one embodiment, the processor is configured to cause the chip to perform the following steps: summing the second image parameters corresponding to each coding block to obtain the image parameter sum value; for each coding block, calculating the ratio of the second image parameter to the image parameter sum value, and multiplying the ratio by the original bit rate to obtain the target bit rate.

[0155] In one embodiment, the processor is configured to cause the chip to perform the following steps: determine the quantization parameters of each target coding block of the current image frame according to the target bitrate, wherein the current image frame and historical image frames are sequentially continuous, and the positions of each target coding block and each coding block correspond one-to-one; and perform image encoding on the current image frame according to each quantization parameter to obtain an image bitstream.

[0156] It is understood that the chip involved in the embodiments of this application may be a field-programmable gate array (FPGA), may be an application-specific integrated circuit (ASIC), may be a system on chip (SoC), may be a central processor unit (CPU), may be a network processor (NP), may be a digital signal processor (DSP), may be a microcontroller unit (MCU), may be a programmable logic device (PLD), or other integrated chips, etc.

[0157] Based on the same inventive concept, this application also provides a chip module, such as... Figure 12 As shown, the chip module includes a communication module, a power module, a storage module, and a chip. Specifically: the power module provides power to the chip module; the storage module stores data and instructions; the communication module enables internal communication within the chip module or communication between the chip module and external devices; and the chip corresponds to the chip in the aforementioned chip embodiment.

[0158] The implementation method of this chip module can be found in the relevant content of the above chip embodiment, and will not be repeated here.

[0159] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon. When executed by a processor, the computer program performs the following steps: obtaining first image parameters of a plurality of coded blocks in a historical image frame, the first image parameters being used to characterize the permissible degree of brightness distortion and / or texture distortion; for each coded block, determining second image parameters of the coded block based on the first image parameters, the second image parameters being used to characterize the permissible degree of image distortion; and determining a target bitrate for each coded block based on each second image parameter and the original bitrate of the historical image frame, the target bitrate being used to encode the current image frame.

[0160] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the luminance distortion parameter corresponding to each coding block based on the pixel value of each coding block; determining the texture distortion parameter corresponding to each coding block based on the pixel value and the luminance distortion parameter, and determining the luminance distortion parameter and the texture distortion parameter as the first image parameter.

[0161] In one embodiment, when the processor executes the computer program, it further performs the following steps: for each coded block, determining the average background brightness based on the pixel value and brightness operator corresponding to the coded block; determining the target parameter calculation function based on the average background brightness, and substituting the average background brightness into the target parameter calculation function to calculate the brightness distortion parameter.

[0162] In one embodiment, when the processor executes the computer program, it further performs the following steps: for each coded block, calculates multiple gradients of the coded block based on the pixel values ​​corresponding to the coded block and each gradient operator, wherein the image pixel directions corresponding to each gradient operator are different; and determines the texture distortion parameters based on the brightness distortion parameters and each gradient.

[0163] In one embodiment, when the processor executes the computer program, it further performs the following steps: selecting the image parameter with the largest value from the brightness distortion parameter and the texture distortion parameter; and determining the ratio of the reference distortion degree to the image parameter with the largest value as the second image parameter.

[0164] In one embodiment, when the processor executes the computer program, it further performs the following steps: summing the second image parameters corresponding to each coding block to obtain the image parameter sum value; for each coding block, calculating the ratio of the second image parameter to the image parameter sum value, and multiplying the ratio by the original bit rate to obtain the target bit rate.

[0165] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the quantization parameters of each target coding block of the current image frame according to the target bitrate, wherein the current image frame and the historical image frames are sequentially continuous, and the positions of each target coding block and each coding block correspond one-to-one; and performing image encoding on the current image frame according to each quantization parameter to obtain an image bitstream.

[0166] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: acquiring first image parameters of a plurality of coded blocks in a historical image frame, the first image parameters being used to characterize the permissible degree of brightness distortion and / or texture distortion; for each coded block, determining second image parameters of the coded block based on the first image parameters, the second image parameters being used to characterize the permissible degree of image distortion; and determining a target bitrate for each coded block based on each second image parameter and the original bitrate of the historical image frame, the target bitrate being used to encode the current image frame.

[0167] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the luminance distortion parameter corresponding to each coding block based on the pixel value of each coding block; determining the texture distortion parameter corresponding to each coding block based on the pixel value and the luminance distortion parameter, and determining the luminance distortion parameter and the texture distortion parameter as the first image parameter.

[0168] In one embodiment, when the processor executes the computer program, it further performs the following steps: for each coded block, determining the average background brightness based on the pixel value and brightness operator corresponding to the coded block; determining the target parameter calculation function based on the average background brightness, and substituting the average background brightness into the target parameter calculation function to calculate the brightness distortion parameter.

[0169] In one embodiment, when the processor executes the computer program, it further performs the following steps: for each coded block, calculates multiple gradients of the coded block based on the pixel values ​​corresponding to the coded block and each gradient operator, wherein the image pixel directions corresponding to each gradient operator are different; and determines the texture distortion parameters based on the brightness distortion parameters and each gradient.

[0170] In one embodiment, when the processor executes the computer program, it further performs the following steps: selecting the image parameter with the largest value from the brightness distortion parameter and the texture distortion parameter; and determining the ratio of the reference distortion degree to the image parameter with the largest value as the second image parameter.

[0171] In one embodiment, when the processor executes the computer program, it further performs the following steps: summing the second image parameters corresponding to each coding block to obtain the image parameter sum value; for each coding block, calculating the ratio of the second image parameter to the image parameter sum value, and multiplying the ratio by the original bit rate to obtain the target bit rate.

[0172] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the quantization parameters of each target coding block of the current image frame according to the target bitrate, wherein the current image frame and the historical image frames are sequentially continuous, and the positions of each target coding block and each coding block correspond one-to-one; and performing image encoding on the current image frame according to each quantization parameter to obtain an image bitstream.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, image information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0174] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0176] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining the bitrate of an image frame, characterized in that, The method includes: Obtain first image parameters of multiple coded blocks in historical image frames, wherein the first image parameters are used to characterize the permissible degree of brightness distortion and / or texture distortion. For each of the coded blocks, a second image parameter is determined based on the first image parameter, the second image parameter being used to characterize the permissible degree of image distortion; Based on each of the second image parameters and the original bitrate of the historical image frame, the target bitrate of each coding block is determined, and each target bitrate is used to encode the current image frame.

2. The method according to claim 1, characterized in that, The step of obtaining the first image parameters of multiple coded blocks in historical image frames includes: The luminance distortion parameters corresponding to each coding block are determined based on the pixel values ​​of each coding block; The texture distortion parameters corresponding to each coding block are determined based on each pixel value and the brightness distortion parameter, and each brightness distortion parameter and each texture distortion parameter are determined as the first image parameters.

3. The method according to claim 2, characterized in that, The step of determining the luminance distortion parameter corresponding to each coding block based on the pixel value of each coding block includes: For each of the coded blocks, the average background brightness is determined based on the pixel value and brightness operator corresponding to the coded block; The target parameter calculation function is determined based on the average background brightness, and the average background brightness is substituted into the target parameter calculation function to calculate the brightness distortion parameter.

4. The method according to claim 2, characterized in that, The step of determining the texture distortion parameter corresponding to each coded block based on each pixel value and the brightness distortion parameter includes: For each coding block, multiple gradients of the coding block are calculated based on the pixel value corresponding to the coding block and each gradient operator, wherein each gradient operator corresponds to a different image pixel direction; The texture distortion parameters are determined based on the brightness distortion parameters and each of the gradients.

5. The method according to claim 2, characterized in that, Determining the second image parameters of the coded block based on the first image parameters includes: Among the brightness distortion parameter and the texture distortion parameter, the image parameter with the largest value is selected; The ratio of the reference distortion to the image parameter with the largest value is determined as the second image parameter.

6. The method according to any one of claims 1-5, characterized in that, Determining the target bitrate of each coding block based on each of the second image parameters and the original bitrate of the historical image frames includes: The image parameters and values ​​are obtained by summing the second image parameters corresponding to each of the coded blocks. For each of the coded blocks, the ratio of the second image parameter to the image parameter and value is calculated, and the ratio is multiplied by the original bitrate to obtain the target bitrate.

7. The method according to claim 1, characterized in that, The method further includes: The quantization parameters of each target coding block of the current image frame are determined according to the target bit rate. The current image frame and the historical image frames are sequentially continuous, and the positions of each target coding block and each coding block correspond one-to-one. The current image frame is encoded according to the quantization parameters to obtain an image bitstream.

8. A bitrate determination device for an image frame, characterized in that, The device includes: The first parameter acquisition module is used to acquire the first image parameters of multiple coded blocks in historical image frames. The first image parameters are used to characterize the allowable degree of brightness distortion and / or texture distortion. The second parameter acquisition module is used to determine, for each coding block, a second image parameter of the coding block based on the first image parameter, wherein the second image parameter is used to characterize the permissible degree of image distortion. The bitrate determination module is used to determine the target bitrate of each of the coding blocks based on each of the second image parameters and the original bitrate of the historical image frames, and each target bitrate is used to encode the current image frame.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A chip, characterized in that, The device includes a processor and a communication interface, wherein the processor is configured to cause the chip to perform the steps of the method described in any one of claims 1 to 7.

Images