An image encoding / decoding method, apparatus, encoder, decoder, and system.

Abstract

A method, apparatus, encoder, decoder, and system for image encoding and decoding are disclosed, relating to the field of multimedia technology. The method includes: decoding the bitstream of a coding unit to be decoded in an image bitstream, or dynamically setting a target bit count while encoding a coding unit to be encoded in the current frame, taking into account three factors: the image content of the coding unit, the number of lossy bits, and the fullness of the bitstream buffer. The encoder encodes the coding unit using quantization parameters determined by the target bit count; the decoder decodes the bitstream of the coding unit using quantization parameters determined by the target bit count. This improves the accuracy of bitrate control and minimizes the number of encoded bits after encoding the image while ensuring the quality of the reconstructed image.

Description

[0001] This application is a divisional application. The original application has the application number 202310301666.X and the original application date is March 13, 2023. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of multimedia technology, and in particular to an image encoding and decoding method, apparatus, encoder, decoder and system. Background Technology

[0003] Currently, the encoder performs encoding operations such as prediction, quantization, and entropy coding on image frames to obtain a bitstream. The decoder performs decoding operations such as entropy decoding, dequantization, and prediction reconstruction on the bitstream to obtain the reconstructed image of the image frame. A larger quantization parameter value results in less effective information from the image frame contained in the bitstream, leading to poorer reconstructed image quality; conversely, a smaller quantization parameter value results in higher reconstructed image quality, as the bitstream contains more redundant information from the image frame and has a larger number of bits. However, the code control module clamps the quantization parameters of the coding units based on pre-defined full-scale intervals. Since these full-scale intervals are determined based on statistical experience, reconstructing the coding units based on these quantization parameters may result in poor reconstructed image quality. Therefore, determining the quantization parameters used for image encoding and decoding is a problem that urgently needs to be solved. Summary of the Invention

[0004] This application provides an image encoding and decoding method, apparatus, encoder, decoder, and system, which solves the problem of poor image quality caused by inaccurate quantization parameters.

[0005] In a first aspect, this application provides an image decoding method. This decoding method is applied to an encoding / decoding system and is executed by a decoder included in the encoding / decoding system. The image decoding method includes: the decoder acquiring the bitstream of a coding unit to be decoded in an image bitstream; and the decoder determining the target number of bits for the coding unit based on the image content, the lossy bit count, and the fullness of the bitstream buffer, and determining quantization parameters based on the target number of bits for the coding unit, the quantization parameters being used to decode the bitstream of the coding unit. Wherein, the image content is used to indicate the complexity of different pixel regions in the coding unit, the lossy bit count is used to indicate the expected number of bits after lossy encoding of the coding unit without referencing the image content, the bitstream buffer is used to store the encoded bit count after decoding one or more coding units, and the target number of bits for the coding unit is used to indicate the expected number of bits after lossy encoding of the coding unit with reference to the image content of the coding unit.

[0006] A higher complexity of the coding unit indicates that the image contains more information, i.e., less repetitive information; conversely, a lower complexity of the coding unit indicates that the image contains less information, i.e., more repetitive information. When decoding the bitstream of a coding unit, three factors are considered: the image content of the coding unit, the number of lossy bits, and the fullness of the bitstream buffer. By dynamically setting the target number of bits and using the quantization parameters determined by the target number of bits to decode the bitstream of the coding unit, the accuracy of rate control is improved, minimizing the number of encoded bits after encoding the image while ensuring the quality of the reconstructed image.

[0007] As a feasible example, the image content of the above coding unit includes the complexity level of the coding unit.

[0008] For example, complexity levels include: luminance complexity level.

[0009] For example, complexity levels include: chromaticity complexity level.

[0010] For example, complexity levels include: luminance complexity level and chrominance complexity level.

[0011] In this embodiment, the image content of the encoding unit is represented by the complexity level of the encoding unit, so that the QP decision process of each encoding unit refers to the complexity level. This avoids the problem of reduced accuracy caused by the decoder not considering the content contained in the encoding unit when making QP decisions, which is beneficial to improving the accuracy of image decoding and the subjective quality of the reconstructed image.

[0012] For example, when encoding video using a constant bitrate control strategy, if the number of bits in the bitstream of the encoding unit is small, but the image content of the unit to be encoded is complex, the number of bits in the bitstream of the encoding unit can be reasonably increased while ensuring a constant bitrate, and the quantization parameter value can be smaller to improve the quality of the reconstructed image. Conversely, when encoding video using a constant bitrate control strategy, if the number of bits in the bitstream of the encoding unit is large, but the image content of the unit to be encoded is simple, the quantization parameter value can be larger while ensuring a constant bitrate and the quality of the reconstructed image, thus reasonably reducing the number of bits in the bitstream of the encoding unit.

[0013] In one alternative implementation, the image decoding method provided in this application further includes: the decoder decodes the bit stream of the coding unit according to the quantization parameters to obtain the reconstructed image of the coding unit.

[0014] Secondly, this application provides an image encoding method. This encoding method is applied to an encoding / decoding system, and the decoding method is executed by an encoder included in the encoding / decoding system. The image encoding method includes: the encoder acquiring the encoding unit to be encoded in the current frame. Then, the encoder determines the target number of bits for the encoding unit based on the image content, the lossy bit count, and the fullness of the bitstream buffer, and determines quantization parameters based on the target number of bits for the encoding unit. The quantization parameters are used to encode the encoding unit. Wherein, the image content indicates the complexity of different pixel regions in the encoding unit; the lossy bit count indicates the expected number of bits after lossy encoding of the encoding unit without referencing the image content; the bitstream buffer stores the encoded bit count of one or more encoding units; and the target number of bits for the encoding unit indicates the expected number of bits after lossy encoding of the encoding unit with reference to the image content of the encoding unit.

[0015] When encoding a coding unit, three factors are considered: the image content of the coding unit, the number of lossy bits, and the fullness of the bitstream buffer. For coding units with lower complexity, fewer target bits are preferred, while for coding units with higher complexity, more target bits are preferred. That is, while ensuring the quality of the reconstructed image, the number of encoded bits after encoding the image is minimized, taking into account the complexity of the image content expressed by the coding unit, the number of lossy bits, and the fullness of the bitstream buffer.

[0016] In one optional implementation, the encoder determines the target number of bits for the coding unit based on the image content, the number of lossy bits, and the full capacity of the bitstream buffer. This includes: determining the minimum number of coded bits for the coding unit based on the number of lossy bits and the full capacity of the buffer, and determining the lossless number of coded bits for the coding unit based on the image content, where the lossless number of coded bits indicates the expected number of bits after lossless encoding of the coding unit. Furthermore, the encoder determines an offset value for the coding unit based on the image content, the full capacity of the buffer, and the lossless number of coded bits; this offset value indicates the difference between a maximum number of coded bits and the number of lossy bits after lossy encoding of the coding unit. Thus, after the encoder determines the maximum target number of bits for the coding unit based on the aforementioned offset value, the number of lossy bits, and the minimum number of coded bits, the encoder clamps the target number of the coding unit based on the maximum target number and the minimum number of coded bits to obtain the clamped target number of bits. This clamped target number of bits is used to determine the quantization parameters of the coding unit.

[0017] In one alternative example, the encoder determines the quantization parameters based on the target number of bits of the coding unit, including: the encoder determines the quantization parameters of the coding unit based on the lossless number of coded bits of the coding unit and the target number of bits after clamping.

[0018] Lossless coded bit data is used to indicate the expected number of bits after lossless encoding of a coding unit; that is, lossless coded bit data represents the expected number of bits under an encoding method that can fully express the information of the coding unit. The minimum coded bit number indicates the minimum number of coded bits for a coding unit, while the maximum target bit number limits the maximum number of coded bits for a coding unit. By measuring the lossless coded bit data of a coding unit using the minimum coded bit number and the maximum target bit number, the target bit number of the coding unit is determined. That is, while fully preserving the information of the coding unit based on the minimum coded bit number, the number of coded bits after encoding the image is reduced, thereby improving the accuracy of the target bit number determined for the coding unit.

[0019] In one possible implementation, the encoder determines the offset value of the coding unit based on the image content, buffer fullness, and lossless bit count of the coding unit. This includes: the encoder determining the complexity level of the coding unit based on the image content, and determining the relative lossless bit count of the coding unit based on the maximum relative lossless bit count, the maximum lossless bit count of the coding unit, and the lossless bit count; the maximum relative lossless bit count indicates the maximum value of the relative lossless bit count of the coding unit at the pixel bit depth of the current frame. Additionally, the encoder acquires one or more offset values.

[0020] In the first possible example, the encoder obtains the first offset value based on the relative lossless bit count of the coding unit, the complexity level, the lossless bit count, and the buffer fullness.

[0021] In the second possible example, the encoder obtains the second offset value based on the relative lossless number of bits and the lossy number of bits of the coding unit.

[0022] In the third possible example, the encoder obtains the third offset value based on the fullness of the encoding unit's buffer.

[0023] The offset value used to determine the maximum target bit count of the coding unit is one of a first offset value, a second offset value, and a third offset value. For example, the offset value for the maximum target bit count of the coding unit is determined as follows: the encoder selects the minimum offset value from the first and second offset values, and then uses the minimum of the first and third offset values ​​as the offset value for the maximum target bit count of the coding unit. Selecting the minimum value among multiple offset values ​​comparing the maximum target bit count to the lossy bit count, and clamping the target bit count of the coding unit based on the maximum target bit count corresponding to this minimum value, helps improve the accuracy of the target bit count of the coding unit, thereby improving the accuracy of the quantization parameters and the subjective quality of the reconstructed image after the decoder decodes the bitstream of the coding unit based on these quantization parameters.

[0024] To achieve better coding performance and reconstruction quality, the code control module needs to allocate different target bit counts to different coding units within the image. This maximizes the utilization of the specified total number of coding bits, resulting in the best possible image quality after decoding. The decoder can determine the offset value of the maximum target bit count for a coding unit using different offset acquisition methods, thereby fully utilizing the number of coding bits and improving the image quality of the reconstructed image.

[0025] When the encoder obtains the offset value used to determine the maximum target number of bits of the coding unit, it can be obtained through any one or a combination of the above three possible examples, and this application does not limit it.

[0026] The following example illustrates how the encoder obtains the aforementioned first offset value. The encoder obtains the first offset value based on the relative lossless bit count, complexity level, lossless bit count, and buffer fullness of the encoding unit. This includes: the encoder determining whether the relative lossless bit count of the encoding unit is greater than or equal to a set first threshold, and whether the complexity level of the encoding unit is less than or equal to a set second threshold. If so, the encoder processes the lossless bit count and buffer fullness according to a first rule to obtain the first offset value. If not, the encoder processes the lossless bit count and buffer fullness according to a second rule to obtain the first offset value. When the input data for both rules is the same (i.e., the lossless bit count and buffer fullness), the first offset value determined by the encoder according to the first rule is greater than or equal to the first offset value determined according to the second rule.

[0027] When the encoder determines the first offset value using the first rule, the encoding unit is an encoding unit that needs to be protected for subjective quality. The encoder increases the first offset value to increase the maximum number of target bits that the encoding unit is allowed to use, thereby reducing the quantization parameters of the encoding unit, which is beneficial to improving the subjective quality of the reconstructed image after decoding the bit stream of the encoding unit.

[0028] When the encoder determines the first offset value using the second rule, the encoding unit is an encoding unit that does not require subjective quality protection. The encoder reduces the first offset value to reduce the maximum number of target bits allowed for the encoding unit, thereby increasing the quantization parameters of the encoding unit. This is beneficial for reducing the number of encoded bits in the bit stream obtained after encoding the encoding unit, reducing the storage space required for bit stream storage, and improving the communication efficiency for transmitting the bit stream.

[0029] In another alternative implementation, the image encoding method provided in this application further includes: an encoder encoding the encoding unit according to quantization parameters to obtain the bit stream of the encoding unit.

[0030] Thirdly, this application provides an image encoding / decoding apparatus. The apparatus includes modules for performing the method of the first aspect or any possible design of the first aspect, and modules for performing the method of the second aspect or any possible design of the second aspect.

[0031] Fourthly, this application provides an encoder. The encoder includes at least one processor and a memory, wherein the memory stores a computer program such that when the computer program is executed by the at least one processor, it implements the method described in the second aspect or any possible design of the second aspect.

[0032] Fifthly, this application provides a decoder. The decoder includes at least one processor and a memory, wherein the memory stores a computer program such that when the computer program is executed by at least one processor, it implements the method described in the first aspect or any possible design of the first aspect.

[0033] Sixthly, this application provides an encoding / decoding system. The encoding / decoding system includes the encoder described in the fourth aspect and the decoder described in the fifth aspect.

[0034] In a seventh aspect, this application provides a chip. The chip includes a processor and a power supply circuit.

[0035] The power supply circuit is used to supply power to the processor; the processor is used to execute the operation steps of the method in the first aspect or any possible implementation of the first aspect, and to execute the operation steps of the method in the second aspect or any possible implementation of the second aspect.

[0036] Eighthly, this application provides a computer-readable storage medium. The computer-readable storage medium includes: computer software instructions.

[0037] When computer software instructions are executed in a computing device, they cause the computing device to perform operational steps of the method in the first aspect or any possible implementation thereof, as well as operational steps of the method as described in the second aspect or any possible implementation thereof. This applies if the computing device is the aforementioned encoder or decoder.

[0038] Ninthly, this application provides a computer program product. When the computer program product is run on a computer, it causes a computing device to perform the operational steps of the method in the first aspect or any possible implementation of the first aspect, as well as the operational steps of the method as described in the second aspect or any possible implementation of the second aspect. For example, the computer may be the aforementioned encoder or decoder.

[0039] The beneficial effects of aspects three through nine above can be described with reference to the implementation of any of aspects one or two, and will not be repeated here. Based on the implementations provided in the above aspects, this application can be further combined to provide more implementations. Attached Figure Description

[0040] Figure 1 A schematic diagram of a video transmission system provided in this application;

[0041] Figure 2 A schematic diagram of a video encoding and decoding system provided in this application;

[0042] Figure 3 A schematic diagram of the structure of an encoder and decoder provided in this application;

[0043] Figure 4 A flowchart illustrating an image encoding / decoding method provided in this application;

[0044] Figure 5 A flowchart illustrating an image encoding method provided in this application;

[0045] Figure 6 A schematic diagram illustrating the determination of a quantization parameter provided in this application;

[0046] Figure 7 A flowchart illustrating an image decoding method provided in this application;

[0047] Figure 8 A schematic diagram of the structure of an encoding / decoding device provided in this application;

[0048] Figure 9 This is a schematic diagram of the structure of an image processing system provided in this application. Detailed Implementation

[0049] The technical solutions involved in this application may be applied not only to video coding standards (such as H.264, HEVC, etc.), but also to future video coding standards (such as H.266). The terminology used in the implementation section of this application is only for explaining specific embodiments of this application and is not intended to limit this application. A brief introduction to some concepts that may be involved in this application is given below.

[0050] Video consists of multiple consecutive images. When these multiple consecutive images change at more than 24 frames per second, according to the principle of visual persistence, the human eye cannot distinguish a single static image. Thus, multiple images that appear smooth and continuous constitute video.

[0051] Video coding refers to the processing of a sequence of images that forms a video or video sequence. In the field of video coding, the terms "picture," "frame," or "image" can be used synonymously. As used herein, video coding refers to either video encoding or video decoding. Video coding is performed on the source side and typically involves processing (e.g., compressing) the original video images to reduce the amount of data required to represent them, thereby enabling more efficient storage and / or transmission, while maintaining a certain image quality. Video decoding is performed on the destination side and typically involves the inverse processing relative to the encoder to reconstruct the video images. The "encoding" of video images in the embodiments should be understood as involving the "encoding" or "decoding" of a video sequence. The combination of encoding and decoding is also called encoding and decoding (encoding and decoding). Video coding can also be called image coding or image compression. Image decoding refers to the inverse process of image coding.

[0052] A video sequence consists of a series of pictures, which are further divided into slices, and slices into blocks. Video coding is performed on a block-by-block basis. In some newer video coding standards, the concept of a block has been further expanded. For example, the H.264 standard uses macroblocks (MBs), which can be further divided into multiple prediction blocks (partitions) for predictive coding. The High Efficiency Video Coding (HEVC) standard uses basic concepts such as coding units (CUs), prediction units (PUs), and transform units (TUs) to functionally divide various block units, and employs a novel tree-based structure for description. For instance, a CU can be divided into smaller CUs using a quadtree, and these smaller CUs can be further divided, forming a quadtree structure. The CU is the basic unit for partitioning and encoding the image. Similar tree structures exist for PUs and TUs. A PU corresponds to a prediction block and is the basic unit for predictive coding. CUs are further divided into multiple PUs according to partitioning patterns. TU can correspond to transform block, which is the basic unit for transforming prediction residuals. However, whether it is CU, PU or TU, they all essentially belong to the concept of block (or coding unit).

[0053] For example, in HEVC, the CTU is split into multiple CUs using a quadtree structure represented as a coding tree. At the CU level, a decision is made on whether to use inter-picture (temporal) or intra-picture (spatial) prediction to encode picture regions. Each CU can be further split into one, two, or four PUs based on the PU splitting type. The same prediction process is applied within a PU, and relevant information is transmitted to the decoder based on the PU. After obtaining residual blocks by applying the prediction process based on the PU splitting type, the CU can be segmented into transform units (TUs) according to other quadtree structures similar to the coding tree used for CUs. In the latest developments in video compression technology, quadtree and binary tree (QTBT) frame segmentation is used to divide coding blocks. In the QTBT block structure, CUs can be square or rectangular in shape.

[0054] In this paper, for ease of description and understanding, the coding unit to be encoded in the current encoded image is referred to as the current block. For example, in encoding, it refers to the block currently being encoded; in decoding, it refers to the block currently being decoded. The decoded coding unit in the reference image used to predict the current block is called the reference block. That is, the reference block is the block that provides a reference signal for the current block, where the reference signal represents the pixel value within the coding unit. The block in the reference image that provides the prediction signal for the current block can be called the prediction block, where the prediction signal represents the pixel value, sampled value, or sampled signal within the prediction block. For example, after traversing multiple reference blocks, an optimal reference block is found. This optimal reference block will provide prediction for the current block; this block is called the prediction block.

[0055] Lossless video coding refers to the ability to reconstruct the original video images, meaning that the reconstructed video images have the same quality as the original video images (assuming no transmission loss or other data loss during storage or transmission).

[0056] Lossy video coding refers to reducing the number of bits required to represent video images by performing further compression, such as quantization, while the decoder cannot fully reconstruct the video images. In other words, the quality of the reconstructed video images is lower or worse than that of the original video images.

[0057] Bitstream (data rate) refers to the binary stream generated after encoding an image or video. Also known as bitrate or bit depth, it represents the number of bits transmitted per unit of time. It's a crucial part of image quality control in image encoding. For images of the same resolution, a higher bitstream results in a lower compression ratio and better image quality.

[0058] Rate control refers to the adjustment of the bit rate during encoding and decoding. Hereinafter abbreviated as rate control. Rate control modes include Constant Bit Rate (CBR) and Variable Bit Rate (VBR).

[0059] Constant Bit Rate (CBR) means ensuring a stable bit rate over a given period of time.

[0060] Dynamic bit rate (VBR) allows bit rate fluctuations within a bit rate statistical period, thereby ensuring stable image quality after encoding.

[0061] Quantization refers to the process of mapping continuous values ​​of a signal into multiple discrete amplitude values.

[0062] The quantization parameter (QP) is used during the encoding process to quantize the residual values ​​from the prediction operation or the coefficients from the transform operation. During decoding, the syntax elements are dequantized to obtain the residual values ​​or coefficients. The quantization parameter is a parameter used in the quantization process; generally, a larger quantization parameter value results in a more pronounced quantization, but also lower image quality and bitrate after reconstruction; conversely, a smaller quantization parameter value results in better image quality and higher bitrate after reconstruction.

[0063] Bitstream buffer fullness refers to the ratio of the number of bits of data in the bitstream buffer to the storage capacity of the bitstream buffer. At the encoding end, the number of bits of data in the bitstream buffer includes the number of encoded bits in the encoding unit. At the decoding end, the number of bits of data in the bitstream buffer includes the number of decoded bits in the encoding unit.

[0064] Clamping refers to the operation of restricting a certain value to a specified range.

[0065] The embodiments of this application will now be described with reference to the accompanying drawings.

[0066] Figure 1 This application provides a schematic diagram of a video transmission system. The video processing includes video acquisition, video encoding, video transmission, video decoding, and display. The video transmission system includes multiple terminal devices (such as…). Figure 1 The terminal devices 111 to 115 shown are a network, wherein the network can realize the function of video transmission, and the network can include one or more network devices, such as routers or switches.

[0067] Figure 1 The terminal devices shown can be, but are not limited to, user equipment (UE), mobile station (MS), and mobile terminal (MT). The terminal device can be a mobile phone (e.g., Figure 1 The terminal device 114 shown), tablet computer, computer with wireless transceiver function (such as...) Figure 1 The terminal device 115 shown), virtual reality (VR) terminal device (such as...) Figure 1 The terminal devices shown include: terminal device 113, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in smart cities, wireless terminals in smart homes, and so on.

[0068] like Figure 1 As shown, the terminal devices differ in different video processing stages.

[0069] For example, during video acquisition, the terminal device 111 can be a camera device (such as a camcorder or camera) used for road monitoring, or a mobile phone, tablet computer, or smart wearable device with video acquisition function.

[0070] For example, in the video encoding process, the terminal device 112 can be a server or a data center. The data center may include one or more physical devices with encoding functions, such as servers, mobile phones, tablets or other encoding devices.

[0071] For example, during video decoding and display, terminal device 113 can be VR glasses, and users can control the field of view by turning their heads. Terminal device 114 can be a mobile phone, and users can control the field of view on the mobile phone 114 by touch operation or air operation; terminal device 115 can be a personal computer, and users can control the field of view displayed on the screen by input devices such as a mouse or keyboard.

[0072] Understandably, "video" is a general term. A video consists of a sequence of multiple consecutive frames, with each frame corresponding to one image. For example, panoramic video can refer to a 360° video or a 180° video. In some possible cases, panoramic video can also refer to a "large" range of video that exceeds the human eye's field of view (such as 110° to 120°), such as a 270° video.

[0073] Figure 1 This is just an illustration; the video transmission system may also include other devices. Figure 1 Not shown in the diagram. The embodiments of this application do not limit the number or type of terminal devices included in the system.

[0074] exist Figure 1 Based on the video transmission system shown, Figure 2 This is a schematic diagram of a video encoding and decoding system provided in this application. The video encoding and decoding system 200 includes an encoding device 210 and a decoding device 220. The encoding device 210 establishes a communication connection with the decoding device 220 through a communication channel 230.

[0075] The aforementioned encoding device 210 can perform video encoding functions, such as... Figure 1 As shown, the encoding device 210 can be the terminal device 112, or it can be a data center with video encoding capabilities, for example, the data center includes multiple servers.

[0076] The encoding device 210 may include a data source 211, a preprocessing module 212, an encoder 213, and a communication interface 214.

[0077] Data source 211 may include or can be any type of electronic device for capturing video, and / or any type of source video generation device, such as a computer graphics processor for generating computer animation scenes or any type of device for acquiring and / or providing source video, or computer-generated source video. Data source 211 may be any type of memory or storage device for storing the aforementioned source video. The aforementioned source video may include multiple video streams (bit streams) or images captured by multiple video capture devices (such as cameras).

[0078] An image can be viewed as a two-dimensional array or matrix of pixels (picture elements). Pixels in an array are also called sample points. The number of sample points in the array or image along the horizontal and vertical directions (or axes) defines the image's size and / or resolution. To represent color, three color components are typically used; that is, an image can be represented as or contain three sample arrays. For example, in RBG format or color space, an image includes corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in a luma / chroma format or color space. For example, for a YUV format image, this includes a luma component indicated by Y (sometimes also indicated by L) and two chroma components indicated by U and V. The luma component Y represents the brightness or grayscale level intensity (e.g., both are the same in a grayscale image), while the two chroma components U and V represent chroma or color information components. Accordingly, a YUV format image includes a luma sample array of luma sample values ​​(Y) and two chroma sample arrays of chroma values ​​(U and V). An RGB format image can be converted or transformed to YUV format, and vice versa; this process is also called color transformation or conversion. If the image is black and white, it may only include a luminance sampling array. In this application, the image transmitted from data source 211 to encoder 213 can also be referred to as raw image data or source image.

[0079] The preprocessing module 212 receives the source video and preprocesses it to obtain a preprocessed image, such as a panoramic video or a multi-frame image. For example, the preprocessing performed by the preprocessing module 212 may include color format conversion (e.g., from RGB to YCbCr), octree structuring, video stitching, etc.

[0080] Encoder 213 is used to receive the preprocessed image and encode it to obtain encoded data (such as a bitstream). For example, encoder 213 may include a code control unit 2131 and an encoding unit 2132. Code control unit 2131 is used to determine the quantization parameters used to encode each encoding unit in the current frame, so that encoding unit 2132 can predict, quantize, and encode the preprocessed image according to the quantization parameters to obtain a bitstream. For example, encoder 213 determines the target number of bits for the encoding unit based on the image content, lossy bit count, and full buffer capacity of the bitstream buffer, then determines the quantization parameters for the encoding unit based on the target number of bits, and encodes the encoding unit according to the quantization parameters.

[0081] The communication interface 214 in the encoding device 210 can be used to: receive bit streams and send bit streams (or a version of the bit stream after any other processing) to another device such as the decoding device 220 or any other device through the communication channel 230, so as to store, display or directly reconstruct the original image, etc.

[0082] Optionally, the encoding device 210 includes a bitstream buffer for storing bitstreams corresponding to one or more encoding units.

[0083] The aforementioned decoding device 220 can perform image decoding or video decoding functions, such as... Figure 1 As shown, the decoding device 220 can be Figure 1 The terminal device shown is any one of terminal devices 113 to 115.

[0084] The decoding device 220 may include a display device 221, a post-processing module 222, a decoder 223, and a communication interface 224.

[0085] The communication interface 224 in the decoding device 220 is used to receive a bit stream (or a version of the bit stream after any other processing) from the encoding device 210 or from any other encoding device such as a storage device.

[0086] Communication interfaces 214 and 224 can be used to send or receive bit streams via a direct communication link between encoding device 210 and decoding device 220, such as a direct wired or wireless connection, or via any type of network, such as a wired network, a wireless network or any combination thereof, any type of private network and public network or any combination thereof.

[0087] Communication interface 224 corresponds to communication interface 214. For example, it can be used to receive transmitted data and process the transmitted data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain a bit stream.

[0088] Both communication interface 224 and communication interface 214 can be configured as follows: Figure 2 The arrow pointing from the encoding device 210 to the corresponding communication channel 230 of the decoding device 220 indicates a one-way communication interface or a two-way communication interface, and can be used to send and receive messages, etc., to establish a connection, confirm and exchange any other information related to the communication link, or data transmission such as encoded compressed data (e.g., bit stream) transmission, etc.

[0089] Decoder 223 receives encoded data and decodes it to obtain decoded data (images or videos, etc.). For example, decoder 223 performs entropy decoding, dequantization, and prediction reconstruction on the bitstream to obtain a reconstructed image. Decoder 223 may include a code control unit 2231 and a decoding unit 2232. Code control unit 2231 determines the quantization parameters used for decoding each encoded unit in the current frame, so that decoding unit 2232 can decode, dequantize, and predictively reconstruct the bitstream according to the quantization parameters to obtain the reconstructed image. Decoder 223 can determine the target number of bits for the encoded unit based on the image content, lossy bit count, and the full size of the bitstream buffer, and decode the bitstream of the encoded unit based on the quantization parameters determined by the target bit count.

[0090] The post-processing module 222 is used to post-process the decoded data to obtain post-processed data (such as the reconstructed image to be displayed). The post-processing performed by the post-processing module 222 may include, for example, color format conversion (e.g., from YCbCr to RGB), octree reconstruction, video splitting and merging, or any other processing to generate data for display device 221, etc.

[0091] Display device 221 is used to receive post-processed data for display to users or viewers. Display device 221 can be or includes any type of display for representing the reconstructed image, such as an integrated or external display screen or monitor. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen.

[0092] As an optional implementation, the encoding device 210 and the decoding device 220 can transmit the encoded data through a data forwarding device. For example, the data forwarding device can be a router or a switch.

[0093] The structure of the above encoding / decoding system is only illustrative. In some possible implementations, the encoding / decoding system may also include other devices, such as end-side devices or cloud-side devices. Acquisition devices (such as...) Figure 1 After acquiring the original image, the terminal 111 in the process preprocesses the original image to obtain a preprocessed image; and then transmits the preprocessed image to the end-side device or cloud-side device (such as...). Figure 1 The terminal 112 in the middle is used by the end-side device or the cloud-side device to encode and decode the preprocessed image.

[0094] The image encoding and decoding method provided in this application is applied to both the encoding and decoding ends. Figure 3 The structure of the encoder and decoder is described in detail. For example... Figure 3 As shown, the encoder 300 includes a prediction unit 310, a code control unit 320, a quantization unit 330, an encoding unit 340, and a block partitioning unit 350.

[0095] Block partitioning unit 350 is used to divide the original image into multiple coding units.

[0096] The code control unit 320 is used to determine the target number of bits for the encoding unit based on the image content, lossy bit count, and buffer fullness of the bit stream buffer of the encoding unit that needs to be encoded, as output by the block partitioning unit 350, and to determine the quantization parameters based on the target number of bits.

[0097] In some possible embodiments, when the encoder 300 transmits the bit stream of the image frame to the decoder 400 after encoding an image frame, the data in the bit stream buffer includes the bit stream of the image frame, and the number of bits in the data in the bit stream buffer includes the number of bits in the bit stream of the image frame.

[0098] In other possible embodiments, when encoder 300 transmits the bitstream of the encoded units to decoder 400 after encoding the encoded units in a frame image, the data in the bitstream buffer includes the bitstream of one or more encoded units, and the number of bits in the data in the bitstream buffer includes the number of encoded bits of the bitstream of one or more encoded units. Understandably, the bitstream of one or more encoded units can be the bitstream of encoded units already encoded by encoder 300 minus the bitstream of encoded units transmitted by encoder 300 to decoder 400.

[0099] Prediction unit 310 performs intra-frame prediction on the coding units output by block partitioning unit 350 to obtain the predicted bit count, and outputs the residual between the original bit count and the predicted bit count of the coding unit. For example, an explanation of intra-frame prediction can be found in HEVC's intra-frame prediction. Intra-frame prediction is a common method for removing spatial redundancy information in the original image. It uses reconstructed pixels from adjacent coding blocks as reference values ​​to predict the current coding unit. This is equivalent to the coding unit in the original image having a correlation with its surrounding coding blocks. The pixel value of the current coding unit can be estimated using the reconstructed coding units around it. The estimated pixel value is the predicted value. The residual between the predicted value and the original value of the current coding unit is quantized and entropy encoded. The encoded data is often transmitted as the prediction residual. The decoding end performs the same prediction process to obtain the predicted value of the current coding unit, and then adds the predicted value to the solved residual to obtain the reconstructed value of the current coding unit.

[0100] The quantization unit 330 is used to quantize the residual output by the prediction unit 310 according to the quantization parameters output by the code control unit 320 to obtain the quantized residual.

[0101] The encoding unit 340 is used to encode the quantized residual output by the quantization unit 330 to obtain the bit stream of the encoding unit. For example, entropy encoding is performed on the quantized residual output by the quantization unit 330.

[0102] Combination Figure 3 The structure of the decoder is described in detail. For example... Figure 3 As shown, the decoder 400 includes a decoding unit 410, a code control unit 420, an inverse quantization unit 430, and a prediction and reconstruction unit 440.

[0103] The decoding unit 410 is used to decode the bit stream of the encoding unit to obtain the quantized residual and image content.

[0104] The code control unit 420 determines the target number of bits for the encoding unit based on the image content, lossy bit count, and fullness of the bit stream buffer of the encoding unit that needs to be decoded, as output by the decoding unit 410, and determines the quantization parameters based on the target number of bits.

[0105] In some embodiments, when the encoder 300 transmits the bit stream of the image frame to the decoder 400 after encoding an image frame, the data in the bit stream buffer of the decoder 400 includes the decoded data of the image frame, and the number of bits in the data in the bit stream buffer includes the number of bits of the decoded data of the image frame.

[0106] In other embodiments, when the encoder 300 transmits the bit stream of the encoding unit to the decoder 400 after encoding the encoding unit in a frame image, the data in the bit stream buffer of the decoder 400 includes the decoded data of one or more encoding units, and the number of bits in the data in the bit stream buffer includes the number of bits of the decoded data of one or more encoding units.

[0107] The dequantization unit 430 is used to dequantize the quantized residual output by the decoding unit 410 according to the quantization parameters output by the code control unit 420 to obtain the residual.

[0108] The prediction and reconstruction unit 440 is used to predict and reconstruct the image based on the residual output by the inverse quantization unit 430 so that the reconstructed image can be displayed on the display.

[0109] To address the challenge of determining the quantization parameters used for image encoding and decoding to ensure the quality of the reconstructed image, this application provides an image encoding and decoding method. This method considers three factors: the image content of the encoding unit, the number of lossy bits, and the fullness of the bitstream buffer. A target number of bits is dynamically set. At the encoding end, the encoding unit is encoded using the quantization parameters determined by the target number of bits. This reduces the number of encoded bits while ensuring the quality of the reconstructed image.

[0110] Next, the image encoding and decoding process will be explained with reference to the accompanying drawings. Figure 4 This is a flowchart illustrating an image encoding / decoding method provided in this application. Figure 2 The image encoding and decoding process performed by the encoding device 210 and decoding device 220 will be used as an example for illustration. Figure 4 As shown, the method includes the following steps.

[0111] S410, encoding device 210 obtains the encoding unit to be encoded in the current frame.

[0112] As described in the above embodiments, if the encoding device 210 carries a data source 211, the encoding device 210 can acquire the original image through the data source 211. Optionally, the encoding device 210 can also receive original images acquired by other devices; or obtain the original image from the memory or other memory in the encoding device 210. The original image may include at least one of real-world images acquired in real time, images stored in the device, and images synthesized from multiple images. This embodiment does not limit the method of acquiring the original image or the type of the original image.

[0113] The current frame refers to the frame or original image that is being encoded and decoded at the current moment. The preceding frame refers to the frame or original image that was encoded and decoded at a time prior to the current moment. The preceding frame can be a frame from the moment before the current moment or several moments before.

[0114] The encoding device 210 can divide the current frame into multiple encoding units and encode the multiple encoding units.

[0115] S420 and encoding device 210 determine the target number of bits for the encoding unit based on the image content of the encoding unit, the number of lossy bits, and the full capacity of the bit stream buffer, and determine the quantization parameters of the encoding unit based on the target number of bits.

[0116] The lossy bit count is used to indicate the expected number of bits after the coding unit performs lossy coding without referencing the image content.

[0117] A bitstream buffer is used to store the encoded bits of one or more coding units. Buffer fullness refers to the ratio of the amount of data stored in the bitstream buffer to its storage capacity. Before encoding, a physical buffer for storing the bitstream can be pre-configured in memory. The buffer fullness of the bitstream buffer can be obtained based on the number of bits of the encoded coding unit stored in the physical buffer.

[0118] The target number of bits for a coding unit is used to indicate the desired number of bits after lossy coding of the coding unit when referencing the image content of the coding unit.

[0119] The image content of a coding unit indicates the complexity of different pixel regions within that unit. This includes, for example, the complexity of the pixel region's color, texture, and shape. For instance, this complexity can be used to indicate the relative size of the expected number of bits determined after encoding the coding unit.

[0120] In some alternative implementations, the image content includes the complexity level of the coding unit.

[0121] In the first example, the complexity levels include: luminance complexity level.

[0122] In the second example, the complexity levels include: chromaticity complexity level.

[0123] In the third example, the complexity levels include: luminance complexity level and chrominance complexity level.

[0124] In this embodiment, the image content of the encoding unit is represented by the complexity level of the encoding unit, so that the QP decision process of each encoding unit refers to the complexity level. This avoids the problem of reduced accuracy caused by the encoding device not considering the content contained in the encoding unit when making QP decisions, which is beneficial to improving the accuracy and encoding quality of image encoding.

[0125] As an optional implementation, the process of determining the complexity level of the coding unit includes the following steps ① to ④.

[0126] Step ①: The encoding device 210 divides the encoding unit into multiple sub-blocks.

[0127] In one possible example, the coding unit can first be divided into coding blocks for different channels, and then further divided for a specific channel to obtain multiple sub-blocks under that specific channel. These different channels include the Y channel, U channel, and V channel.

[0128] In another possible example, instead of dividing the coding unit into coding blocks for different channels, the coding unit can be directly divided to obtain multiple sub-blocks.

[0129] The two possible examples above are merely possible implementations provided in this embodiment and should not be construed as limiting this application.

[0130] Step 2: For each of the multiple sub-blocks, the encoding device 210 obtains the texture complexity level of each sub-block.

[0131] In step ②, the texture complexity level is one of several defined complexity levels. These defined complexity levels may include a first level, a second level, a third level, or a fourth level, etc., and this application does not limit this. It is worth noting that the range of complexity values ​​corresponding to different complexity levels is different.

[0132] Step 3: The encoding device 210 obtains the texture complexity level of the encoding unit based on the multiple texture complexity levels of multiple sub-blocks.

[0133] Regarding step ③ above, this example provides a possible implementation: the encoding device 210 processes multiple texture complexity levels of multiple sub-blocks according to a set rule to determine the complexity level of the encoding unit. The set rule can be, for example, addition, deduplication, or weighting.

[0134] For example, the encoding device 210 adds up multiple texture complexity levels of multiple sub-blocks, thereby determining the complexity level of the encoding unit based on the sum of the added texture complexity levels.

[0135] For example, the encoding device 210 removes duplicate texture complexity levels from multiple texture complexity levels of multiple sub-blocks and selects the texture complexity level with the highest texture complexity level as the complexity level of the encoding unit.

[0136] For example, the encoding device 210 assigns different weights to different sub-blocks, thereby weighting and summing the weights and texture complexity levels of each sub-block to determine the complexity level of the encoding unit.

[0137] In this embodiment, the encoding device processes multiple texture complexity levels of multiple sub-blocks according to set rules, such as addition, deduplication, or weighting, to determine the complexity level of the encoding unit. This avoids the problem of inaccurate determination of the complexity level of the encoding unit using only the texture complexity level of a single sub-block or a small number of sub-blocks. It helps to improve the accuracy of the input information for the QP decision of the encoding unit, making the QP value of the encoding unit more consistent with the image content of the encoding unit, thereby improving the image encoding effect.

[0138] Step 4: The encoding device 210 determines the complexity level of the encoding unit based on the texture complexity level of the encoding unit.

[0139] Taking texture complexity level as an example, the calculation process includes: the encoding device divides the encoding unit into several sub-blocks. For each sub-block, the difference between adjacent pixel values ​​is calculated step by step in the horizontal, vertical, and diagonal directions (such as 45° or -135°). Then, the encoding device sums the absolute values ​​of these differences to obtain the texture complexity value corresponding to each sub-block. This texture complexity value is compared with a pre-defined threshold to obtain the complexity type (such as texture complexity level) of the sub-block. Finally, the encoding device performs rule-based operations on the texture complexity types of each sub-block to obtain the texture complexity level of the current encoding unit.

[0140] In this embodiment, texture complexity is used to indicate the grayscale changes in the image. After the encoding device obtains the texture complexity level of each sub-block, it determines the complexity level of the encoding unit based on the texture complexity. This makes the QP decision process of the encoding unit refer to the grayscale changes in the encoding unit, which is beneficial to improving the accuracy of image encoding and the encoding quality of image encoding.

[0141] Optionally, the encoding device may also determine the complexity level of the encoding unit based on other complexities. For example, these other complexities could be the information entropy of the bit plane in the image, the intra-block copy (IBC) complexity level, and so on. In some optional examples, the encoding device may determine the complexity level of the encoding unit based on the above-mentioned texture complexity and other complexities, which is not limited in this application.

[0142] Taking the IBC complexity level of the coding unit as an example, the calculation process of the complexity level includes: the coding unit obtains the IBC similarity prediction sample matrix from the similar IBC prediction pattern; the coding device divides the coding unit into several sub-blocks; each sub-block calculates the sum of the absolute values ​​of the differences between its IBC similarity prediction samples and the original values, which is called the sub-block SAD; the sub-block SAD is compared with the agreed threshold to obtain the IBC complexity level of the sub-block; after the IBC complexity levels of each sub-block are subjected to rule operations, the IBC complexity level of the current coding unit is obtained.

[0143] The two methods for obtaining the complexity levels described above are merely examples provided in this embodiment and should not be construed as limiting this application. After the encoding device obtains the texture complexity level and IBC complexity level of the encoding unit, it can also use the minimum value between the texture complexity level and the IBC complexity level as the image complexity level of the current encoding unit. The image complexity level is one of a preset K values.

[0144] For a detailed explanation of how the encoding device 210 determines the quantization parameters, please refer to the descriptions in steps S510 to S550 below, which will not be repeated here.

[0145] S430 and encoding device 210 encode the encoding unit according to the quantization parameters to obtain the bit stream of the encoding unit.

[0146] Encoding device 210 can perform encoding operations such as transformation, quantization, and entropy encoding on encoding units to generate a bit stream, thereby achieving the purpose of data compression of the encoding units to be encoded. The number of bits in the bit stream of the encoding unit can be less than or greater than the target number of bits. The specific method for generating the bit stream can refer to common techniques and the description of encoding unit 330 in the above embodiments.

[0147] S440, encoding device 210 sends a bit stream to decoding device 220.

[0148] In a first possible example, the encoding device 210 can send the video bitstream to the decoding device 220 after the entire video has been encoded.

[0149] In a second possible example, the encoding device 210 can also encode the original image in real time, frame by frame, and send a bit stream of a frame after encoding a frame.

[0150] In a third possible example, the encoding device 210 encodes the encoding units of the original image and sends the bit stream of the encoding units after the encoding is completed.

[0151] The above three examples are merely possible implementations of the transmitted bit stream provided in this embodiment and should not be construed as limiting this application. Specific methods for the encoding device to transmit the bit stream can be found in conventional techniques and the descriptions of communication interfaces 214 and 224 in the above embodiments.

[0152] S450, Decoding device 220 acquires the bit stream of the encoding unit to be decoded in the image bit stream.

[0153] S460 and decoding device 220 determine the target number of bits based on the image content of the encoding unit and the number of bits of data in the bit stream buffer, and determine the quantization parameters based on the target number of bits of the encoding unit.

[0154] After receiving the bitstream of the encoding unit, the decoding device 220 determines the target number of bits for the encoding unit based on the image content of the encoding unit, the number of lossy bits, and the full capacity of the bitstream buffer, and then determines the quantization parameters based on the target number of bits. For a detailed explanation of determining the quantization parameters, please refer to steps S510 to 550 below.

[0155] S470 and decoding device 220 decode the bit stream of the coding unit of the current frame according to the quantization parameters to obtain the reconstructed image.

[0156] The decoding device 220 decodes the encoded data of the encoding unit according to the quantization parameters determined by the target number of bits of the encoding unit, and obtains the reconstructed image.

[0157] Decoding device 220 displays the reconstructed image. Alternatively, decoding device 220 transmits the reconstructed image to other display devices for display.

[0158] The reverse process of encoding a coding unit, i.e. decoding the bitstream of a coding unit, takes into account three factors: the image content of the coding unit, the number of lossy bits, and the fullness of the bitstream buffer. By dynamically setting the target number of bits and using the quantization parameters determined by the target number of bits to decode the bitstream of the coding unit, the accuracy of bit rate control is improved, and the number of encoded bits after encoding the image is reduced as much as possible while ensuring the quality of the reconstructed image.

[0159] Next, the process of determining the quantization parameters will be explained in detail with reference to the accompanying drawings. Figure 5 This is a flowchart illustrating an image encoding method provided in this application. Figure 3 The process of determining quantization parameters during the operation of the encoder 300 will be used as an example for explanation. Specifically, Figure 5 The method and process are Figure 4 The specific operational procedures included in S420 and S460 are described. For example... Figure 5 As shown, the method includes the following steps S510 to S550.

[0160] The S510 and encoder 300 determine the minimum number of encoded bits for the encoding unit based on the number of lossy bits and the fullness of the buffer.

[0161] In some alternative implementations, the lossy bit count is also called the block bit expectation of the coding unit or the lossy coding bit count.

[0162] The encoder 300 determines the adjustment value of the lossy bit count based on the number of bits of the encoded units already encoded in the bitstream buffer. Furthermore, the encoder 300 determines the lossy bit count based on the initial value and the adjustment value of the lossy bit count. The lossy bit count satisfies the following formula (1).

[0163] Bpp = Bpp INI +Bpp ADJ Formula (1)

[0164] Where Bpp represents the number of lossy bits, Bpp INI This represents the initial value of the lossy number of bits, Bpp. ADJ This represents the adjustment value for the number of lossy bits. The initial value for the number of lossy bits is determined based on the number of bits in the coding unit and the compression ratio. The compression ratio is determined according to the requirements of the actual application scenario.

[0165] The adjustment value for the number of lossy bits and (RcBuf) END -RcBuf T Proportional to RcBuf END This indicates the expected number of bits in the bitstream buffer at the end of the current frame encoding or decoding. RcBuf T This indicates the number of bits of the encoded units already encoded in the bitstream buffer.

[0166] If RcBuf END -RcBuf T If the difference is greater than 0, it means that the number of bits of the encoded units in the bitstream buffer has not exceeded the expected number of bits in the bitstream buffer at the end of the current frame encoding or decoding, and more target bits can be allocated to the unencoded units.

[0167] If RcBuf END -RcBuf T If the difference is less than 0, it means that the number of bits of the encoded units in the bitstream buffer exceeds the expected number of bits in the bitstream buffer at the end of the current frame encoding or decoding, and a smaller number of target bits can be allocated to the unencoded units.

[0168] If RcBuf END -RcBuf TA difference of 0 indicates that the number of bits of the encoded units in the bitstream buffer is equal to the expected number of bits in the bitstream buffer at the end of the current frame encoding or decoding, which allows for the allocation of fewer target bits to unencoded units.

[0169] RcBuf corresponding to different coding units in the current frame END The same. The RcBuf corresponding to the coding unit in different frames. END They can be the same or different.

[0170] The full size of the bitstream buffer satisfies the following formula (2).

[0171] F = RcBuf T / RcBuf MAX +X0 formula(2)

[0172] Where F represents the full size of the bitstream buffer, RcBuf T RcBuf represents the number of bits of the encoded units already encoded in the bitstream buffer. MAX This represents the maximum number of bits (storage capacity) allowed in the bitstream buffer, where X0 is a predefined parameter. To maintain the number of bits in the physical buffer, if the bitstream buffer is too full, the expected number of bits after lossy encoding of the coding unit when referencing the content of the coding unit is reduced, thereby decreasing the number of bits in the physical buffer.

[0173] The minimum number of encoded bits is obtained using the following formula (3):

[0174] B MIN = MAX((K4-K5×F) ×Bpp +K6,0) Formula (3)

[0175] Where F is the full size of the input buffer, Bpp is the expected number of input block bits (lossy bits), and K4, K5, and K6 are predefined parameters.

[0176] The S520 and encoder 300 determine the number of lossless encoding bits for the encoding unit based on the image content of the encoding unit.

[0177] The encoder 300 can determine the complexity level of the encoding unit based on the image content of the encoding unit. As described in S420, it will not be repeated here.

[0178] The lossless encoding bit count is used to indicate the expected number of bits after lossless encoding of the encoding unit.

[0179] In some embodiments, the number of lossless encoded bits may be a default value configured empirically.

[0180] In other embodiments, the encoder 300 sets the desired number of bits after lossless encoding of the unencoded encoding unit based on the number of bits of the already encoded encoding unit. For example, the encoder 300 can determine the lossless encoded bit count of the encoding unit by looking up a table based on the identifier and complexity level of the encoding unit. Assume B... LL This represents the number of bits in the lossless encoding. (B) LL =Record BLL [T][k], where T represents the identifier of the encoding unit and k represents the complexity level of the encoding unit.

[0181] S530 and encoder 300 determine the offset value of the encoding unit based on the image content of the encoding unit, the full size of the buffer, and the number of lossless encoding bits.

[0182] This offset value indicates the difference between the maximum number of encoded bits and the number of lossy bits after lossy encoding of the coding unit.

[0183] Optionally, depending on the lossy coding method, the coding unit has different offset values ​​for the maximum number of coded bits.

[0184] It is worth noting that before the encoder 300 determines different offset values, the encoder 300 can determine the relative lossless bit number of the encoding unit based on the maximum relative lossless bit number of the encoding unit, the maximum lossless bit number of the encoding unit, and the lossless bit number.

[0185] The maximum relative lossless bit count of the coding unit is used to indicate the maximum value of the relative lossless bit count of the coding unit at the pixel bit depth of the current frame.

[0186] For example, the relative lossless coding bit count of a coding unit can be determined by the following formula (4).

[0187] B relative =Clip(0,RcMaxRelativeBits,B LLMAX –B LL ) Formula (4)

[0188] Among them, B relative RcMaxRelativeBits represents the relative lossless bit count of the coding unit; RcMaxRelativeBits represents the maximum relative lossless bit count of the coding unit determined by the code control unit; B LLMAX B represents the maximum number of lossless encoded bits in the code control unit at the current pixel depth; LL Represents the number of lossless encoded bits in the current coding unit; Clip indicates that B... LLMAX –B LLThe value is limited to the range of 0 to RcMaxRelativeBits, i.e., B. LLMAX –B LL The minimum value of B is 0. LLMAX –B LL The maximum value is RcMaxRelativeBits.

[0189] In the first possible example, encoder 300 obtains the first offset value based on the relative lossless bit count of the encoding unit, the complexity level, the lossless bit count, and the buffer fullness.

[0190] For example, encoder 300 compares the relative lossless bit count of the encoding unit with a set first threshold and the complexity level of the encoding unit with a set second threshold. Under different comparison results, it processes the lossless bit count of the encoding unit and the buffer fullness of the bit stream buffer to obtain the offset value of the first maximum bit count (first offset value).

[0191] The first threshold is RcRelativeTh, which specifies the number of relatively lossless coded bits to be protected. In one feasible scenario, RcRelativeTh is also called the code control relatively lossless coding rate threshold, and RcRelativeTh is a 10-bit unsigned integer.

[0192] The second threshold is the complexity threshold RcComplexTh that is specified for protection. In one feasible case, RcComplexTh is also called the code control complexity level threshold, and RcComplexTh is a 3-bit unsigned integer.

[0193] The two thresholds mentioned above can be preset or customized according to the user's requirements for encoding and decoding performance; this application does not limit this.

[0194] For the different comparison results obtained by comparing the relative lossless encoded bits of the encoding unit of encoder 300 with a set first threshold and the complexity level of the encoding unit with a set second threshold, such as Figure 6 Example 1 in the document provides two possible scenarios.

[0195] Case 1: The relative lossless coded bits B of the coding unit relative If the value is greater than RcRelativeTh and the complexity level (K) of the coding unit is less than or equal to RcComplexTh, then the encoder 300 processes the lossless coding bits and buffer fullness according to the first rule to obtain the first offset value. This first rule can be implemented by the following formula (5).

[0196] bppOffset1=K9-(K10×F +K11)- (Sr-Scale)×B relative Formula (5)

[0197] Where bppOffset1 is the offset value of the first maximum number of encoded bits (i.e., the first offset value), F is the full size of the bitstream buffer, and B... relative Sr is the relative lossless coding bit count of the coding unit, Sr is a parameter related to the image chroma sampling rate, Scale is the set subjective quality protection value, and K9, K10 and K11 are agreed-upon parameters.

[0198] When the encoder determines the first offset value using the first rule, the coding unit is a coding unit that needs to be protected for subjective quality. The encoder increases the first offset value by setting Scale to increase the maximum number of target bits (Bpp+bppOffset1) that the coding unit can use, thereby reducing the quantization parameter of the coding unit, which is beneficial to improving the subjective quality of the reconstructed image after decoding the bit stream of the coding unit.

[0199] The second case: the relative lossless coded bits B of the coding unit. relative If the value is less than or equal to RcRelativeTh, or the complexity level of the coding unit is greater than RcComplexTh, then encoder 300 processes the lossless coding bit count and buffer fullness according to the second rule to obtain the first offset value. This second rule can be implemented by the following formula (6).

[0200] bppOffset1=K9-( K10×F +K11)- Sr×B relative Formula (6)

[0201] Where bppOffset1 is the first offset value, F is the full size of the bitstream buffer, and B... relative Sr is the relative lossless coding bit count of the coding unit, Sr is a parameter related to the image chroma sampling rate, and K9, K10 and K11 are predefined parameters.

[0202] When the encoder determines the first offset value using the second rule, the encoding unit is an encoding unit that does not require subjective quality protection. The encoder reduces the first offset value to reduce the maximum number of target bits allowed for the encoding unit, thereby increasing the quantization parameters of the encoding unit. This is beneficial for reducing the number of encoded bits in the bit stream obtained after encoding the encoding unit, reducing the storage space required for bit stream storage, and improving the communication efficiency for transmitting the bit stream.

[0203] It is worth noting that when the input data for formulas (5) and (6) are the same (i.e., the number of lossless encoded bits and the full size of the buffer), the first offset value determined by the encoder according to the first rule is greater than or equal to the first offset value determined according to the second rule. Optionally, in the B of the encoding unit... LLMAX –B LL =0 (i.e., B) LLMAX =B LL In the case of B, relative If the value is also 0, then Scale in the first rule will not affect the calculation result of bppOffset1; that is, in this case, the first offset value determined by the encoder according to the first rule is equal to the first offset value determined according to the second rule.

[0204] Optionally, the encoder 300 may determine whether to perform subjective quality protection based on the value of the Pro_Flag flag for subjective quality protection of the encoding unit.

[0205] For example, if Pro_Flag = 1, then the encoding unit needs to be subject to subjective quality protection, and the encoder determines the first offset value according to the first rule mentioned above.

[0206] For example, if Pro_Flag = 0, then the encoding unit does not need to perform subjective quality protection, and the encoder determines the first offset value according to the aforementioned second rule.

[0207] To illustrate with the above explanation of Scale: A larger Scale results in a larger bppOffset1 for a coding unit with Pro_Flag = 1, leading to stronger subjective protection of the target bit count for that coding unit. Conversely, a smaller Scale results in a smaller bppOffset1 for a coding unit with Pro_Flag = 1, leading to weaker subjective protection of the target bit count for that coding unit.

[0208] It is worth noting that the Pro_Flag value mentioned above is determined by comparing the relative lossless bit count of the encoding unit with a set first threshold and the complexity level of the encoding unit with a set second threshold, as obtained by the encoder 300. In some optional implementations, the Pro_Flag value may not be "0" or "1". The encoder 300 can set different values ​​for Pro_Flag depending on the type of encoding / decoding system or the supported encoded data format.

[0209] In the second possible example, such as Figure 6 In Example 2, encoder 300 obtains a second offset value based on the relative lossless and lossy bit counts of the coding unit. This second offset value is the offset of the second maximum number of coding bits and can be calculated using the following formula (7).

[0210] bppOffset2=MAX(K12-Bpp,K13)-B relative Formula (7)

[0211] Where bppOffset2 is the second offset value, Bpp is the expected number of input block bits (the number of lossy bits in the coding unit), and B... relative K12 and K13 are the relative lossless coding bits of the coding unit, and are pre-defined parameters.

[0212] In the third possible example, such as Figure 6 In Example 3, encoder 300 obtains the third offset value based on the fullness of the encoding unit's buffer. This third offset value is the offset value of the third maximum number of encoded bits, which can be calculated using the following formula (8).

[0213] bppOffset3 = K14 × (F) limit - F) Formula (8)

[0214] Where bppOffset3 is the third offset value, and F is the full size of the input buffer. limit K14 is a pre-defined limit for the full size of the buffer.

[0215] The offset value used to determine the maximum target number of bits for the coding unit is one of the first offset value, the second offset value, and the third offset value.

[0216] For example, the offset of the maximum target number of bits of the coding unit is determined as follows: the encoder selects the minimum offset value from the first offset value and the second offset value, and then uses the minimum of the minimum offset value and the third offset value as the offset of the maximum target number of bits of the coding unit. Optionally, the offset of the maximum target number of bits of the coding unit is determined by the following formula (9).

[0217] bppOffset=MIN (MIN(bppOffset1, bppOffset2), bppOffset3) Formula (9)

[0218] Where bppOffset is the offset value used to determine the maximum target number of bits of the coding unit, bppOffset1 is the first offset value in the first example, bppOffset2 is the second offset value in the second example, and bppOffset3 is the third offset value in the third example.

[0219] To achieve better coding performance and reconstruction quality, a code control module (or code control unit) needs to allocate different target bit counts to different coding units within the image. This maximizes the utilization of the specified total number of coding bits, resulting in the best possible quality for the decoded image. The decoder can determine the offset value of the maximum target bit count for a coding unit using different offset acquisition methods, thereby fully utilizing the number of coding bits and improving the image quality of the reconstructed image.

[0220] Furthermore, selecting the minimum value among multiple offset values ​​of the maximum target bit number relative to the lossy bit number, and clamping the target bit number of the coding unit based on the maximum target bit number corresponding to the minimum value, helps to improve the accuracy of the target bit number of the coding unit, thereby improving the accuracy of the quantization parameters and the subjective quality of the reconstructed image after the decoder decodes the bit stream of the coding unit according to the quantization parameters.

[0221] When the encoder obtains the offset value used to determine the maximum target number of bits for the coding unit, it can be obtained through any one or a combination of the three possible examples above.

[0222] Please continue to refer to Figure 5 The image encoding method provided in this embodiment also includes the following steps S540 and S550.

[0223] S540 and encoder 300 determine the maximum target number of bits for the coding unit based on the offset value, the number of lossy bits, and the minimum number of coding bits.

[0224] For example, encoder 300 can calculate the maximum target number of bits using the following formula (10).

[0225] B MAX =MAX(Bpp+bppOffset, B MIN ) Formula (10)

[0226] Among them, B MAX Let Bpp be the maximum target number of bits for the coding unit, Bpp be the lossy number of bits for the coding unit (expected block bit count), and bppOffset be the offset determined by formula (9). MIN The minimum number of coded bits is determined by formula (3).

[0227] S550 and encoder 300 clamp the target bit number of the encoding unit according to the maximum target bit number and the minimum encoding bit number, and obtain the clamped target bit number.

[0228] The number of target bits after clamping is used to determine the quantization parameters of the coding unit. The clamping process can be implemented by the following formula (11).

[0229] B` TGT=MIN(MAX(B MIN B TGT ), B MAX ) Formula (11)

[0230] Among them, B` TGT B represents the target number of bits after clamping. TGT B represents the target number of bits before clamping. MIN B is the minimum number of encoded bits determined by formula (3). MAX The maximum target number of bits for the coding unit is determined by formula (10).

[0231] For example, if B MIN >B TGT B MIN MAX Then the number of target bits after clamping is B. MIN .

[0232] For example, if B MIN >B TGT B MIN >B MAX Then the number of target bits after clamping is B. MAX .

[0233] For example, if B MIN TGT B TGT MAX Then the number of target bits after clamping is B. TGT .

[0234] The target number of bits after clamping can be used to determine the quantization parameters of the coding unit. For example, encoder 300 determines the quantization parameters of the coding unit based on the lossless coding number of the coding unit and the target number of bits after clamping.

[0235] QP = (B LL -( B` TGT +X8) ×X9) ×X10 Formula (12)

[0236] Where QP is the quantization parameter of the coding unit, B LL B' is the number of lossless coded bits in a coding unit. TGT The target number of bits after clamping the coding unit determined by formula (11), X8, X9 and X10 are agreed parameters.

[0237] ​​​In the image encoding method provided in this embodiment, the encoder can use a formula to quantitatively and in real time calculate the more accurate clamping range of the current encoding unit. That is, the number of lossless encoded bits of the current encoding unit is obtained by using the complexity level of the current encoding unit, and the clamping range is quantitatively calculated by the number of lossless encoded bits, the number of lossy encoded bits, and the full extent of the bitstream buffer of the current encoding unit.

[0238] In addition, the encoder can set thresholds for subjective quality protection during the bit rate control process (such as RcRelativeTh and RcComplexTh mentioned above). By flexibly setting the thresholds and adjusting the clamping force, the subjective quality of the coding unit can be effectively protected.

[0239] For example, clamping strength refers to determining the number of coding units that perform subjective quality protection during the encoding process of multiple coding units by adjusting the threshold size. If the clamping strength is large (low complexity threshold): the number of coding units that need subjective quality protection is small, and the coverage of coding units performing subjective quality protection during the encoding process is smaller. If the clamping strength is small (high complexity threshold): the number of coding units that need subjective quality protection is large, and the coverage of coding units performing subjective quality protection during the encoding process is larger.

[0240] Understandably, the encoder determines the expected value of the encoded unit based on the complexity of the current encoded unit relative to the entire frame image, i.e., derived from the complexity level of the encoded unit and the average complexity level of the entire frame. Therefore, to achieve better encoding performance and reconstructed image quality, the code control module (or code control unit) allocates different expected bit counts to different encoded units within the image, thereby maximizing the utilization of the specified total number of encoded bits and achieving the best possible quality for the decompressed image.

[0241] Figure 7 A flowchart illustrating the rate control process during decoding is shown. Figure 7 and Figure 5 The difference lies in the fact that the quantization parameters of the bitrate control output are used in the inverse quantization process. For the process of determining the quantization parameters, please refer to the above. Figure 5 The explanation in the text.

[0242] It is understood that, in order to achieve the functions in the above embodiments, the computing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0243] The above text combines Figures 1 to 7 The image encoding and decoding method provided according to this embodiment is described in detail below. Figure 8 This describes the encoding / decoding apparatus provided according to this embodiment.

[0244] Figure 8 This is a schematic diagram of an encoding / decoding device provided in this application. These encoding / decoding devices can be used to implement the functions of the encoder and decoder in the above method embodiments, and therefore can also achieve the beneficial effects of the above method embodiments. In this embodiment, the encoding / decoding device can be as follows: Figure 3 The encoder 300 and decoder 400 shown can also be modules (such as chips) applied to computing devices.

[0245] like Figure 8 As shown, the encoding / decoding device 800 includes a communication module 810, a code control module 820, an encoding module 830, and a storage module 840. The encoding / decoding device 800 is used to implement the above-mentioned... Figure 4 The methods illustrated in the figures show the functions of the encoding and decoding devices, or the encoder and decoder in other accompanying drawings.

[0246] When the encoder / decoder device 800 is used to implement the encoder function, the specific functions of each module are as follows.

[0247] The communication module 810 is used to acquire the coding units to be encoded in the current frame. For example, the communication module 810 is used to perform... Figure 4 S410.

[0248] The code control module 820 is used to determine the target number of bits for the encoding unit based on the image content, the number of lossy bits, and the full capacity of the bitstream buffer, and to determine quantization parameters based on the target number of bits for the encoding unit. For example, the code control module 820 is used to execute... Figure 4 S420.

[0249] Encoding module 830 is used to encode the encoding unit according to the quantization parameters to obtain the bit stream of the encoding unit. For example, encoding module 830 is used to perform... Figure 4 S430.

[0250] When the encoding / decoding device 800 is used to implement the function of the decoder, the specific functions of each module are as follows.

[0251] The communication module 810 is used to acquire the bit stream of the encoded unit to be decoded in the image bit stream. For example, the communication module 810 is used to perform... Figure 4 S450.

[0252] The code control module 820 is used to determine the target number of bits for the encoding unit based on the image content, the number of lossy bits, and the full capacity of the bitstream buffer, and to determine quantization parameters based on the target number of bits for the encoding unit. For example, the code control module 820 is used to execute... Figure 4 S460.

[0253] Encoding module 830 is used to decode the bitstream of the encoding unit according to the quantization parameters to obtain the reconstructed image of the encoding unit. For example, encoding module 830 is used to perform... Figure 4 Medium S470.

[0254] The storage module 840 is used to store the number of bits of data in the bit stream buffer so that the code control module 820 can determine the quantization parameters.

[0255] It should be understood that the encoding / decoding device 800 in this application embodiment can be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can also be implemented using software. Figure 4 The method shown, and its various modules, can also be software modules, and the encoding / decoding device 800, and its various modules can also be software modules.

[0256] The encoding / decoding apparatus 800 according to the embodiments of this application can be used to execute the methods described in the embodiments of this application. The above and other operations and / or functions of each unit in the encoding / decoding apparatus 800 are respectively for implementing the corresponding processes of the methods in the foregoing figures. For the sake of brevity, they will not be described again here.

[0257] Figure 9 This is a schematic diagram of an image processing system provided in this application. The image processing system is described using a mobile phone as an example. The mobile phone or a chip system built into the mobile phone includes: a memory 910, a processor 920, a sensor component 930, a multimedia component 940, and an input / output interface 950. The following description, in conjunction with... Figure 9 This section provides a detailed introduction to the various components of a mobile phone or the chip system built into it.

[0258] The memory 910 can be used to store data, software programs, and modules; it mainly includes a program storage area and a data storage area. The program storage area can store software programs, including instructions formed by code, including but not limited to the operating system and applications required for at least one function, such as sound playback function, image playback function, etc.; the data storage area can store data created according to the use of the mobile phone, such as audio data, image data, phonebook, etc. In the embodiments of this application, the memory 910 can be used to store the number of bits of data in the bit stream buffer, etc. In some feasible embodiments, there can be one memory or multiple memories; the memory can include floppy disks, hard disks such as internal hard disks and external hard disks, magnetic disks, optical disks, magneto-optical disks such as CD_ROM, DCD_ROM, non-volatile storage devices such as RAM, ROM, PROM, EPROM, EEPROM, flash memory, or any other form of storage medium known in the art.

[0259] The processor 920 is the control center of the mobile phone, connecting various parts of the device through various interfaces and lines. It executes software programs and / or software modules stored in the memory 910, and calls data stored in the memory 910 to perform various functions and process data, thereby providing overall monitoring of the mobile phone. In this application embodiment, the processor 920 can be used to execute one or more steps in the method embodiments of this application. For example, the processor 920 can be used to execute one or more steps S420 to S470 in the following method embodiments. In some feasible embodiments, the processor 920 can be a single-processor architecture, a multi-processor architecture, a single-threaded processor, or a multi-threaded processor, etc.; in some feasible embodiments, the processor 920 can include at least one of a central processing unit, a general-purpose processor, a digital signal processor, a neural network processor, an image processing unit, an image signal processor, a microcontroller, or a microprocessor, etc. In addition, the processor 920 may further include other hardware circuits or accelerators, such as application-specific integrated circuits, field-programmable gate arrays, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 920 can also be a combination of functions that perform computing, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc.

[0260] Sensor component 930 includes one or more sensors for providing various aspects of the mobile phone's status assessment. Sensor component 930 may include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications, i.e., as part of a camera or webcam. In this embodiment, sensor component 930 can be used to support image acquisition by the webcam in multimedia component 940. Furthermore, sensor component 930 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor. Sensor component 930 can detect the mobile phone's acceleration / deceleration, orientation, on / off state, relative positioning of components, or temperature changes of the mobile phone.

[0261] The multimedia component 940 provides a screen that serves as an output interface between the mobile phone and the user. This screen can be a touch panel, and when it is a touch panel, it can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. Furthermore, the multimedia component 940 includes at least one camera, for example, a front-facing camera and / or a rear-facing camera. When the mobile phone is in an operating mode, such as a shooting mode or video mode, the front-facing camera and / or the rear-facing camera can sense external multimedia signals, which are used to form image frames. Each front-facing and rear-facing camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0262] Input / output interface 950 provides an interface between processor 920 and peripheral interface modules, such as keyboards, mice, or USB (Universal Serial Bus) devices. In one possible implementation, input / output interface 950 may have only one input / output interface or multiple input / output interfaces.

[0263] Although not shown, the mobile phone may also include audio components and communication components, such as a microphone for audio components and a wireless fidelity (WiFi) module and a Bluetooth module for communication components, which will not be described in detail in the embodiments of this application.

[0264] The image processing system described above can be a general-purpose device or a dedicated device. For example, the image processing system can be an edge device (e.g., a box carrying a chip with processing capabilities). Alternatively, the image processing system can also be a server or other device with computing capabilities.

[0265] It should be understood that the image processing system according to this embodiment can correspond to the encoding and decoding device 800 in this embodiment, and can correspond to the corresponding subject executing any method according to the foregoing figures. The above and other operations and / or functions of each module in the encoding and decoding device 800 are respectively for implementing the corresponding processes of each method in the foregoing figures. For the sake of brevity, they will not be described in detail here.

[0266] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a computing device. Of course, the processor and storage medium can also exist as discrete components in a network device or terminal device.

[0267] This application also provides a chip system including a processor for implementing the functions of the data processing unit in the above-described method. In one possible design, the chip system further includes a memory for storing program instructions and / or data. This chip system may be composed of chips or may include chips and other discrete devices.

[0268] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0269] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image decoding method, characterized in that, The method includes: Obtain the bitstream of the coding units in the image bitstream; The quantization parameters of the encoding unit are determined based on the complexity level of the encoding unit, the current number of lossy bits, and the fullness of the bitstream buffer. The quantization parameters are used to decode the bitstream of the encoding unit. The complexity level includes at least one of luminance complexity level and chrominance complexity level, and the current number of lossy bits is determined based on the initial value of the number of lossy bits of the encoding unit and the adjustment value of the number of lossy bits.

2. The method according to claim 1, characterized in that, The step of determining the quantization parameters of the encoding unit based on its complexity level, the current number of lossy bits, and the fullness of the bitstream buffer includes: The minimum number of encoded bits for the encoding unit is determined based on the current number of lossy bits and the full size of the buffer. The number of lossless encoded bits of the encoding unit is determined according to the complexity level of the encoding unit; The offset value of the encoding unit is determined based on the complexity level of the encoding unit, the full size of the buffer, and the number of lossless encoded bits. The maximum target number of bits for the coding unit is determined based on the offset value, the current number of lossy bits, and the minimum number of encoded bits. The target bit count of the coding unit is clamped according to the maximum target bit count and the minimum coding bit count to obtain the clamped target bit count; the clamped target bit count is used to determine the quantization parameters of the coding unit.

3. The method according to claim 2, characterized in that, Determining the offset value of the encoding unit based on the complexity level of the encoding unit, the full size of the buffer, and the number of lossless encoded bits includes: The relative lossless bit count of the coding unit is determined based on the maximum relative lossless bit count of the coding unit, the maximum lossless bit count of the coding unit, and the lossless bit count. The first offset value is obtained based on the relative lossless encoding bits of the encoding unit, the complexity level, the lossless encoding bits, and the fullness of the buffer. And / or, obtain a second offset value based on the relative lossless coded bits of the coding unit and the current lossy bits; And / or, obtain a third offset value based on the fullness of the buffer of the encoding unit; The offset value used to determine the maximum target number of bits of the coding unit is one of the first offset value, the second offset value, and the third offset value.

4. The method according to claim 3, characterized in that, The offset of the maximum target number of bits of the coding unit is determined in the following way: Select the minimum offset value from the first offset value and the second offset value; The minimum value between the minimum offset value and the third offset value is used as the offset value for the maximum target number of bits of the coding unit.

5. The method according to claim 3 or 4, characterized in that, The step of obtaining the first offset value based on the relative lossless bit count of the encoding unit, the complexity level, the lossless bit count, and the buffer fullness includes: Determine whether the relative lossless coded bits of the coding unit are greater than a set first threshold and whether the complexity level of the coding unit is less than or equal to a set second threshold; If so, then process the number of lossless encoded bits and the full size of the buffer according to the first rule to obtain the first offset value; If not, then process the lossless encoded bit count and the buffer fullness according to the second rule to obtain the first offset value; the first offset value determined according to the first rule is greater than or equal to the first offset value determined according to the second rule.

6. The method according to any one of claims 2-4, characterized in that, The target number of bits for the encoding unit is used to indicate the expected number of bits after lossy encoding of the encoding unit with reference to the complexity level of the encoding unit.

7. The method according to any one of claims 1-4, characterized in that, The method further includes: The bitstream of the coding unit is decoded according to the quantization parameters to obtain the reconstructed image of the coding unit.

8. An image encoding method, characterized in that, The method includes: Get the coding units in the current frame; The quantization parameters of the encoding unit are determined based on the complexity level of the encoding unit, the current number of lossy bits, and the fullness of the bitstream buffer. The quantization parameters are used to encode the bitstream of the encoding unit. The complexity level includes at least one of luminance complexity level and chrominance complexity level, and the current number of lossy bits is determined based on the initial value of the number of lossy bits of the encoding unit and the adjustment value of the number of lossy bits.

9. The method according to claim 8, characterized in that, The step of determining the quantization parameters of the encoding unit based on its complexity level, the current number of lossy bits, and the fullness of the bitstream buffer includes: The minimum number of encoded bits for the encoding unit is determined based on the current number of lossy bits and the full size of the buffer. The number of lossless encoded bits of the encoding unit is determined according to the complexity level of the encoding unit; The offset value of the encoding unit is determined based on the complexity level of the encoding unit, the full size of the buffer, and the number of lossless encoded bits. The maximum target number of bits for the coding unit is determined based on the offset value, the current number of lossy bits, and the minimum number of encoded bits. The target bit count of the coding unit is clamped according to the maximum target bit count and the minimum coding bit count to obtain the clamped target bit count; the clamped target bit count is used to determine the quantization parameters of the coding unit.

10. The method according to claim 9, characterized in that, Determining the offset value of the encoding unit based on the complexity level of the encoding unit, the full size of the buffer, and the number of lossless encoded bits includes: The relative lossless bit count of the coding unit is determined based on the maximum relative lossless bit count of the coding unit, the maximum lossless bit count of the coding unit, and the lossless bit count. The first offset value is obtained based on the relative lossless encoding bits of the encoding unit, the complexity level, the lossless encoding bits, and the fullness of the buffer. And / or, obtain a second offset value based on the relative lossless coded bits of the coding unit and the current lossy bits; And / or, obtain a third offset value based on the fullness of the buffer of the encoding unit; The offset value used to determine the maximum target number of bits of the coding unit is one of the first offset value, the second offset value, and the third offset value.

11. The method according to claim 10, characterized in that, The offset of the maximum target number of bits of the coding unit is determined in the following way: Select the minimum offset value from the first offset value and the second offset value; The minimum value between the minimum offset value and the third offset value is used as the offset value for the maximum target number of bits of the coding unit.

12. The method according to claim 10 or 11, characterized in that, The step of obtaining the first offset value based on the relative lossless bit count of the encoding unit, the complexity level, the lossless bit count, and the buffer fullness includes: Determine whether the relative lossless coded bits of the coding unit are greater than a set first threshold and whether the complexity level of the coding unit is less than or equal to a set second threshold; If so, then process the number of lossless encoded bits and the full size of the buffer according to the first rule to obtain the first offset value; If not, then process the lossless encoded bit count and the buffer fullness according to the second rule to obtain the first offset value; the first offset value determined according to the first rule is greater than or equal to the first offset value determined according to the second rule.

13. The method according to any one of claims 9-11, characterized in that, The target number of bits for the encoding unit is used to indicate the expected number of bits after lossy encoding of the encoding unit with reference to the complexity level of the encoding unit.

14. The method according to any one of claims 8-11, characterized in that, The method further includes: The bit stream of the coding unit is encoded according to the quantization parameters to obtain the bit stream of the coding unit.

15. A codec device, characterized in that, The encoding / decoding device includes: The communication module is used to obtain the bit stream of coding units in the image bit stream, or to obtain the coding units in the current frame; The code control module is used to determine the quantization parameters of the encoding unit based on the complexity level of the encoding unit, the current number of lossy bits, and the fullness of the bit stream buffer. The quantization parameters are used to encode the bit stream of the encoding unit. The complexity level includes at least one of luminance complexity level and chrominance complexity level, and the current number of lossy bits is determined based on the initial value of the number of lossy bits of the encoding unit and the adjustment value of the number of lossy bits.

16. A decoder, characterized in that, The decoder includes at least one processor and a memory, wherein the memory is used to store a computer program such that when the computer program is executed by the at least one processor, it implements the method as described in any one of claims 1-7.

17. An encoder, characterized in that, The encoder includes at least one processor and a memory, wherein the memory is used to store a computer program such that when the computer program is executed by the at least one processor, it implements the method as described in any one of claims 8-14.

18. A codec system, characterized in that, The encoding / decoding system includes an encoder as described in claim 17 and a decoder as described in claim 16, wherein the encoder is used to perform the operational steps of the method according to any one of claims 8-14, and the decoder is used to perform the method according to any one of claims 1-7.

19. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when executed by an electronic device, implement the method of any one of claims 1-7. And / or, when the computer program or instructions are executed by an electronic device, the method of any one of claims 8-14 is implemented.

20. A computer program product, characterized in that, The computer program product includes instructions that, when the computer program product is run on an encoder, the encoder executes the instructions to implement the method of any one of claims 8-14; And / or, when the computer program product is run on the decoder, the decoder executes the instructions to implement the method of any one of claims 1 to 7.

Images