Coding method and device

Abstract

The invention discloses a coding method and device, and relates to the field of image processing. The method comprises the following steps: acquiring quantization parameters of image blocks in an image frame; updating a quantization parameter of an image block satisfying a condition in the image frame, wherein the condition is used for indicating that the image content of the position of the image block changes relative to a reference frame of inter-frame prediction; and performing quantization coding based on the quantization parameter of the image block in the image frame to obtain a code stream.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an encoding method and apparatus. Background Technology

[0002] In conventional image processing scenarios (such as using cameras, camcorders, or other image capture devices), in order to reduce end-side transmission and storage bandwidth, the acquired image data (single-frame images or image frame sequences) is first compressed and encoded according to digital video coding standards before transmission or storage.

[0003] During encoding, appropriate bitrate control algorithms are employed to balance image quality and bitrate. Commonly used bitrate control algorithms include constant bitrate (CBR), variable bitrate (VBR), and adaptive variable bitrate (AVBR). CBR maintains a constant bitrate, while VBR and AVBR adjust the bitrate based on the dynamic or static scenes in the image and set a maximum bitrate limit.

[0004] Therefore, current encoding schemes may have insufficient bitrates for complex images (images with complex details or a high proportion of dynamic objects, or others), resulting in image quality loss such as ghosting or mosaic. Summary of the Invention

[0005] This application provides an encoding method and apparatus that reduces the access time of terminal nodes and improves the management efficiency of management nodes for carriers in carrier aggregation scenarios.

[0006] In a first aspect, an encoding method is provided, which can be applied to an encoding device. The method may include: obtaining quantization parameters of image blocks in an image frame; updating the quantization parameters of image blocks in the image frame that meet certain conditions, wherein the conditions are used to indicate that the image content at the location of the image block has changed relative to a reference frame for inter-frame prediction; and quantizing and encoding based on the quantization parameters of the image blocks in the image frame to obtain a bitstream.

[0007] The encoding method provided in this application updates the quantization parameters for image blocks whose image content has changed relative to the reference frame for inter-frame prediction. Since image blocks whose image content has changed relative to the reference frame for inter-frame prediction are highly likely to experience quality loss phenomena such as motion blur and mosaic, updating the quantization parameters for these image blocks replaces the initial quantization parameters provided in the encoding parameters. The updated and more reasonable quantization parameters can guarantee the image quality of this type of image block, thereby avoiding image quality loss caused by the encoding process and improving the quality of image transmission.

[0008] One possible implementation is that the above conditions may include: the position pointed to by the motion vector of the image block in the image frame that has undergone inter-frame prediction, and intra-frame prediction has been performed.

[0009] The position pointed to by the motion vector can be the coordinates of the image block referenced in the reference frame when the motion vector is determined, or the position of the image block in the current image frame.

[0010] In another possible implementation, the quantization parameters obtained by the encoding device can be the quantization parameters of each image block determined by the rate control scheme.

[0011] In another possible implementation, updating the quantization parameters of image blocks that meet the conditions in the image frame includes: calculating the complexity of the image blocks that meet the conditions; and updating the quantization parameters of the image blocks that meet the conditions to quantization parameters that are inversely proportional to the complexity. For image blocks that are at risk of image quality loss due to bitrate, quantization parameters that are inversely proportional to their complexity are configured to ensure that image blocks with higher complexity use the lowest possible quantization parameters, thereby increasing the bitrate of images with higher complexity and avoiding image quality loss caused by insufficient bitrate.

[0012] In another possible implementation, the encoding device records the mapping relationship between different complexities and quantization parameters. For the first image block (any image block in the current image frame that satisfies the above conditions), the quantization parameters are updated to be inversely proportional to the complexity based on the first image block's quantization parameters. Specifically, this can be achieved by determining the quantization parameter corresponding to the complexity of the first image block in the mapping relationship as the updated quantization parameter for the first image block. Based on practical experience, matching complexities and quantization parameters can be configured in the mapping relationship. During the actual encoding process, the matching quantization parameter is selected based on the image's complexity to ensure the quality of the image block.

[0013] In another possible implementation, the aforementioned complexity can be the variance mean, and the quantization parameters included in the mapping relationship are absolute values. For example, the mapping relationship can be shown in the following table:

[0014]

[0015]

[0016] In another possible implementation, the aforementioned complexity can be the variance mean, and the quantization parameters included in the mapping relationship are relative values, which represent the difference from the initial quantization parameters in the encoding parameters. For example, the mapping relationship can be shown in the following table:

[0017] variance mean C The corresponding relative value of the quantization parameter ＜6 0 6＜C＜12 -2 12＜C＜28 -4 28＜C＜48 -6 48＜C＜68 -8 68＜C＜88 -10 88＜C＜108 -12 108＜C＜128 -13 ＞128 -14

[0018] In another possible implementation, the encoding device employs region of interest (ROI) encoding. For the second image block (any image block in the current image frame that meets certain conditions), the quantization parameters of the second image block that meets the conditions are updated. Specifically, the quantization parameters of the second image block are updated to the quantization parameters of a third image block, which is the image block in the ROI region closest to the second image block. Since the quantization parameters of the ROI region are relatively small, image quality can be guaranteed. Therefore, configuring the quantization parameters of the image blocks that meet the conditions as those of the image blocks in the ROI region ensures quality and avoids compromising the quality of image blocks with complex content.

[0019] In another possible implementation, updating the quantization parameters of image blocks that meet the conditions in the image frame includes: updating the quantization parameters of image blocks that meet the conditions and whose complexity is greater than or equal to a threshold.

[0020] In another possible implementation, the image block is a largest coding unit (LCU) or a coding tree unit (CTU).

[0021] Secondly, a decoding method is provided, which can be applied to a decoding device. The method may include: acquiring a bitstream and a set of quantization parameters, wherein the quantization parameters include the quantization parameters of each image block in the image frame; and performing an inverse quantization operation on the bitstream according to the quantization parameters.

[0022] The quantization parameters obtained by the decoding device are the quantization parameters used in the encoding method described in the first aspect or any possible implementation thereof when performing quantization operations.

[0023] It should be noted that the second aspect provides a decoding method for decoding the bitstream obtained by the encoding method described in the first aspect or any possible line-of-sight method; the specific implementation will not be elaborated further.

[0024] Thirdly, an encoding apparatus is provided, comprising modules for performing the method of the first aspect or any possible implementation thereof. For example, the encoding apparatus includes a communication module and an encoding module. Wherein:

[0025] The communication module is used to obtain the quantization parameters of image blocks in an image frame.

[0026] The encoding module updates the quantization parameters of image blocks that meet certain conditions within the image frame. Based on these quantization parameters, it performs quantization encoding to obtain the bitstream. The condition indicates that the image content at the location of the image block has changed relative to the reference frame used for inter-frame prediction.

[0027] One possible implementation is that the above conditions may include: the position pointed to by the motion vector of the image block in the image frame that has undergone inter-frame prediction, and intra-frame prediction has been performed.

[0028] The position pointed to by the motion vector can be the coordinates of the image block referenced in the reference frame when the motion vector is determined, or the position of the image block in the current image frame.

[0029] In another possible implementation, the quantization parameters obtained by the encoding device can be the quantization parameters of each image block determined by the rate control scheme.

[0030] In another possible implementation, the aforementioned encoding module can specifically be used to: calculate the complexity of image blocks that meet the conditions; and update the quantization parameters of the image blocks that meet the conditions to quantization parameters that are inversely proportional to the complexity. For image blocks that pose a risk of image quality loss due to bitrate issues, quantization parameters that are inversely proportional to their complexity are configured to ensure that image blocks with higher complexity use the lowest possible quantization parameters, thereby increasing the bitrate of images with higher complexity and avoiding image quality loss caused by insufficient bitstream.

[0031] In another possible implementation, the encoding device records the mapping relationship between different complexities and quantization parameters. For the first image block (any image block in the current image frame that satisfies the above conditions), the encoding module updates the quantization parameters of the first image block to quantization parameters inversely proportional to the complexity. Specifically, this can be achieved by determining the quantization parameter corresponding to the complexity of the first image block in the mapping relationship as the updated quantization parameter for the first image block. Based on practical experience, matching complexities and quantization parameters in the mapping relationship can be configured. During the actual encoding process, the matching quantization parameter is selected based on the image complexity to ensure the quality of the image block.

[0032] In another possible implementation, the aforementioned complexity can be the variance mean, and the quantization parameters included in the mapping relationship are absolute values. For example, the mapping relationship can be shown in the following table:

[0033] variance mean C Corresponding quantization parameters ＜6 45 6＜C＜12 43 12＜C＜28 41 28＜C＜48 39 48＜C＜68 37 68＜C＜88 35 88＜C＜108 33 108＜C＜128 30 ＞128 28

[0034] In another possible implementation, the aforementioned complexity can be the variance mean, and the quantization parameters included in the mapping relationship are relative values, which represent the difference from the initial quantization parameters in the encoding parameters. For example, the mapping relationship can be shown in the following table:

[0035] variance mean C The corresponding relative value of the quantization parameter ＜6 0 6＜C＜12 -2 12＜C＜28 -4 28＜C＜48 -6 48＜C＜68 -8 68＜C＜88 -10 88＜C＜108 -12 108＜C＜128 -13 ＞128 -14

[0036] In another possible implementation, the encoding device employs ROI encoding. For the second image block (any image block in the current image frame that meets certain conditions), the encoding module updates the quantization parameters of the second image block that meets the conditions. Specifically, it updates the quantization parameters of the second image block to the quantization parameters of a third image block, which is the image block in the ROI region of the image frame that is closest to the second image block. Since the quantization parameters of the ROI region are relatively small, image quality can be guaranteed. Therefore, configuring the image block that meets the conditions with the quantization parameters of the image block in the ROI region can ensure quality and avoid compromising the quality of image blocks with complex content.

[0037] In another possible implementation, the encoding module is specifically used to update the quantization parameters of image blocks whose complexity is greater than or equal to a threshold among the image blocks that meet the conditions.

[0038] In another possible implementation, the image block is an LCU, or a CTU.

[0039] Fourthly, a decoding apparatus is provided, comprising modules for performing the methods of the second aspect or any possible design of the second aspect. For example, the decoding apparatus includes a communication module and a decoding module.

[0040] A communication module is used to acquire the bitstream and a set of quantization parameters, including the quantization parameters of each image block in the image frame; a decoding module is used to perform inverse quantization on the bitstream according to the quantization parameters. The quantization parameters received by the decoding device are the quantization parameters used in the quantization operation during the encoding method described in the first aspect or any possible implementation thereof.

[0041] Fifthly, an encoder is provided, the encoder including at least one processor and a memory, wherein the memory is used to store a computer program such that when the computer program is executed by at least one processor, it implements the method described in the first aspect or any possible implementation of the first aspect.

[0042] In a sixth aspect, a decoder is provided, the decoder comprising at least one processor and a memory, wherein the memory is used to store a computer program such that when the computer program is executed by at least one processor, it implements the method described in the second aspect or any possible implementation thereof.

[0043] In a seventh aspect, a coding and decoding system is provided, the coding and decoding system comprising an encoder as described in the fifth aspect and a decoder as described in the sixth aspect.

[0044] Eighthly, a chip is provided, comprising: a processor and a power supply circuit; wherein the power supply circuit is used to supply power to the processor; the processor is used to perform operation steps of the method in the first aspect or any possible implementation of the first aspect, or to perform operation steps of the method in the second aspect or any possible implementation of the second aspect.

[0045] Ninth aspect, a computer program product is provided, the computer program product including a computer program or instructions, which, when the computer program product is run on a processor, causes the processor to perform the operation steps of the method in the first aspect or any possible implementation of the first aspect, or to perform the operation steps of the method in the second aspect or any possible implementation of the second aspect.

[0046] A tenth aspect provides a computer-readable storage medium comprising: computer software instructions; wherein, when the computer software instructions are executed in a computing device, the computing device performs operational steps of the method of the first aspect or any possible implementation thereof, or performs operational steps of the method of the second aspect or any possible implementation thereof, or the computer-readable storage medium comprises a bitstream obtained by performing operational steps of the method of the first aspect or any possible implementation thereof.

[0047] Eleventhly, a bitstream is provided, said bitstream being obtained by performing operational steps of the method in the first aspect or any possible implementation of the first aspect.

[0048] In a twelfth aspect, a method for storing a bitstream is provided, the method comprising storing a bitstream obtained by performing operational steps of the method in the first aspect or any possible implementation thereof.

[0049] In a thirteenth aspect, an apparatus for storing a bitstream is provided, the apparatus being used to store a bitstream obtained by performing operational steps of the method in the first aspect or any possible implementation of the first aspect.

[0050] In a fourteenth aspect, a method for transmitting a bitstream is provided, the method comprising transmitting a bitstream obtained by performing operational steps of the method in the first aspect or any possible implementation thereof.

[0051] In a fifteenth aspect, an apparatus for transmitting a bitstream is provided, the apparatus being used to transmit a bitstream obtained by performing operational steps of the method in the first aspect or any possible implementation of the first aspect.

[0052] The technical effects of any of the implementation methods in aspects two through fifteen can be found in the technical effects of the corresponding implementation methods in aspects one through two, and will not be repeated here.

[0053] It should be noted that any of the possible implementations of any of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a bitrate control process based on the RQ model;

[0055] Figure 2 This is a schematic diagram of a bitrate control scenario;

[0056] Figure 3 A schematic diagram of the structure of an encoding / decoding system provided in this application;

[0057] Figure 4 A schematic diagram illustrating an application scenario of the encoding method provided in this application;

[0058] Figure 5 A schematic diagram of the structure of an encoder and decoder provided in this application;

[0059] Figure 6 A flowchart illustrating an encoding method provided in this application;

[0060] Figure 7 A schematic diagram of an encoder architecture provided for this application;

[0061] Figure 8 A schematic diagram illustrating the principle of an encoding method provided in this application;

[0062] Figure 9 A flowchart illustrating another encoding method provided in this application;

[0063] Figure 10 A schematic diagram illustrating the principle of another encoding method provided in this application;

[0064] Figure 11 A comparative diagram of encoding effects provided for this application;

[0065] Figure 12 Another comparative diagram of encoding effects provided for this application;

[0066] Figure 13 A schematic diagram of quantization parameters carried in an coded image provided in this application;

[0067] Figure 14 A schematic diagram of the structure of an encoding device provided in this application;

[0068] Figure 15 This is a schematic diagram of the structure of an encoder provided in this application. Detailed Implementation

[0069] To facilitate understanding, the main terms used in this application will be explained first.

[0070] Bitrate (or data rate) refers to the amount of data a video file uses per unit of time; it's also called bitstream. Generally speaking, at the same resolution, the higher the bitrate of a video file, the lower the compression ratio and the higher the picture quality. A higher bitrate means a higher sampling rate per unit of time, resulting in a higher data stream precision. The processed file is closer to the original file, with better image quality and clearer picture, but it also requires higher decoding capabilities from the playback device.

[0071] Rate control (RC) refers to the process of determining the output bitrate in video encoding.

[0072] CBR (Constant Bitrate Control): Ensures a stable encoding bitrate within the bitrate statistics period. It performs compression operations at a constant encoding bitrate, so the bitrate remains constant regardless of whether there are moving objects in the image.

[0073] VBR (Dynamic Encoding Control Mode): The bitrate of the encoding compression changes with the quality of the picture; the bitrate is low in static scenes and increases in dynamic scenes.

[0074] AVBR (Variable Bitrate Control): This is an improved algorithm of VBR. Internally, it detects the current scene state to determine whether it is dynamic or static. It uses a higher bitrate for encoding and compression when there are moving objects, and actively reduces the bitrate when the scene is static.

[0075] Quantization refers to the process of mapping continuous values ​​of a signal into multiple discrete amplitude values.

[0076] The quantization parameter (QP) is used during the encoding process to quantize the residual values ​​generated by the prediction operation or the coefficients generated by the transform operation. During the decoding process, the syntax elements are dequantized to obtain the residual values ​​or coefficients. The quantization parameter is a parameter used in the quantization process. The QP value is inversely proportional to the bit rate (i.e., the bit rate). Generally, the larger the quantization parameter value, the more pronounced the quantization, the worse the quality of the reconstructed image, and the lower the bit rate; conversely, the smaller the quantization parameter value, the better the quality of the reconstructed image, and the higher the bit rate.

[0077] Rate-distortion optimization (RDO) is an optimization method to improve video compression performance. Its principle is to simultaneously optimize video distortion (picture quality) and bitrate (the number of bits required for encoding) to achieve an optimal balance. It is widely used in coding standards such as H.265, H.264, Audio Video Coding Standard (AVS), and Surveillance Video and Audio Coding (SAVC).

[0078] In typical camera usage scenarios, to reduce end-side transmission and storage bandwidth, the generated video data is compressed using common digital video coding standards such as H.264 and H.265 before transmission and storage. During encoding, to cope with different scenarios and to ensure both compressed image quality and transmission efficiency, appropriate bitrate control algorithms are used to balance image quality and bitrate.

[0079] Rate control (RC) is a crucial technique in video coding, primarily referring to the process of controlling the video bitrate by adjusting the QP (Queries Per Second). A key aspect of RC is the RQ model, which describes the relationship between bitrate and QP. Several commonly used models include:

[0080] First-order model:

[0081] Second-order model:

[0082] R-γ model:

[0083] The first-order model is the simplest and is widely used in encoders such as x264 and OpenH264. In the formula, Q stands for Q-step, i.e., the quantization step size. R stands for Rate, representing the bit rate. Comp represents the image complexity, which is generally calculated using the mean absolute difference (MAD). α and β are coefficients in the model. The second-order model is an advanced version of the first-order model. λ is the Lagrange multiplier used in rate-distortion optimization, and λ is a coefficient in rate-distortion optimization.

[0084] The process of rate control based on the RQ model can be as follows: Figure 1As shown, the RQ model is first initialized, and then the target bitrate (which can be the maximum bitrate expected during encoding) is calculated. Next, the QP is calculated based on the target bitrate and the RQ model, and the calculated QP is used to encode a portion or the entire frame. Afterward, the RQ model is updated based on the actual output bitrate (mainly updating the parameters in the RQ model).

[0085] In the H.265 encoding standard commonly used in the current encoding and decoding field, the R-γ model is mainly used for rate control. It primarily selects a suitable QP value by balancing image quality and bit rate. Based on the R-γ model, three commonly used rate control modes have emerged: CBR, VBR, and AVBR. These three control methods calculate the quantization parameters based on the scene, the number of bits encoded, and other factors. According to the proposed RQ model, during encoding, the quantization parameters (QP) for subsequent encoding can be calculated based on the current number of bits encoded and the calculated λ value (determined through rate-distortion optimization). This QP is then used for encoding compression to achieve the purpose of rate control.

[0086] As mentioned earlier, CBR performs compression at a constant bitrate. VBR is a dynamic bitrate control mode, where the bitrate changes with the image quality; the bitrate is lower in static scenes and higher in dynamic scenes. AVBR, on the other hand, refers to variable bitrate control, an improved version of VBR that allows control over the maximum bitrate in static scenes by setting parameters.

[0087] Current mainstream encoding chips integrate these three bitrate control algorithms (CBR, VBR, AVBR). For example, most current security camera encoding protocols use VBR and CBR bitrate control algorithms, adjusting the bitrate based on the number of bits and quality of the video encoding. However, these three commonly used bitrate control modes only adjust the bitrate based on the dynamic and static scenes and the maximum bitrate, without considering the texture details of the image. This can lead to insufficient bitrate in areas with complex image textures, resulting in severe image quality loss such as ghosting and pixelation.

[0088] like Figure 2 The bitrate control scenario shown illustrates the effects of VBR and AVBR bitrate control modes in a security setting. Figure 2 In the scene shown in (a), at a resolution of 3 megapixels and a maximum bitrate of 1.5 Mbps, in scenes with complex details and many people, issues such as ghosting and pixelation appear within solid lines. Figure 2 In the scenario shown in (b), at a resolution of 4 million (W) and a maximum bitrate of 2 megabits (M), after enabling ROI encoding, the scene with a large number of people exhibits issues such as ghosting and mosaic within solid lines.

[0089] Based on this, this application provides an encoding method that, after obtaining the quantization parameters of image blocks in an image frame (obtained through a rate control algorithm), predicts image blocks that suffer image quality loss (motion causes changes in the image content at the same location), updates the quantization parameters for these image blocks, and then performs quantization encoding to obtain the bitstream. This achieves rate control that combines dynamic and static scenes with image texture, ensuring that the final quantization parameters take into account both dynamic and static scenes and image texture, thus guaranteeing image quality in locations with complex image textures.

[0090] The embodiments of this application will now be described with reference to the accompanying drawings.

[0091] Figure 3 This is a schematic diagram of the structure of an encoding / decoding system provided in this application. Figure 3 As shown, the encoding / decoding system 100 includes a source device 110 and a destination device 120. The source device 110 compresses and encodes video or images to obtain a bitstream, and transmits the bitstream to the destination device 120. The destination device 120 decodes the bitstream, reconstructs the video or image, and displays the reconstructed image.

[0092] Specifically, the source device 110 includes an image acquisition unit 111, a preprocessor 112, an encoder 113, and a communication interface 114.

[0093] Image acquisition device 111 is used to acquire raw images. Image acquisition device 111 may include or be any type of image capture device for, for example, capturing real-world images, and / or any type of image or commentary (for screen content encoding, some text on the screen is also considered as an image to be encoded or part of an image) generation device, such as a computer graphics processor for generating computer-animated images, or any type of device for acquiring and / or providing real-world images, computer-animated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). Image acquisition device 111 may be a camera for capturing images or a memory for storing images. Image acquisition device 111 may also include any type of (internal or external) interface for storing previously captured or generated images and / or acquiring or receiving images. When the image acquisition device 111 is a camera, it may be, for example, a local camera or an integrated camera integrated into the source device; when the image acquisition device 111 is a memory, it may be a local memory or an integrated memory integrated into the source device. When the image acquisition device 111 includes an interface, the interface may be, for example, an external interface for receiving images from an external video source, such as an external image capture device, like a camera, external memory, or an external image generation device, such as an external computer graphics processor, computer, or server. The interface may be any type of interface according to any proprietary or standardized interface protocol, such as a wired or wireless interface, or an optical interface.

[0094] An image can be viewed as a two-dimensional array or matrix of pixels (picture elements). Pixels in an array are also called sample points. The number of sample points in the array or image along the horizontal and vertical directions (or axes) defines the image's size and / or resolution. To represent color, three color components are typically used; that is, an image can be represented as or contain three sample arrays. For example, in RBG format or color space, an image includes corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in a luma / chroma format or color space. For example, for a YUV format image, this includes a luma component indicated by Y (sometimes also indicated by L) and two chroma components indicated by U and V. The luma component Y represents the brightness or grayscale level intensity (e.g., both are the same in a grayscale image), while the two chroma components U and V represent chroma or color information components. Accordingly, a YUV format image includes a luma sample array of luma sample values ​​(Y) and two chroma sample arrays of chroma values ​​(U and V). An RGB format image can be converted or transformed to YUV format, and vice versa; this process is also called color transformation or conversion. If the image is black and white, it may only include a luminance sampling array. In this application, the image transmitted from the image acquisition unit 111 to the encoder 113 can also be referred to as raw image data.

[0095] The preprocessor 112 receives the raw image acquired by the image acquisition unit 111 and preprocesses the raw image to obtain a preprocessed image. For example, the preprocessing performed by the preprocessor 112 includes retouching, color format conversion (e.g., from RGB format to YUV format), color adjustment, or noise reduction.

[0096] Encoder 113 receives the preprocessed image generated by preprocessor 112 and compresses and encodes the preprocessed image to obtain a bitstream. For example, encoder 113 may include a code control unit 1131 and an encoding unit 1132. Code control unit 1131 determines the quantization parameters used to encode each encoding unit in the current frame, so that encoding unit 1132 can predict, quantize, and encode the preprocessed image according to the quantization parameters to obtain a bitstream. Specifically, encoder 113 can determine the target number of bits based on the image content and the number of bits in the bitstream buffer, determine the quantization parameters based on the target number of bits, update the quantization parameters according to changes in the image content, and encode the image blocks.

[0097] The communication interface 114 is used to receive the bit stream generated by the encoder 113 and send the bit stream to the destination device 120 through the communication channel 130 so that the destination device 120 can reconstruct the original image based on the bit stream.

[0098] The target device 120 includes a display 121, a post-processor 122, a decoder 123, and a communication interface 124.

[0099] Communication interface 124 is used to receive the bitstream sent by communication interface 114 and transmit the bitstream to decoder 123 so that decoder 123 can reconstruct the original image based on the bitstream.

[0100] Communication interfaces 114 and 124 can be used to send or receive data related to the original image through a direct communication link between the source device 110 and the destination device 120, such as a direct wired or wireless connection, or through any type of network, such as a wired network, a wireless network or any combination thereof, any type of private network and public network or any combination thereof.

[0101] Both communication interface 114 and communication interface 124 can be configured as follows: Figure 1 The arrow pointing from the source device 110 to the corresponding communication channel 130 of the destination device 120 indicates a one-way or two-way communication interface, which can be used to send and receive messages, establish connections, acknowledge and exchange any other information related to the communication link and / or data transmission such as encoded bitstream transmission, etc.

[0102] Decoder 123 is used to decode the bitstream and reconstruct the original image. For example, decoder 123 performs entropy decoding, inverse quantization, and predictive reconstruction on the bitstream to obtain the reconstructed image. Decoder 123 may include a code control unit 1231 and a decoding unit 1232. The code control unit 1231 is used to determine the quantization parameters used for decoding each image block in the current frame, so that the decoding unit 1232 can decode, inverse quantize, and predictively reconstruct the bitstream according to the quantization parameters to obtain the reconstructed image.

[0103] The post-processor 122 is used to receive the reconstructed image generated by the decoder 123 and perform post-processing on the reconstructed image. For example, the post-processing performed by the post-processor 122 includes color format conversion (e.g., from YUV format to RGB format), color adjustment, retouching or resampling, or any other processing.

[0104] Display 121 is used to display the reconstructed image. Display 121 can be or can include any class of display devices for presenting the reconstructed image, such as integrated or external displays or monitors. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other class of display.

[0105] Both encoder 113 and decoder 123 can be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology is implemented in part in software, the device can store software instructions in a suitable non-transitory computer-readable storage medium, and one or more processors can be used to execute the instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) can be considered as one or more processors.

[0106] The image acquisition unit 111 and the encoder 113 can be integrated into a single physical device or located on different physical devices; there is no limitation on this. For example, such as... Figure 1The source device 110 shown includes an image acquisition unit 111 and an encoder 113, indicating that the image acquisition unit 111 and the encoder 113 are integrated into a single physical device. Therefore, the source device 110 can also be referred to as an acquisition device. The source device 110 can be, for example, a mobile phone, tablet computer, computer, laptop computer, camera, wearable device, in-vehicle device, terminal device, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, extended reality (XR) device, or other image acquisition device. If the source device 110 does not include the image acquisition unit 111, it means that the image acquisition unit 111 and the encoder 113 are two different physical devices, and the source device 110 can acquire raw images from other devices (such as image acquisition devices or image storage devices).

[0107] Furthermore, the display 121 and the decoder 123 can be integrated into a single physical device or located on different physical devices; there is no limitation on this. For example, such as... Figure 1 The destination device 120 shown includes a display 121 and a decoder 123, indicating that the display 121 and decoder 123 are integrated into a single physical device. Therefore, the destination device 120 can also be referred to as a playback device. The destination device 120 has the function of decoding and displaying the reconstructed image. The destination device 120 can be, for example, a monitor, television, digital media player, video game console, in-vehicle computer, or other image display device. If the destination device 120 does not include the display 121, it means that the display 121 and decoder 123 are two different physical devices. After the destination device 120 decodes the bitstream and reconstructs the original image, it transmits the reconstructed image to other display devices (such as a television or digital media player) for display.

[0108] also, Figure 3 It is shown that the source device 110 and the destination device 120 can be integrated into one physical device or set on different physical devices, without limitation.

[0109] For example, Figure 4 This illustrates an application scenario for the encoding method provided in this application. For example... Figure 4 As shown in (a), the source device 110 can be a camera, and the destination device 120 can be a display of various possible forms. The source device 110 can capture video of the first scene, transmit multiple frames of original images from the video to an encoding / decoding device, the encoding / decoding device performs encoding / decoding processing on the original images to obtain the reconstructed image, and the destination device 120 displays the reconstructed image and plays the video.

[0110] For example, such as Figure 4 As shown in (b), if the source device 110 and the destination device 120 are integrated into a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, or an extended reality (XR) device, then the VR / AR / MR / XR device has the functions of acquiring original images, displaying reconstructed images, and encoding / decoding. The source device 110 can acquire images of the real-world scene in which the user is located, and the destination device 120 can display reconstructed images of the real-world scene in a virtual environment.

[0111] In these embodiments, the source device 110 or its corresponding functions and the destination device 120 or its corresponding functions may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof. As described, Figure 1 The presence and division of different units or functions in the source device 110 and / or destination device 120 shown may vary depending on the actual device and application, which is obvious to those skilled in the art.

[0112] The structure of the above-described encoding / decoding system is only illustrative. In some possible implementations, the encoding / decoding system may also include other devices, such as end-side devices or cloud-side devices. After the source device 110 acquires the original image, it preprocesses the original image to obtain a preprocessed image; and then transmits the preprocessed image to the end-side device or cloud-side device, which performs encoding / decoding on the preprocessed image.

[0113] The image encoding and decoding method provided in this application is applied to both the encoding and decoding ends. Figure 5 The structure of the encoder and decoder is described in detail. For example... Figure 5 As shown, the encoder 300 includes a prediction unit 310, a code control unit 320, a quantization unit 330, an encoding unit 340, and a block partitioning unit 350.

[0114] The block division unit 350 is used to divide the original image into multiple image blocks.

[0115] The code control unit 320 is used to determine the quantization parameters based on the image content of the image block that needs to be encoded, output by the block division unit 350.

[0116] The prediction unit 310 performs intra-frame prediction / inter-frame prediction on the image blocks output by the block partitioning unit 350 to obtain the predicted bit count, and outputs the residual between the original bit count and the predicted bit count of the image block. For example, explanations of intra-frame prediction / inter-frame prediction can refer to the definitions in video compression standards (e.g., HEVC, H.265, or others), which will not be repeated here. Intra-frame prediction is a common method for removing spatial redundancy information in the original image. It uses reconstructed pixels from adjacent coded blocks as reference values ​​to predict the current image block. This is equivalent to the image block in the original image having a correlation with its surrounding coded blocks; the pixel value of the current image block can be estimated using the reconstructed image blocks from the surrounding blocks, and the estimated pixel value is the predicted value. Inter-frame prediction is a method for removing temporal redundancy information in the original image. It uses reconstructed pixels from coded blocks in neighboring image frames as reference values ​​to predict the current image block, obtaining the predicted value. Then, the residual between the predicted value and the original value of the current image block is quantized and entropy-coded. The encoded and transmitted value is often the prediction residual. The decoding end performs the same prediction process to obtain the predicted value of the current image block, and then adds the predicted value to the solved residual to obtain the reconstructed value of the current image block.

[0117] The quantization unit 330 is used to quantize the residual output by the prediction unit 310 according to the quantization parameters output by the code control unit 320 to obtain the quantized residual.

[0118] The encoding unit 340 is used to encode the quantized residual output by the quantization unit 330 to obtain the bitstream of the image block. For example, entropy encoding is performed on the quantized residual output by the quantization unit 330.

[0119] The code control unit 320 also provides the function of updating quantization parameters, that is, updating the quantization parameters of image blocks that meet the conditions.

[0120] Combination Figure 5 The structure of the decoder is described in detail. For example... Figure 5 As shown, the decoder 400 includes a decoding unit 410, a code control unit 420, an inverse quantization unit 430, and a prediction and reconstruction unit 440.

[0121] The decoding unit 410 is used to decode the bitstream of the image block to obtain the quantized residual and image content.

[0122] The code control unit 420 determines the quantization parameters based on or using the same quantization step size as the quantization unit 330 (the quantization step size used by the quantization unit 330 can be transmitted with the code stream).

[0123] The dequantization unit 430 is used to dequantize the quantized residual output by the decoding unit 410 according to the quantization parameters output by the code control unit 420 to obtain the residual.

[0124] The prediction and reconstruction unit 440 is used to predict and reconstruct the image based on the residual output by the inverse quantization unit 430 so that the reconstructed image can be displayed on the display.

[0125] Next, the image encoding process provided in this application will be explained with reference to the accompanying drawings.

[0126] Figure 6 This application provides a flowchart illustrating an encoding method, which can be executed by an encoding device. The encoding device can be... Figure 3 Zhongyuan Equipment 110, or Figure 5 The encoder 300 is described in the document. It should be noted that the encoding process is the same for different image frames. The following embodiments of this application only describe the encoding process of the encoding device for one image frame, and other processes will not be described in detail.

[0127] like Figure 6 As shown, the method includes the following steps.

[0128] S601, The encoding device acquires the quantization parameters of the image block in the image frame.

[0129] The size of the image block described in this application can be configured according to actual needs.

[0130] In one possible implementation, the aforementioned image block can be the size of the image region where quantization is performed during the encoding process.

[0131] For example, the above image block can be an LCU or a CTU. Of course, the size of the above image block can also be other. This application embodiment does not limit the size of the image block.

[0132] Specifically, the quantization parameters obtained in S601 are the quantization parameters of the image blocks determined according to the bit rate control algorithm.

[0133] S602, the encoding device updates the quantization parameters of image blocks in the image frame that meet the conditions, which indicate that the image content at the location of the image block has changed relative to the reference frame for inter-frame prediction.

[0134] The above conditions describe changes in image content in the time domain, increasing the probability of image quality loss. The quantization parameters obtained from the bitrate control algorithm can be updated to configure quantization parameters more suitable for the image content, ensuring image quality. The content of these conditions can be configured according to actual needs; this application does not limit this configuration.

[0135] One possible implementation is that the above conditions are used to describe the complexity of the texture details of the image.

[0136] In another possible implementation, the above conditions include: the position pointed to by the motion vector of the image block in the image frame that has undergone inter-frame prediction, and intra-frame prediction has been performed.

[0137] The position pointed to by the motion vector refers to the position of the image patch being referenced during inter-frame prediction within its respective image frame.

[0138] For example, suppose that image block A in the current image frame uses inter-frame prediction, and its reference is image block Z at position A (coordinates (x1, y1)) in the reference image frame. Image block Q at position A (coordinates (x1, y1)) in the current image block uses intra-frame prediction. Then, image block Q satisfies the condition.

[0139] Specifically, the implementation of updating the quantization parameters of image patches that meet the conditions includes, but is not limited to, the following two schemes:

[0140] Option 1: Update the quantization parameters of the image patches that meet the conditions to quantization parameters that are inversely proportional to the complexity.

[0141] In Scheme 1, the encoding device first calculates the complexity of image blocks that meet the conditions; then, it updates the quantization parameters of the image blocks that meet the conditions to quantization parameters that are inversely proportional to the complexity. This ensures that image blocks with higher complexity use the lowest possible quantization parameters, resulting in a larger bitstream for images with higher complexity and avoiding image quality loss due to insufficient bitstream.

[0142] For example, the encoding device records a mapping relationship between different complexities and quantization parameters. In this mapping relationship, the complexity is inversely proportional to its corresponding quantization parameter. For a first image block, which is any image block in an image frame that satisfies the conditions, the quantization parameter of the first image block is updated to a quantization parameter that is inversely proportional to the complexity. This includes updating the quantization parameter of the first image block to the quantization parameter corresponding to the complexity of the first image block in the mapping relationship.

[0143] The complexity can be texture complexity, variance mean, image gradient, or other parameters. This application does not limit the specific metrics for image complexity.

[0144] For example, the mean variance (Complex) of image patch L can be calculated in a way that satisfies the following expressions (1) and (2):

[0145]

[0146] Among them, L (x,y) Represents the pixel at position (x, y) of the image patch. Let r be the mean, and c be the height and width of the image patch, respectively.

[0147] Option 2: Assign image blocks that meet the criteria to the ROI.

[0148] In Scheme 2, the encoding device uses ROI encoding. For the second image block, which is any image block in the image frame that meets the conditions, the quantization parameters of the second image block in the image frame that meets the conditions are updated, including: updating the quantization parameters of the second image block to the quantization parameters of the third image block, which is the image block in the ROI region of the image frame that is closest to the second image block.

[0149] The quantization parameters of the third image block refer to the quantization parameters obtained in S601.

[0150] Furthermore, in S602, the quantization parameters are only updated for image blocks whose complexity is greater than or equal to the threshold. Specifically, S602 can be implemented as follows: update the quantization parameters of image blocks whose complexity is greater than or equal to the threshold among the image blocks that meet the conditions.

[0151] The threshold value is an empirically determined threshold for image quality loss, and its value can be configured according to actual needs. This application embodiment does not limit this value.

[0152] For example, the complexity is the variance mean, and the threshold can be configured to 6.

[0153] S603, The encoding device quantizes and encodes the image blocks in the image frame based on the quantization parameters to obtain the bitstream.

[0154] Specifically, the quantization parameters used by the encoding device in S603 are the latest quantization parameters of each image block in the image frame. That is, for image blocks whose quantization parameters have been updated, S603 performs quantization encoding based on the updated quantization parameters. For image blocks whose quantization parameters have not been updated in S602, S603 performs quantization encoding based on the quantization parameters obtained in S601.

[0155] The specific process of quantization encoding can be referenced from video compression standards, and this application embodiment does not limit it in this regard.

[0156] The encoding scheme provided in this application addresses the high probability of image quality loss phenomena such as motion blur and mosaic effects in image blocks whose image content has changed relative to the reference frame for inter-frame prediction. For these image blocks, the quantization parameters are updated to replace the initial quantization parameters provided in the encoding parameters. The updated quantization parameters are more suitable for the image block's image, ensuring image quality and avoiding image quality loss during the encoding process, thus improving image transmission quality. In other words, by combining dynamic and static scene control with bitstream control in the encoding parameters, and balancing quality and bitstream with image details, the bitstream is reduced as much as possible while ensuring image quality, thereby improving compression efficiency.

[0157] As described above, the solution provided in this application is an encoding / decoding technology based on an improved encoding standard. This technology can be integrated into traditional encoding / decoding frameworks as an enhancement. Furthermore, the encoding architecture still conforms to the original standard after incorporating this technology, and the decoder does not require any additional adaptation; the quantization parameters used are transmitted from the encoding end to the decoding end. Therefore, it can be applied to encoding / decoding protocols of standards such as H.265, H.264, AVS, and SVAC. The encoder architecture provided in the embodiments of this application can be as follows... Figure 7 As shown.

[0158] like Figure 7 As shown, the YUV data of an image frame, after being input into the encoder, undergoes inter-frame / intra-frame prediction to obtain MV information and intra-frame prediction information. Combining the intra-frame prediction information and MV information, image blocks that may experience image quality loss are predicted. Figure 7 (Illustrated as a motion blur image block) The QP is updated for image blocks that may suffer from image quality loss. After inter-frame / intra-frame prediction, the resulting residuals are first transformed, then quantized using the latest quantization parameters for each image block, and finally entropy coding is performed to output the bitstream.

[0159] The solution provided in this application primarily addresses issues such as mosaic and ghosting that may occur during the encoding process. The encoding technology incorporating this solution can be applied to scenarios requiring low bitrate encoding, such as carrier scenarios and wireless connection encoding / decoding scenarios. Of course, it can also be used in complex video surveillance scenarios to ensure image quality.

[0160] exist Figure 7 Based on the described encoder architecture, the principle and flow of the encoding method provided in this application are as follows: Figure 8 As shown, it includes the following three steps:

[0161] Step 1: Obtain MV information and intra-frame prediction information of the current image frame through inter-frame prediction information.

[0162] The motion vector can indicate the position it points to, for example... Figure 8 The middle arrow indicates the motion vector. Intra-frame prediction information indicates the location of image patches in the current image frame that have undergone inter-frame prediction, for example... Figure 8 The orange squares are shown in the intra-frame prediction information.

[0163] Step 2: Combine MV information and intra-frame prediction information to predict image blocks that will exhibit motion blur.

[0164] For example, assuming the above conditions are: the position pointed to by the motion vector of the image block that has undergone inter-frame prediction in the image frame, and intra-frame prediction has been performed, combining the MV information and intra-frame prediction information obtained in step 1, the image block that has undergone intra-frame prediction in the image block pointed to by the MV information is selected as the image block that will be predicted to have motion blur. Figure 8 Image patches with trailing shadows will appear in the image. These are image patches in red square areas. They are located at the positions pointed to by the motion vectors of the image patches that have undergone inter-frame prediction and intra-frame prediction.

[0165] Step 3: Calculate the complexity of image patches that will exhibit motion blur. For those exceeding the threshold, perform encoding quality protection.

[0166] In step 3, for image patches whose complexity is greater than or equal to the threshold and meet the conditions, encoding quality protection is performed, i.e., the quantization parameters are updated, such as... Figure 8 The image block shown is an example of an image block that has undergone encoding quality protection.

[0167] like Figure 9 As shown, this application provides another encoding method, which may specifically include:

[0168] S901. The YUV information of the Nth frame image is input into the encoder for prediction, and the MV information of inter-frame prediction and intra-frame prediction information are recorded.

[0169] S902. Determine the position of the reference block pointed to by the MV information, and determine whether the image block at the position of the reference block in the Nth frame has undergone intra-frame prediction.

[0170] For the reference block in the Nth image frame that is located at the reference block position and has undergone intra-frame prediction, execute S903; for the image block in the Nth image frame that is located at the reference block position but has not undergone intra-frame prediction, keep the quantization parameters unchanged.

[0171] S903. Calculate the image complexity of the image patch.

[0172] Specifically, in S903, the mean variance is calculated with reference to the aforementioned expressions (1) and (2).

[0173] S904. For image blocks with image complexity greater than or equal to the threshold, update the quantization parameters.

[0174] Image blocks with image complexity greater than or equal to a threshold can be marked as motion blur image blocks.

[0175] Specifically, in S904, for image blocks with image complexity greater than or equal to the threshold, the quantization parameters calculated by the rate control algorithm and Rdcost are no longer used; instead, the quantization parameters are updated.

[0176] For example, in S904, the quantized QP value can be obtained directly by looking up a table (looking up the mapping relationship) based on the calculated image complexity.

[0177] For example, the image complexity is the variance mean, and the quantization parameters included in the mapping relationship are absolute values. This mapping relationship can be shown in Table 1.

[0178] Table 1

[0179]

[0180] For example, the image complexity is the variance mean, and the quantization parameters included in the mapping relationship are relative values ​​to the quantization parameters in the encoding parameters. This mapping relationship can be shown in Table 2. The quantization parameters in the encoding parameters can be the quantization parameters calculated by the rate control algorithm.

[0181] Table 2

[0182] variance mean C The corresponding relative value of the quantization parameter ＜6 0 6＜C＜12 -2 12＜C＜28 -4 28＜C＜48 -6 48＜C＜68 -8 68＜C＜88 -10 88＜C＜108 -12 108＜C＜128 -13 ＞128 -14

[0183] It should be noted that Tables 1 and 2 are merely examples to illustrate the mapping relationship, and are not specific limitations on the content and form of the mapping relationship. In practical applications, the content and form of the mapping relationship can be configured according to actual needs.

[0184] For example, when the encoding device uses ROI encoding, in S904, the ROI area can be expanded, that is, the quantization parameters of the trailing image block determined in S903 can be updated to the quantization parameters of the image block in the ROI area. This realizes that during quantization, the trailing image block is treated as part of the ROI area for foreground protection, thereby achieving the purpose of quality protection.

[0185] For example, another principle flow of this application embodiment can be as follows: Figure 10 As shown, first combine the MV information ( Figure 10 The arrows in the image corresponding to the MV information correspond to the MV) and intra-frame prediction information ( Figure 10The orange boxes in the intra-frame prediction information indicate the image blocks that are predicted to be included in the intra-frame prediction. The image blocks that will appear to have motion blur are identified (predicted), and the complexity is calculated. Image blocks that will appear to have motion blur and whose complexity is greater than or equal to the threshold are included as part of the ROI region (essentially, the quantization parameters of the motion blur image blocks are updated to the quantization parameters of the image blocks in the ROI region), which expands the range of the ROI region during quantization.

[0186] S905. Quantize and encode the image blocks in the image frame based on their quantization parameters to obtain the bitstream.

[0187] It is understood that, in order to achieve the functions in the above embodiments, the encoder includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0188] The solution provided in this application is an enhancement of video encoding technology. The solution provided in this application can be used on front-end cameras, back-end servers or other products that use video encoding protocols. The video encoding protocols here can include commonly used H.264, H.265, AVS, SVAC, etc. or others.

[0189] The solution provided in this application (referred to as the protection strategy in the following embodiments) was verified in application scenarios with low bitrate configuration and ROI encoding, and described below.

[0190] Verification 1: The ROI encoding method is used, and the maximum bitrate is set to 2048.

[0191] Using only the ROI encoding method, the image is as follows Figure 11 In (a), details of localized quality loss are shown. Figure 11 (b) The method of overlaying the protection strategy provided in this application with ROI encoding results in an image such as... Figure 11 In (c), local details such as Figure 11 (d) in the middle. From Figure 11 As can be clearly seen in (b) and (d) in the paper, the method of superimposing the protection strategy provided in this application results in higher image quality and less loss.

[0192] Verification 2: The standard H.265 encoding method is used, the code control strategy is VBR, and the maximum code rate is set to 1536.

[0193] Using only the standard H.265 encoding method, the image is as follows Figure 12In (a), details of localized quality loss are shown. Figure 12 (b) The method of superimposing the protection strategy provided in this application using the ordinary H.265 encoding method, as shown in the image... Figure 12 In (c), local details such as Figure 12 (d) in the middle. From Figure 12 As can be clearly seen in (b) and (d) in the paper, the method of superimposing the protection strategy provided in this application results in higher image quality and less loss.

[0194] Analysis shows that using the ordinary H.265 encoding method with VBR code control strategy and a maximum bitrate of 1536, the data size is 1164KB. Under the same parameters, using the ordinary H.265 encoding method combined with the protection strategy provided in this application, with VBR code control strategy and a maximum bitrate of 1536, the data size is 1196KB. It can be seen that the protection strategy provided in this application can bring bitrate gains, with the bitrate increase resulting from adopting this protection strategy being approximately 2%.

[0195] Furthermore, after encoding is completed, the encoding end can carry the quantization parameter table (QP table) used in the actual encoding in the bitstream information transmitted to the decoding end. The QP table in the bitstream can be extracted using a bitstream analysis tool (such as StreamEye). In the quantization parameter table used in the actual encoding, the QP used by the image blocks that have undergone quality protection according to the scheme provided in this application is not affected by the bitrate control parameters.

[0196] Figure 13 This illustrates the quantization parameters carried by the coded image. Figure 13 The small yellow block in the middle can be used to represent the quantization parameters of the image block at that location using numbers.

[0197] The above text combines Figures 1 to 13 The encoding method provided according to this embodiment is described in detail below, and will be combined with Figure 14 and Figure 15 This describes the encoding apparatus provided according to this embodiment.

[0198] Figure 14 This is a schematic diagram of a possible encoding device provided in this embodiment. These encoding devices can be used to implement the functions of the encoder or encoding device in the above method embodiments, and therefore can also achieve the beneficial effects of the above method embodiments. In this embodiment, the encoding device can be as follows: Figure 5 The encoder 300 shown can also be a module (such as a chip) applied to terminal devices or servers.

[0199] like Figure 14 As shown, the encoding device 140 includes a communication module 1401 and an encoding module 1402.

[0200] Encoding device 140 is used to implement the above. Figure 6 or Figure 9 The encoder functionality is shown in the method embodiment.

[0201] The communication module 1401 is used to acquire quantization parameters of image blocks in an image frame. For example, the communication module 1401 is used to perform... Figure 4 or Figure 9 Step S601.

[0202] Encoding module 1402 is used to update the quantization parameters of image blocks that meet certain conditions in the image frame, and to quantize and encode the image blocks based on their quantization parameters to obtain a bitstream. For example, encoding module 1402 is used to perform... Figure 6 or Figure 9 Steps S602 and S603.

[0203] The communication module 1401 is also used to send code streams.

[0204] It should be understood that the encoding device 140 in this application embodiment can be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can also be implemented in software. Figure 6 or Figure 9 In the encoding method shown, the encoding device 140 and its various modules can also be software modules.

[0205] For a more detailed description of the aforementioned communication module and encoding module, please refer to [link / reference]. Figure 6 or Figure 9 The relevant descriptions in the method embodiments shown are directly obtained and will not be repeated here.

[0206] Figure 15 This is a schematic diagram of the structure of an encoder 150 provided in this embodiment. Figure 15 As shown, encoder 150 includes processor 1510, bus 1520, memory 1530 and communication interface 1540.

[0207] It should be understood that in this embodiment, the processor 1510 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0208] The processor may also be a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, or one or more integrated circuits used to control the execution of the program in this application.

[0209] The communication interface 1540 is used to enable communication between the encoder 150 and external devices or components.

[0210] Bus 1520 may include a pathway for transmitting information between the aforementioned components (such as processor 1510 and memory 1530). In addition to a data bus, bus 1520 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus 1520 in the figure.

[0211] As an example, encoder 150 may include multiple processors. A processor may be a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or computing units used to process data (e.g., computer program instructions).

[0212] It is worth noting that, Figure 15 Taking encoder 150 as an example, which includes one processor 1510 and one memory 1530, the processor 1510 and memory 1530 are used to indicate a type of device or equipment. In specific embodiments, the number of each type of device or equipment can be determined according to business needs.

[0213] The memory 1530 can correspond to the storage medium used to store information such as the reference grid in the above method embodiments, such as a disk, like a mechanical hard disk or a solid-state hard disk.

[0214] The encoder 150 described above can be a general-purpose device or a special-purpose device. For example, the encoder 150 can be an x86 or ARM-based server, or other special-purpose servers, such as a policy control and charging (PCC) server. This application does not limit the type of encoder 150.

[0215] It should be understood that the encoder 150 in this embodiment may correspond to the encoding device 140 in this embodiment, and may correspond to the execution according to Figure 6 or Figure 9 The corresponding subject in any of the methods, and the above and other operations and / or functions of each module in the encoding device 140 are respectively for implementing Figure 6 or Figure 9 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0216] Furthermore, embodiments of this application also provide a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the above-described... Figure 6 or Figure 9 An example of an encoding method.

[0217] Furthermore, embodiments of this application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the above-described... Figure 6 or Figure 9 An example of an encoding method.

[0218] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a network device or terminal device. Of course, the processor and storage medium can also exist as discrete components in the network device or terminal device.

[0219] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0220] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0221] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

[0222] In the description of this application, unless otherwise stated, " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B can mean A or B. "And / or" in this application is merely a description of the relationship between the related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. In the description of this application, unless otherwise stated, "multiple" means two or more. "At least one of the following or similar expressions" refers to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and / or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0223] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0224] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0225] It is understood that the term "embodiment" used throughout the specification means that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, throughout the specification, various embodiments do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0226] Some optional features in the embodiments of this application can be implemented independently in certain scenarios without relying on other features, such as the current underlying solution, to solve the corresponding technical problems and achieve the corresponding effects. Alternatively, they can be combined with other features as needed in other scenarios. Correspondingly, the apparatus given in the embodiments of this application can also implement these features or functions, which will not be elaborated upon here.

[0227] In this application, unless otherwise specified, the same or similar parts between the various embodiments can be referred to each other. In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments are consistent and can be mutually referenced. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. The following embodiments of this application do not constitute a limitation on the scope of protection of this application.

Claims

1. An encoding method characterized by comprising: Applied to an encoding device, the method comprises: obtaining a quantization parameter of an image block in an image frame; updating the quantization parameter of an image block in the image frame that meets a condition, the condition being used to indicate that image content at a position of the image block has changed relative to a reference frame of inter-frame prediction; quantizing encoding based on the quantization parameter of the image block in the image frame to obtain a code stream.

2. The method of claim 1, wherein, The condition comprises: a position pointed to by a motion vector of an image block in the image frame that has undergone inter-frame prediction, and the image block having undergone intra-frame prediction.

3. The method according to claim 1 or 2, characterized in that, The updating of the quantization parameter of the image block in the image frame that meets the condition comprises: calculating a complexity of the image block that meets the condition; updating the quantization parameter of the image block that meets the condition to a quantization parameter that is inversely proportional to the complexity.

4. The method of claim 3, wherein, The encoding device records a mapping relationship between different complexities and quantization parameters, and the updating of the quantization parameter of a first image block that is any image block in the image frame that meets the condition to a quantization parameter that is inversely proportional to the complexity comprises: updating the quantization parameter of the first image block to a quantization parameter corresponding to the complexity of the first image block in the mapping relationship.

5. The method of claim 4, wherein, The complexity is a variance mean, and the quantization parameter included in the mapping relationship is an absolute value; the mapping relationship is: 。 6. The method of claim 4, wherein, The complexity is a variance mean, and the quantization parameter included in the mapping relationship is a relative value relative to a quantization parameter in an encoding parameter; the mapping relationship is: 。 7. The method according to claim 1 or 2, characterized in that, The encoding device adopts region of interest (ROI) encoding, and the updating of the quantization parameter of a second image block that is any image block in the image frame that meets the condition comprises: updating the quantization parameter of the second image block to a quantization parameter of a third image block that is closest to the second image block in an ROI region of the image frame.

8. The method according to any one of claims 1 to 7, characterized in that, The updating of the quantization parameter of the image block in the image frame that meets the condition comprises: updating the quantization parameter of an image block in the image block that meets the condition and has a complexity greater than or equal to a threshold value.

9. The method according to any one of claims 1 to 8, characterized in that, The image block is a largest coding unit (LCU) or a coding tree unit (CTU).

10. An encoding apparatus, comprising: The device comprises: a communication module configured to obtain a quantization parameter of an image block in an image frame; an encoding module configured to update the quantization parameter of an image block in the image frame that meets a condition, the condition being used to indicate that image content at a position of the image block has changed relative to a reference frame of inter-frame prediction; and quantize encoding based on the quantization parameter of the image block in the image frame to obtain a code stream.

11. The apparatus of claim 10, wherein, The condition comprises: a position pointed to by a motion vector of an image block in the image frame that has undergone inter-frame prediction, and the image block having undergone intra-frame prediction.

12. The apparatus of claim 10 or 11, wherein, The encoding module is specifically configured to: calculate a complexity of the image block that meets the condition; update the quantization parameter of the image block that meets the condition to a quantization parameter that is inversely proportional to the complexity.

13. The apparatus of claim 12, wherein, The encoding device records a mapping relationship between different complexities and quantization parameters, and the encoding module is specifically configured to: update the quantization parameter of the first image block to a quantization parameter corresponding to the complexity of the first image block in the mapping relationship.

14. The apparatus of claim 13, wherein, The complexity is a variance mean, and the quantization parameter included in the mapping relationship is an absolute value; the mapping relationship is: 。 15. The apparatus of claim 13, wherein, The complexity is a variance mean, and the quantization parameter included in the mapping relationship is a relative value relative to a quantization parameter in an encoding parameter; the mapping relationship is:

16. The apparatus of claim 10 or 11, wherein, The encoding device adopts region of interest (ROI) encoding, and the encoding module is specifically configured to: update the quantization parameter of the second image block to a quantization parameter of a third image block, the third image block being an image block closest to the second image block in a ROI region of the image frame.

17. The apparatus of any of claims 10-16, wherein, The encoding module is specifically configured to: update the quantization parameter of an image block with a complexity greater than or equal to a threshold value in the image blocks satisfying the condition.

18. The apparatus of any of claims 10-17, wherein, The image block is a largest coding unit (LCU) or a coding tree unit (CTU).

19. An encoder comprising: The encoder includes at least one processor and a memory, wherein the memory is configured to store a computer program, so that the computer program is executed by the at least one processor to implement the method according to any one of claims 1-9.

20. A coding system characterized by The codec system includes the encoder according to claim 19.

21. A computer program product, characterised in that, The computer program product includes a computer program or instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1-9.

22. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions or programs, which, when executed on a computer, implement the method according to any one of claims 1-9.

Images