Image encoding and decoding method, device and storage medium

Abstract

The present application provides an image encoding and decoding method, device and storage medium, which relates to the field of image encoding and decoding. The method includes: obtaining the state of the code stream buffer; when the state of the code stream buffer meets the preset conditions, performing a first operation on the estimated number of bits of the current block to be encoded, or performing a second operation on the estimated quantization parameter of the current block to be encoded; the first operation refers to reducing the estimated number of bits by the number of penalty bits; the estimated quantization parameter is obtained based on the estimated number of bits; the second operation refers to increasing the estimated quantization parameter by a reference quantization parameter; the reference quantization parameter is obtained based on the number of penalty bits; based on the estimated number of bits after performing the first operation, or the estimated quantization parameter after performing the second operation, the current block to be encoded is encoded. The method is applicable to the image encoding and decoding process, and is used to solve the problem of code stream buffer overflow.

Description

[0001] This invention patent application is a divisional application of the Chinese invention patent application with application date of November 18, 2022, application number 202211448399.0, and name “Image Coding and Decoding Method, Device and Storage Medium”. Technical Field

[0002] The present invention relates to the field of image coding and decoding, and more particularly to an image coding and decoding method, device and storage medium. Background Art

[0003] In the high-speed video transmission interface scenario, hardware resources are limited and overly complex code control algorithms cannot be used. However, overly simple code control algorithms make it difficult to achieve lossless compression.

[0004] To solve this problem, the relevant information of the historical blocks can be used to control the bit rate of the current block. For example, taking the encoder as an example, the bit rate control information of the coded block at time t can be determined based on the bit stream buffer status and lossless coding bit number information after the coding of the coded block at time t-3 before time t.

[0005] However, using the relevant information of the historical blocks to control the bit rate of the current block may cause the state of the current code stream buffer to be unclear, thereby causing the code stream buffer to overflow. Summary of the invention

[0006] Based on the above technical problems, the present application provides an image encoding and decoding method, which can reduce the encoding bit rate by reducing the estimated number of bits or increasing the target quantization parameter when the number of bits remaining in the code stream buffer is small.

[0007] In a first aspect, the present application provides an image encoding method, the method comprising: obtaining the state of a code stream buffer; the code stream buffer is used to store the code stream obtained after encoding the block to be encoded; the state of the code stream buffer is the number of bits of the code stream buffer after encoding the Mth block to be encoded before the current block to be encoded; M is a positive integer; when the state of the code stream buffer meets a preset condition, performing a first operation on the estimated number of bits of the current block to be encoded; the estimated number of bits is the number of bits required to encode the current block to be encoded predicted under the state of the code stream buffer; the first operation refers to reducing the estimated number of bits by the number of penalty bits indicated by the penalty parameter; based on the estimated number of bits after performing the first operation, encoding the current block to be encoded.

[0008] Optionally, the code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0009] In a possible implementation, when the state of the code stream buffer satisfies a preset condition, a first operation is performed on the estimated number of bits of the current block to be encoded, including: determining an available buffer size based on the state of the code stream buffer; the available buffer size is used to characterize the available size of the code stream buffer after the current block to be encoded is predicted to be encoded using the estimated number of bits; obtaining an additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; when the available buffer size is less than the additional buffer threshold, the first operation is performed on the estimated number of bits of the current block to be encoded.

[0010] In a possible implementation, the penalty parameter includes a first penalty parameter; obtaining an additional buffer threshold includes: when the current block to be encoded is the end block of the strip where the current block to be encoded is located, determining the additional buffer threshold based on the difference between the initial size of the additional buffer and a reference product; the end block is a block to be encoded that needs to be filled with preset placeholder bits; the reference product is the product of a reference ranking and a first penalty parameter; the reference ranking is the ranking of the current block to be encoded in the end block; the first penalty parameter is the value by which the additional buffer threshold becomes smaller when one more block to be encoded is added to the reference ranking.

[0011] In another possible implementation, when the current block to be encoded is a block to be encoded other than the end block, the additional buffer threshold is the initial size of the additional buffer.

[0012] Optionally, before obtaining the additional buffer threshold, the method further includes: obtaining the number of tail blocks; and determining a first penalty parameter according to the initial size of the additional buffer and the number of tail blocks.

[0013] It should be understood that the end block is usually a number of to-be-encoded blocks (image blocks) that are at the end of the encoding order in a strip. The closer a strip is to encoding completion, the greater the reference ranking of the current to-be-encoded block in the end block, the greater the reference product, the smaller the additional buffer threshold, the less likely it is that the estimated number of bits of the current to-be-encoded block will be used to perform the first operation, and more encoding blocks will be filled into the additional buffer to make full use of encoding resources.

[0014] In a possible implementation manner, obtaining the additional buffer threshold includes: determining an initial size of the additional buffer as the additional buffer threshold.

[0015] In a possible implementation, determining the available buffer size based on the state of the code stream buffer, the estimated number of bits of the current block to be encoded, and the estimated number of bits of each of the M blocks to be encoded before the current block to be encoded includes: obtaining the estimated number of bits of the M blocks to be encoded before the current block to be encoded; determining the available buffer size according to the state of the code stream buffer and the sum of a reference change value; the reference change value is the difference between the fixed number of outgoing bits of the M+1 blocks of the code stream buffer and the estimated number of incoming bits of the M+1 blocks of the code stream buffer; the fixed number of outgoing bits of the M+1 blocks of the code stream buffer is the product of the number of pixels of each block to be encoded, a preset pixel depth, and M+1; the estimated number of incoming bits of the M+1 blocks of the code stream buffer is the sum of the estimated number of bits of the current block to be encoded and the estimated number of bits of each of the M blocks to be encoded before the current block to be encoded.

[0016] Optionally, for a historical block, the historical block is any one of the M blocks to be encoded before the current block to be encoded, and the estimated number of bits of the M blocks to be encoded before the current block to be encoded is obtained, including: obtaining the complexity level of the historical block and the complexity level of the current block to be encoded; determining the estimated number of bits of the historical block based on the quotient of the complexity level of the historical block and a reference value; the reference value is the product of the complexity level of the current block to be encoded and the estimated number of bits of the current block to be encoded.

[0017] In one possible implementation, the penalty parameter also includes a second penalty parameter; the second penalty parameter is the number of blocks to be encoded required to consume the number of bits in the additional buffer; before performing the first operation on the estimated number of bits of the current block to be encoded, the method also includes: obtaining the number of remaining blocks to be encoded in the strip where the current block to be encoded is located; if the number of remaining blocks to be encoded is less than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the current number of remaining blocks to be encoded and the preset pixel depth.

[0018] In another possible implementation, if the number of remaining blocks to be encoded is greater than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0019] In the image encoding method provided by the present application, when the state of the code stream buffer satisfies a preset condition, a first operation can be performed on the estimated number of bits of the current block to be encoded to reduce the predicted estimated number of bits for encoding the current block to be encoded. The estimated number of bits is inversely proportional to the quantization parameter, and the quantization parameter is inversely proportional to the encoding bit rate. Therefore, when the estimated number of bits is reduced, the bit rate when encoding the current block to be encoded is also reduced, thereby avoiding code stream buffer overflow caused by using the code stream buffer state of the historical block to control the encoding bit rate of the current block to be encoded in the high-speed video transmission interface.

[0020] In a second aspect, the present application provides an image encoding device, which includes various functional modules used for the method described in the first aspect above.

[0021] In a third aspect, the present application provides an image decoding method, the method comprising: obtaining the state of a code stream buffer; the code stream buffer is used to store the code stream after decoding the block to be decoded; the state of the code stream buffer is the number of bits of the code stream buffer after decoding the Mth block to be decoded before the current block to be decoded; M is a positive integer; when the state of the code stream buffer meets a preset condition, performing a first operation on the estimated number of bits of the current block to be decoded; the estimated number of bits is the number of bits required to decode the current block to be decoded predicted under the state of the code stream buffer; the first operation refers to reducing the estimated number of bits by the number of penalty bits indicated by the penalty parameter; based on the estimated number of bits obtained after performing the first operation, decoding the current block to be decoded.

[0022] Optionally, the code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0023] In a possible implementation, when the state of the code stream buffer satisfies a preset condition, a first operation is performed on the estimated number of bits of the current block to be decoded, including: determining an available buffer size based on the state of the code stream buffer; the available buffer size is used to characterize the available size of the code stream buffer after the current block to be decoded is decoded using the estimated number of bits; obtaining an additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; when the available buffer size is less than the additional buffer threshold, the first operation is performed on the estimated number of bits of the current block to be decoded.

[0024] In a possible implementation, the penalty parameter includes a first penalty parameter; obtaining an additional buffer threshold includes: when the current block to be decoded is the end block of the strip where the current block to be decoded is located, determining the additional buffer threshold according to the difference between the initial size of the additional buffer and the reference product; the end block is a block to be decoded filled with preset placeholder bits; the reference product is the product of the reference ranking and the first penalty parameter; the reference ranking is the ranking of the current block to be decoded in the end block; the first penalty parameter is the value by which the size of the additional buffer decreases when one block to be decoded is added to the reference ranking.

[0025] In another possible implementation, when the current block to be decoded is a decoded block other than the end block, the additional buffer threshold is the initial size of the additional buffer.

[0026] Optionally, before acquiring the additional buffer threshold, the method further includes: parsing a first penalty parameter from the bitstream stored in the bitstream buffer.

[0027] In a possible implementation manner, obtaining the additional buffer threshold includes: determining an initial size of the additional buffer as the additional buffer threshold.

[0028] In a possible implementation, determining the available buffer size based on the state of the code stream buffer includes: obtaining an estimated number of bits of M blocks to be decoded before a current block to be decoded; determining the available buffer size according to the state of the code stream buffer and the sum of a reference change value; the reference change value is a difference between a fixed number of inflow bits of M+1 blocks in the code stream buffer and an estimated number of outflow bits of M+1 blocks in the code stream buffer; the fixed number of outflow bits of M+1 blocks in the code stream buffer is a product of the number of pixels of each block to be decoded, a preset pixel depth, and M+1; the estimated number of inflow bits of M+1 blocks in the code stream buffer is a sum of an estimated number of bits of the current block to be decoded and the estimated number of bits of each of the M blocks to be decoded before the current block to be decoded.

[0029] Optionally, before determining the available buffer size according to the state of the code stream buffer and the sum of the reference change values, the method further includes: obtaining the complexity level of the historical block and the complexity level of the current block to be decoded; the historical block is any one of the M blocks to be decoded before the current block to be decoded; based on the quotient of the complexity level of the historical block and the reference value, determining the estimated number of bits of the historical block; the reference value is the product of the complexity level of the current block to be decoded and the estimated number of bits of the current block to be decoded.

[0030] In a possible implementation, the penalty parameter also includes a second penalty parameter; the second penalty parameter is the number of blocks to be decoded required to consume the number of bits in the additional buffer; before performing the first operation on the estimated number of bits of the current block to be decoded, the method also includes: parsing the second penalty parameter from the bitstream stored in the bitstream buffer; obtaining the number of remaining blocks to be decoded in the stripe where the current block to be decoded is located; if the number of remaining blocks to be decoded is less than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the number of currently remaining blocks to be decoded and a preset pixel depth.

[0031] In another possible implementation, if the number of blocks currently remaining to be decoded is greater than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0032] In a fourth aspect, the present application provides an image decoding device, which includes various functional modules used for the method described in the third aspect above.

[0033] In a fifth aspect, the present application provides an image encoding method, the method comprising: obtaining the state of a code stream buffer; the code stream buffer is used to store the code stream obtained after encoding the block to be encoded; the state of the code stream buffer is the number of bits after encoding the Mth block to be encoded in the code stream buffer before the current block to be encoded; M is a positive integer; when the state of the code stream buffer meets a preset condition, performing a second operation on the estimated quantization parameter of the current block to be encoded to obtain a target quantization parameter of the current block to be encoded; the estimated quantization parameter is obtained based on the estimated number of bits; the estimated number of bits is the number of bits required to encode the current block to be encoded predicted under the state of the code stream buffer; the second operation refers to adding a reference quantization parameter to the estimated quantization parameter; the reference quantization parameter is determined based on the number of penalty bits indicated by the penalty parameter; based on the target quantization parameter obtained after performing the second operation, encoding the current block to be encoded.

[0034] Optionally, the code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0035] Optionally, when the state of the code stream buffer satisfies a preset condition, a second operation is performed on the estimated quantization parameter of the current block to be encoded, including: determining the available buffer size based on the state of the code stream buffer; the available buffer size is used to characterize the available size of the code stream buffer after the current block to be encoded is encoded using the estimated number of bits predicted; obtaining an additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; when the size of the available buffer is less than the additional buffer threshold, the second operation is performed on the estimated quantization parameter of the current block to be encoded.

[0036] Optionally, the penalty parameter includes a first penalty parameter; obtaining an additional buffer threshold includes: when the current block to be encoded is the end block of the strip to be encoded, determining the additional buffer threshold based on the difference between the initial size of the additional buffer and the reference product; the end block is a block to be encoded that needs to be filled with preset placeholder bits; the reference product is the product of the reference ranking and the first penalty parameter; the reference ranking is the ranking of the current block to be encoded in the end block; the first penalty parameter is the value by which the additional buffer threshold becomes smaller for each additional block to be encoded in the reference ranking.

[0037] Optionally, when the current block to be encoded is a block to be encoded other than the end block, the additional buffer threshold is an initial size of the additional buffer.

[0038] Optionally, before obtaining the additional buffer threshold, the method further includes: obtaining the number of tail blocks; and determining a first penalty parameter according to the initial size of the additional buffer and the number of tail blocks.

[0039] Optionally, obtaining the additional buffer threshold includes: determining an initial size of the additional buffer as the additional buffer threshold.

[0040] Optionally, determining the available buffer size based on the state of the code stream buffer, the estimated number of bits of the current block to be encoded, and the estimated number of bits of each of the M blocks to be encoded before the current block to be encoded includes: obtaining the estimated number of bits of the M blocks to be encoded before the current block to be encoded; determining the available buffer size according to the state of the code stream buffer and the sum of a reference change value; the reference change value is a difference between a fixed number of outgoing bits of M+1 blocks in the code stream buffer and an estimated number of incoming bits of M+1 blocks in the code stream buffer; the fixed number of outgoing bits of the M+1 blocks in the code stream buffer is a product of the number of pixels of each block to be encoded, a preset pixel depth, and M+1; the estimated number of incoming bits of the M+1 blocks in the code stream buffer is the sum of the estimated number of bits of the current block to be encoded and the estimated number of bits of each of the M blocks to be encoded before the current block to be encoded.

[0041] Optionally, for a historical block, the historical block is any one of the M blocks to be encoded before the current block to be encoded, and the estimated number of bits of the M blocks to be encoded before the current block to be encoded is obtained, including: obtaining the complexity level of the historical block and the complexity level of the current block to be encoded; determining the estimated number of bits of the historical block based on the quotient of the complexity level of the historical block and a reference value; the reference value is the product of the complexity level of the current block to be encoded and the estimated number of bits of the current block to be encoded.

[0042] Optionally, the penalty parameter also includes a second penalty parameter; the second penalty parameter is the number of blocks to be encoded required to consume the number of bits in the additional buffer; before performing the second operation on the estimated quantization parameter of the current block to be encoded, the method also includes: obtaining the number of remaining blocks to be encoded in the strip where the current block to be encoded is located; if the number of remaining blocks to be encoded is less than the second penalty parameter, then in the second operation performed on the estimated quantization parameter of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the current remaining number of blocks to be encoded and a preset pixel depth.

[0043] Optionally, if the number of currently remaining blocks to be encoded is greater than the second penalty parameter, in the second operation performed on the estimated quantization parameter of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0044] In the image encoding method provided by the embodiment of the present application, when the state of the code stream buffer satisfies a preset condition, the encoding end can perform a second operation on the estimated quantization parameter of the current block to be encoded to obtain a target quantization parameter after adding the reference quantization parameter. The quantization parameter is inversely proportional to the encoding bit rate. Therefore, as the quantization parameter increases, the bit rate when encoding the current block to be encoded also decreases, thereby avoiding code stream buffer overflow caused by using the code stream buffer state of the historical block to control the encoding bit rate of the current block to be encoded in the high-speed video transmission interface.

[0045] In a sixth aspect, the present application provides an image encoding device, which includes various functional modules used for the method described in the fifth aspect above.

[0046] In a seventh aspect, the present application provides an image decoding method, the method comprising: obtaining the state of a code stream buffer; the code stream buffer is used to store the decoded code stream of a block to be decoded; the state of the code stream buffer is the number of bits of the code stream buffer after decoding the Mth block to be decoded before the current block to be decoded; M is a positive integer; when the state of the code stream buffer meets a preset condition, performing a second operation on an estimated quantization parameter of the current block to be decoded to obtain a target quantization parameter of the current block to be decoded; the estimated quantization parameter is obtained based on an estimated number of bits; the estimated number of bits is the number of bits required to decode the current block to be decoded predicted under the state of the code stream buffer; the second operation refers to adding a reference quantization parameter to the estimated quantization parameter; the reference quantization parameter is determined based on the number of penalty bits indicated by the penalty parameter; based on the target quantization parameter obtained after performing the second operation, the current block to be decoded is decoded.

[0047] Optionally, the code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0048] Optionally, when the state of the code stream buffer satisfies a preset condition, a second operation is performed on the estimated quantization parameter of the current block to be decoded, including: determining an available buffer size based on the state of the code stream buffer; the available buffer size is used to characterize the available size of the code stream buffer after the current block to be decoded is decoded using the estimated number of bits predicted; obtaining an additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; when the size of the available buffer is less than the additional buffer threshold, the second operation is performed on the estimated quantization parameter of the current block to be decoded.

[0049] Optionally, the penalty parameter includes a first penalty parameter; obtaining an additional buffer threshold includes: when the current block to be decoded is the end block of the strip to be decoded, determining the additional buffer threshold based on the difference between the initial size of the additional buffer and the reference product; the end block is a block to be decoded that needs to be filled with preset placeholder bits; the reference product is the product of the reference ranking and the first penalty parameter; the reference ranking is the ranking of the current block to be decoded in the end block; the first penalty parameter is the value by which the additional buffer threshold becomes smaller when one more block to be decoded is added to the reference ranking.

[0050] Optionally, when the current block to be decoded is a block to be decoded other than the end block, the additional buffer threshold is an initial size of the additional buffer.

[0051] Optionally, before acquiring the additional buffer threshold, the method further includes: parsing a first penalty parameter from the bitstream stored in the bitstream buffer.

[0052] Optionally, obtaining the additional buffer threshold includes: determining an initial size of the additional buffer as the additional buffer threshold.

[0053] Optionally, determining the available buffer size based on the state of the code stream buffer, the estimated number of bits of the current block to be decoded, and the estimated number of bits of each of the M blocks to be decoded before the current block to be decoded includes: obtaining the estimated number of bits of the M blocks to be decoded before the current block to be decoded; determining the available buffer size according to the state of the code stream buffer and the sum of a reference change value; the reference change value is a difference between a fixed number of inflow bits of M+1 blocks in the code stream buffer and an estimated number of outflow bits of M+1 blocks in the code stream buffer; the fixed number of outflow bits of M+1 blocks in the code stream buffer is a product of the number of pixels of each block to be decoded, a preset pixel depth, and M+1; the estimated number of inflow bits of M+1 blocks in the code stream buffer is the sum of the estimated number of bits of the current block to be decoded and the estimated number of bits of each of the M blocks to be decoded before the current block to be decoded.

[0054] Optionally, for a historical block, the historical block is any one of the M blocks to be decoded before the current block to be decoded, and the estimated number of bits of the M blocks to be decoded before the current block to be decoded is obtained, including: obtaining the complexity level of the historical block and the complexity level of the current block to be decoded; determining the estimated number of bits of the historical block based on the quotient of the complexity level of the historical block and a reference value; the reference value is the product of the complexity level of the current block to be decoded and the estimated number of bits of the current block to be decoded.

[0055] Optionally, the penalty parameter also includes a second penalty parameter; the second penalty parameter is the number of blocks to be decoded required to consume the number of bits in the additional buffer; before performing the second operation on the estimated quantization parameter of the current block to be decoded, the method also includes: parsing the second penalty parameter from the bitstream stored in the bitstream buffer; obtaining the number of remaining blocks to be decoded in the strip where the current block to be decoded is located; if the number of remaining blocks to be decoded is less than the second penalty parameter, then in the second operation performed on the estimated quantization parameter of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the current number of remaining blocks to be decoded and a preset pixel depth.

[0056] Optionally, if the number of currently remaining blocks to be decoded is greater than the second penalty parameter, in the second operation performed on the estimated quantization parameter of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0057] In an eighth aspect, the present application provides an image decoding device, which includes various functional modules used for the method described in the seventh aspect above.

[0058] In a ninth aspect, the present application provides an image encoding device, which includes a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the image encoding device implements the image encoding method described in the first and fifth aspects above.

[0059] In a tenth aspect, the present application provides an image decoding device, which includes a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the image decoding device implements the image decoding method described in the third and seventh aspects above.

[0060] In the eleventh aspect, the present application provides a computer program product. When the computer program product is run on an image coding and decoding device, the image coding and decoding device executes the steps of the related methods described in any one of the first aspect, the third aspect, the fifth aspect, and the seventh aspect, so as to implement the related methods described in any one of the first aspect, the third aspect, the fifth aspect, and the seventh aspect.

[0061] In the twelfth aspect, the present application provides a readable storage medium, which includes: computer program instructions; when the computer program instructions are executed in an image coding and decoding device, the image coding and decoding device implements the method described in any one of the first aspect, the third aspect, the fifth aspect, and the seventh aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 It is a flowchart of the code control solution;

[0064] Figure 2 A schematic diagram of the composition of a video encoding and decoding system provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of the composition of the video encoder 102 provided in an embodiment of the present application;

[0066] Figure 4 A schematic diagram of the structure of the video decoder 112 provided in an embodiment of the present application;

[0067] Figure 5 A schematic diagram of a video encoding and decoding process provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of the composition of an image encoding and decoding device provided in an embodiment of the present application;

[0069] Figure 7 A schematic diagram of a flow chart of an image encoding method provided in an embodiment of the present application;

[0070] Figure 8 A flowchart of an image decoding method provided in an embodiment of the present application;

[0071] Fig. 9 A schematic diagram of a flow chart of another image encoding method provided in an embodiment of the present application;

[0072] Fig.10 A schematic diagram of a flow chart of another image decoding method provided in an embodiment of the present application;

[0073] Fig.11 A schematic diagram of the composition of an image encoding device provided in an embodiment of the present application;

[0074] Fig.12 A schematic diagram of the composition of an image decoding device provided in an embodiment of the present application;

[0075] Fig.13 A schematic diagram of another image encoding device provided in an embodiment of the present application;

[0076] Fig.14 A schematic diagram of the composition of another image decoding device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] First, the terms involved in the embodiments of the present application are introduced.

[0078] 1. Video decoding technology:

[0079] Video decoding technology includes video encoding technology and video decoding technology, which can also be collectively referred to as video coding and decoding technology. Video sequences have a series of redundant information such as spatial redundancy, temporal redundancy, visual redundancy, information entropy redundancy, structural redundancy, knowledge redundancy, and importance redundancy. In order to remove redundant information in video sequences as much as possible and reduce the amount of data representing the video, video encoding technology is proposed to achieve the effect of reducing storage space and saving transmission bandwidth. Video encoding technology is also called video compression technology.

[0080] In order to obtain the data stored or transmitted based on the above video compression technology, video decoding technology is required to implement it accordingly.

[0081] Within the international scope, video compression coding standards are used to standardize video coding and decoding methods, such as: Advanced Video Coding (AVC) in Part 10 of the MPEG-2 and MPEG-4 standards developed by the Motion Picture Experts Group (MPEG), and H.263, H.264 and H.265 (also known as High Efficiency Video Coding standard, HEVC) developed by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T).

[0082] It should be noted that in a coding algorithm based on a hybrid coding architecture, the above compression coding methods can be used in a mixed manner.

[0083] The basic processing unit in the video encoding and decoding process is the image block, which is obtained by dividing a frame / image at the encoder. The divided image blocks are usually processed line by line. Taking HEVC as an example, HEVC defines Coding Tree Unit (CTU), Coding Unit (CU), Prediction Unit (PU) and Transform Unit (TU). CTU, CU, PU and TU can all be used as image blocks obtained after division. Among them, PU and TU are divided based on CU.

[0084] 2. Video sampling:

[0085] A pixel is the smallest complete sample of a video or image, so the data processing of an image block is done in units of pixels. Each pixel records color information. One sampling method is to represent the color through RGB, which includes three image channels, R represents red, G represents green, and B represents blue. Another sampling method is to represent the color through YUV, which includes three image channels, Y represents luminance, U represents the first chrominance Cb, and V represents the second chrominance Cr. Since people are more sensitive to brightness than to chrominance, storage space can be reduced by storing more brightness and less chrominance. Specifically, in video encoding and decoding, the YUV format is usually used for video sampling, including 420 sampling format, 422 sampling format, etc. This sampling format determines the number of samples of two chrominances based on the number of samples of brightness. For example, assuming that a CU has 4×2 pixels, the format is as follows:

[0086] [Y0,U0,V0][Y1,U1,V1][Y2,U2,V2][Y3,U3,V3];

[0087] [Y4, U4, V4] [Y5, U5, V5] [Y6, U6, V6] [Y7, U7, V7];

[0088] The 420 sampling format indicates that YUV is sampled in a 4:2:0 format, that is, the brightness and the first chroma or the second chroma are selected in a 4:2 ratio, where the first chroma and the second chroma are selected alternately. The above CU sampling selects the brightness Y0-Y3 of the first row, and the first chroma U0 and U2, and selects the brightness Y4-Y7 of the second row, and the second chroma V4 and V6. The CU is sampled and composed of a brightness coding unit and a chroma coding unit, where the brightness coding unit is:

[0089] [Y0][Y1][Y2][Y3];

[0090] [Y4][Y5][Y6][Y7];

[0091] The first chroma coding unit is:

[0092] [U0][U2];

[0093] The second chroma coding unit is:

[0094] [V4][V6];

[0095] It can be seen that the image block size after sampling in the above sampling format has changed. The luminance coding unit block size remains unchanged, still 4×2, while the first chroma coding unit block size becomes 2×1, and the second chroma coding unit block size also becomes 2×1. Therefore, if the CU size is assumed to be X×Y, the chroma coding unit block size after sampling based on the 420 sampling format is X / 2×Y / 2.

[0096] Similarly, the 422 sampling format indicates that YUV is sampled in a 4:2:2 format, that is, the brightness, the first chroma, and the second chroma are selected in a ratio of 4:2:2. Then the sampled brightness coding unit of the above CU is:

[0097] [Y0][Y1][Y2][Y3];

[0098] [Y4][Y5][Y6][Y7];

[0099] The first chroma coding unit is:

[0100] [U0][U2];

[0101] [U4][U6];

[0102] The second chroma coding unit is:

[0103] [V1][V3];

[0104] [V5][V7];

[0105] Among them, the luma coding unit block size remains unchanged, still 4×2, while the first chroma coding unit block size becomes 2×2, and the second chroma coding unit block size also becomes 2×2. Therefore, if the CU size is assumed to be X×Y, the chroma coding unit block size after sampling based on the 422 sampling format is X / 2×Y.

[0106] The luminance coding unit, the first chrominance coding unit and the second chrominance coding unit obtained through sampling are used as data units of each channel to be subsequently processed for the current block.

[0107] 3. Substream parallelism (or substream interleaving):

[0108] For the encoding end, substream parallelism means that the encoding end encodes the syntax elements of the coding blocks (CB) of different channels (such as the luminance channel, the first chrominance channel, and the second chrominance channel) of the coding unit (CU) using multiple entropy encoders to obtain multiple substreams, and interleaves the multiple substreams into a compressed bit stream with a fixed-size data packet. Correspondingly, for the decoding end, substream parallelism means that the decoding end uses different entropy decoders to decode different substreams in parallel.

[0109] 4. Other terms:

[0110] In the following, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second", etc. may explicitly or implicitly include one or more of the features.

[0111] In the high-speed video transmission interface scenario, hardware resources are limited and overly complex code control algorithms cannot be used. However, overly simple code control algorithms make it difficult to achieve lossless compression.

[0112] To solve this problem, the relevant information of the historical blocks can be used to control the bit rate of the current block. For example, taking the encoder as an example, the bit rate control information of the coded block at time t can be determined based on the bit stream buffer status and lossless coding bit number information after the coding of the coded block at time t-3 before time t.

[0113] For example, Figure 1 Figure 1 is a flow chart of the code control solution. Figure 1 As shown, the encoding end can obtain the state of the buffer and the complexity level of the current block to be encoded, determine the number of lossless coding bits according to the complexity level of the current block to be encoded, determine the estimated number of bits according to the number of lossless coding bits, clamp the estimated number of bits according to the state of the buffer and the complexity level, determine the main quantization parameter, brightness quantization parameter, and chrominance quantization parameter of the current block to be encoded according to the estimated number of bits, encode the current block to be encoded according to the main quantization parameter, brightness quantization parameter, and chrominance quantization parameter, and then estimate the number of lossless coding bits according to the brightness quantization parameter, the chrominance quantization parameter, and the actual number of encoded bits, and update the number of lossless coding bits.

[0114] However, using the relevant information of the historical blocks to control the bit rate of the current block may cause the state of the current code stream buffer to be unclear, thereby causing the code stream buffer to overflow.

[0115] In this case, the embodiments of the present application provide an image encoding and decoding method, device and storage medium, which can reduce the estimated number of bits or increase the quantization parameter when the state of the code stream buffer meets the preset conditions to reduce the encoding bit rate.

[0116] The following is an introduction with reference to the accompanying drawings.

[0117] Figure 2 The following is a schematic diagram of the composition of the video encoding and decoding system provided in the embodiment of the present application. Figure 2 As shown, the video coding and decoding system includes a source device 10 and a destination device 11 .

[0118] The source device 10 generates encoded video data, and the source device 10 may also be referred to as an encoding end, a video encoding end, a video encoding device, or a video encoding apparatus, etc. The destination device 11 may decode the encoded video data generated by the source device 10, and the destination device 11 may also be referred to as a decoding end, a video decoding end, a video decoding device, or a video decoding apparatus, etc. The source device 10 and / or the destination device 11 may include at least one processor and a memory coupled to the at least one processor. The memory may include but is not limited to a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures that can be accessed by a computer, and the embodiments of the present application do not specifically limit this.

[0119] The source device 10 and the destination device 11 may include a variety of devices, including electronic devices such as desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0120] The destination device 11 may receive the encoded video data from the source device 10 via the link 12. The link 12 may include one or more media and / or devices capable of moving the encoded video data from the source device 10 to the destination device 11. In one example, the link 12 may include one or more communication media that enable the source device 10 to transmit the encoded video data directly to the destination device 11 in real time. In this example, the source device 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and may transmit the modulated video data to the destination device 11. The one or more communication media may include wireless and / or wired communication media, such as a radio frequency (RF) spectrum, one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices that enable communication from the source device 10 to the destination device 11.

[0121] In another example, the source device 10 may output the encoded video data from the output interface 103 to the storage device 13. Similarly, the destination device 11 may access the encoded video data from the storage device 13 through the input interface 113. The storage device 13 may include a variety of locally accessible data storage media, such as a Blu-ray disc, a high-density digital video disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, or other suitable digital storage media for storing encoded video data.

[0122] In another example, the storage device 13 may correspond to a file server or another intermediate storage device storing the encoded video data generated by the source device 10. In this example, the destination device 11 may obtain the video data stored therein from the storage device 13 via streaming or downloading. The file server may be any type of server capable of storing the encoded video data and transmitting the encoded video data to the destination device 11. For example, the file server may include a World Wide Web (Web) server (e.g., for a website), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, and a local disk drive.

[0123] The destination device 11 can access the encoded video data through any standard data connection (e.g., an Internet connection). Example types of data connections include wireless channels suitable for accessing the encoded video data stored on a file server, wired connections (e.g., cable modems, etc.), or a combination of both. The encoded video data can be transmitted from the file server in the form of streaming, downloading, or a combination of both.

[0124] It should be noted that the image encoding and decoding method provided in the embodiment of the present application is not limited to wireless application scenarios.

[0125] Exemplarily, the image coding and decoding method provided in the embodiments of the present application can be applied to video coding and decoding supporting the following multimedia applications: over-the-air television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., via the Internet), encoding of video data stored on a data storage medium, decoding of video data stored on a data storage medium, or other applications. In some examples, the video coding and decoding system can be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting and / or video telephony.

[0126] It should be noted that Figure 2 The video coding and decoding system shown is only an example of a video coding and decoding system, and is not a limitation of the video coding and decoding system in the present application. The image coding and decoding method provided in the present application can also be applied to the scenario of wireless data communication between the encoding device and the decoding device. In other embodiments, the video data to be decoded or the encoded video data can be retrieved from a local memory, or can be streamed on a network, etc. The video encoding device can encode the video data to be encoded and store the encoded video data in a memory, and the video decoding device can also obtain the encoded video data from the memory and decode the encoded video data.

[0127] exist Figure 2 In an embodiment of the present invention, source device 10 includes a video source 101, a video encoder 102, and an output interface 103. In some embodiments, output interface 103 may include a modulator / demodulator (modem) and / or a transmitter. Video source 101 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video input interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources of video data.

[0128] The video encoder 102 may encode the video data from the video source 101. In some embodiments, the source device 10 transmits the encoded video data directly to the destination device 11 via the output interface 103. In other embodiments, the encoded video data may also be stored on the storage device 13 for later access by the destination device 11 for decoding and / or playback.

[0129] exist Figure 2 In an embodiment of the present invention, the destination device 11 includes a display device 111, a video decoder 112, and an input interface 113. In some embodiments, the input interface 113 includes a receiver and / or a modem. The input interface 113 can receive the encoded video data via the link 12 and / or from the storage device 13. The display device 111 can be integrated with the destination device 11 or can be external to the destination device 11. In general, the display device 111 displays the decoded video data. The display device 111 can include a variety of display devices, such as a liquid crystal display, a plasma display, an organic light emitting diode display, or other types of display devices.

[0130] Alternatively, the video encoder 102 and the video decoder 112 may each be integrated with an audio encoder and decoder, and may include appropriate multiplexer-demultiplexer units or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams.

[0131] The video encoder 102 and the video decoder 112 may include at least one microprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. If the decoding method provided in the present application is implemented in software, the instructions for the software may be stored in a suitable non-volatile computer-readable storage medium, and at least one processor may be used to execute the instructions to implement the present application.

[0132] The video encoder 102 and the video decoder 112 in the present application may operate according to a video compression standard (eg, HEVC), or may operate according to other industry standards, which is not specifically limited in the present application.

[0133] Figure 3 FIG. 1 is a schematic diagram of the composition of the video encoder 102 provided in an embodiment of the present application. Figure 3As shown, the video encoder 102 can perform prediction, transformation, quantization, and entropy coding processes in the prediction module 21, the transformation module 22, the quantization module 23, and the entropy coding module 24, respectively. The video encoder 102 also includes a pre-processing module 20 and a summer 202, wherein the pre-processing module 20 includes a segmentation module and a bit rate control module. For video block reconstruction, the video encoder 102 can also include an inverse quantization module 25, an inverse transformation module 26, a summer 201, and a reference image memory 27.

[0134] like Figure 3 As shown, the video encoder 102 receives video data, and the preprocessing module 20 receives input parameters of the video data. The input parameters include information such as the resolution of the image in the video data, the sampling format of the image, the pixel depth (bits perpixel, BPP), and the bit width (or it can also be called the image bit width). BPP refers to the number of bits occupied by a unit pixel. The bit width refers to the number of bits occupied by a pixel channel in a unit pixel. For example, a pixel is represented by the value of three YUV pixel channels. If each pixel channel occupies 8 bits (bits), the bit width of the pixel is 8, and the BPP of the pixel is 3×8=24 bits.

[0135] The segmentation module in the preprocessing module 20 divides the image into original blocks (or may also be referred to as coding units (CU)). The original block (or may also be referred to as coding unit (CU)) may include coding blocks of multiple channels. For example, the multiple channels may be RGB channels or YUV channels, etc. The embodiment of the present application is not limited to this. Optionally, this segmentation may also include segmentation into slices, image blocks or other larger units, and segmentation of video blocks according to the quadtree structure of the largest coding unit (LCU) and CU. Exemplarily, the video encoder 102 encodes the components of the video blocks in the video slice to be encoded. Generally, the slice can be divided into multiple original blocks (and may be divided into a set of original blocks called image blocks). The sizes of CU, PU and TU are usually determined in the segmentation module. In addition, the segmentation module is also used to determine the size of the rate control unit. The rate control unit refers to a basic processing unit in the rate control module, for example, the rate control module calculates complexity information for the original block based on the rate control unit, and then calculates the quantization parameter of the original block based on the complexity information. Among them, the segmentation strategy of the segmentation module can be preset or continuously adjusted based on the image during the encoding process. When the segmentation strategy is a preset strategy, the same segmentation strategy is also preset in the decoding end accordingly, so as to obtain the same image processing unit. The image processing unit is any of the above-mentioned image blocks and corresponds one-to-one with the encoding side. When the segmentation strategy is continuously adjusted based on the image during the encoding process, the segmentation strategy can be directly or indirectly encoded into the bitstream, and accordingly, the decoding end obtains the corresponding parameters from the bitstream, obtains the same segmentation strategy, and obtains the same image processing unit.

[0136] The rate control module in the preprocessing module 20 is used to generate quantization parameters so that the quantization module 23 and the inverse quantization module 25 perform related calculations. In the process of calculating the quantization parameters, the rate control module can obtain the image information of the original block for calculation, such as the above-mentioned input information; it can also obtain the reconstructed value obtained by the summer 201 after reconstruction for calculation, and this application does not limit this.

[0137] The prediction module 21 may provide the prediction block to the summer 202 to generate a residual block, and provide the prediction block to the summer 201 for reconstruction to obtain a reconstructed block, which is used as a reference pixel for subsequent prediction. The video encoder 102 forms a pixel difference by subtracting the pixel value of the prediction block from the pixel value of the original block. The pixel difference is the residual block, and the data in the residual block may include brightness difference and chrominance difference. The summer 201 represents one or more components that perform this subtraction operation. The prediction module 21 may also send related syntax elements to the entropy coding module 24 for merging into the bitstream.

[0138] The transform module 22 may divide the residual block into one or more TUs for transform. The transform module 22 may convert the residual block from the pixel domain to the transform domain (e.g., the frequency domain). For example, the residual block may be transformed using a discrete cosine transform (DCT) or a discrete sine transform (DST) to obtain a transform coefficient. The transform module 22 may send the obtained transform coefficient to the quantization module 23.

[0139] The quantization module 23 may perform quantization based on the quantization unit. The quantization unit may be the same as the CU, TU, and PU described above, or may be further divided in the segmentation module. The quantization module 23 quantizes the transform coefficients to further reduce the bit rate to obtain quantized coefficients. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameter. In some feasible embodiments, the quantization module 23 may then perform a scan of the matrix containing the quantized transform coefficients. Alternatively, the entropy coding module 24 may perform the scan.

[0140] After quantization, the entropy coding module 24 may entropy encode the quantized coefficients. For example, the entropy coding module 24 may perform context-adaptive variable-length coding (CAVLC), context-based adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic decoding (SBAC), probability interval segmentation entropy (PIPE) decoding, or another entropy coding method or technique. After entropy encoding by the entropy coding module 24, a substream is obtained, and after entropy encoding by multiple entropy coding modules 24, a bitstream can be obtained, which is transmitted to the video decoder 112 or archived for later transmission or retrieval by the video decoder 112.

[0141] The inverse quantization module 25 and the inverse transform module 26 apply inverse quantization and inverse transform respectively, and the summer 201 adds the inverse transformed residual block to the predicted residual block to generate a reconstructed block, which is used as a reference pixel for subsequent prediction of the original block. The reconstructed block is stored in the reference image memory 27.

[0142] Figure 4 FIG. 1 is a schematic diagram of the structure of the video decoder 112 provided in the embodiment of the present application. Figure 4 As shown, the video decoder 112 includes an entropy decoding module 30 , a prediction module 31 , an inverse quantization module 32 , an inverse transform module 33 , a summer 301 and a reference image memory 34 .

[0143] The entropy decoding module 30 includes a parsing module and a bit rate control module. In some feasible implementations, the video decoder 112 may perform Figure 3 The decoding process of the video encoder 102 is an exemplary reciprocal of the encoding process described.

[0144] During the decoding process, the video decoder 112 receives a bitstream of the encoded video from the video encoder 102. The parsing module in the entropy decoding module 30 of the video decoder 112 may entropy decode the bitstream to generate quantized coefficients and syntax elements. The entropy decoding module 30 may pass the syntax elements to the prediction module 31. The video decoder 112 may receive the syntax elements at the video slice layer and / or the video block layer.

[0145] The rate control module in the entropy decoding module 30 generates quantization parameters based on the information of the image to be decoded obtained by the analysis module so that the inverse quantization module 32 can perform relevant calculations. The rate control module can also calculate the quantization parameters based on the reconstructed block obtained by the summer 301 through reconstruction.

[0146] The inverse quantization module 32 inversely quantizes (e.g., dequantizes) the quantization coefficients and the generated quantization parameters provided in the bitstream and decoded by the entropy decoding module 30. The inverse quantization process may include determining the degree of quantization using the quantization parameters calculated by the video encoder 102 for each video block in the video slice, and similarly determining the degree of inverse quantization to be applied. The inverse transform module 33 applies an inverse transform (e.g., a transform method such as DCT, DST, etc.) to the inversely quantized transform coefficients, and generates an inversely transformed residual block in the pixel domain from the inversely quantized transform coefficients according to an inverse transform unit. The size of the inverse transform unit is the same as the size of the TU, and the inverse transform method and the transform method use the corresponding forward transform and inverse transform in the same transform method, for example, the inverse transform of DCT, DST is an inverse DCT, an inverse DST, or a conceptually similar inverse transform process.

[0147] After the prediction module 31 generates the prediction block, the video decoder 112 forms a decoded video block by summing the inverse transformed residual block from the inverse transform module 33 with the prediction block. The summer 301 represents one or more components that perform this summing operation. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded image blocks in a given frame or image are stored in the reference image memory 34 as reference pixels for subsequent prediction.

[0148] The present application embodiment provides a possible implementation method of video (image) encoding and decoding, see Figure 5 , Figure 5A flow chart of a video encoding and decoding process provided in an embodiment of the present application, wherein the video encoding and decoding implementation method includes process ① to process ⑤, and process ① to process ⑤ can be executed by any one or more of the above-mentioned source device 10, video encoder 102, destination device 11 or video decoder 112.

[0149] Process ①: Divide a frame of image into one or more non-overlapping parallel coding units. The one or more parallel coding units have no dependencies and can be completely parallel / independently encoded and decoded, such as Figure 5 A parallel encoding unit 1 and a parallel encoding unit 2 are shown.

[0150] Process ②: Each parallel coding unit can be further divided into one or more non-overlapping independent coding units. The independent coding units may not depend on each other, but may share some parallel coding unit header information.

[0151] The independent coding unit may include three channels of brightness Y, first chromaticity Cb, and second chromaticity Cr, or three channels of RGB, or may include only one of the channels. If the independent coding unit includes three channels, the sizes of the three channels may be exactly the same or different, which is related to the input format of the image. The independent coding unit can also be understood as one or more processing units formed by the N channels contained in each parallel coding unit. For example, the three channels Y, Cb, and Cr mentioned above are the three channels constituting the parallel coding unit, which can be an independent coding unit respectively, or Cb and Cr can be collectively referred to as chromaticity channels, then the parallel coding unit includes an independent coding unit consisting of a brightness channel, and an independent coding unit consisting of a chromaticity channel.

[0152] Process ③: Each independent coding unit can be further divided into one or more non-overlapping coding units. The coding units within the independent coding unit can be interdependent, such as multiple coding units can be pre-encoded and decoded with reference to each other.

[0153] If the coding unit and the independent coding unit have the same size (ie, the independent coding unit is divided into only one coding unit), the size thereof can be any size described in process ②.

[0154] The coding unit may include three channels (or three channels of RGB) of brightness Y, first chrominance Cb, and second chrominance Cr, or only one of the channels. If three channels are included, the sizes of the channels may be exactly the same or different, depending on the image input format.

[0155] It is worth noting that process ③ is an optional step in the video coding and decoding method, and the video codec can encode / decode the residual coefficients (or residual values) of the independent coding units obtained in process ②.

[0156] Process ④: For the coding unit, it can be further divided into one or more non-overlapping prediction groups (prediction groups, PG), PG can also be referred to as Group, each PG is encoded and decoded according to the selected prediction mode to obtain the prediction value of PG, which constitutes the prediction value of the entire coding unit, and the residual value of the coding unit is obtained based on the prediction value and the original value of the coding unit.

[0157] Process ⑤: Based on the residual values ​​of the coding units, the coding units are grouped to obtain one or more non-overlapping residual blocks (RBs). The residual coefficients of each RB are encoded and decoded according to the selected mode to form a residual coefficient stream. Specifically, it can be divided into two categories: transforming the residual coefficients and not transforming them.

[0158] Among them, the selected mode of the residual coefficient encoding and decoding method in process ⑤ may include, but is not limited to any of the following: semi-fixed length coding method, exponential Golomb coding method, Golomb-Rice coding method, truncated unary code coding method, run-length coding method, direct encoding of original residual value, etc.

[0159] For example, a video encoder may directly encode coefficients within an RB.

[0160] For another example, the video encoder may also transform the residual block, such as DCT, DST, Hadamard transform, etc., and then encode the transformed coefficients.

[0161] As a possible example, when RB is small, the video encoder can directly perform uniform quantization on each coefficient in RB, and then perform binary encoding. If RB is large, it can be further divided into multiple coefficient groups (coefficient groups, CG), and then each CG is uniformly quantized and then binary encoded. In some embodiments of the present application, the coefficient group (CG) and the quantization group (QG) can be the same.

[0162] The following is an example of the residual coefficient encoding part using a semi-fixed length coding method. First, the maximum value of the absolute value of the residual in an RB block is defined as the modified maximum value (modified maximum, mm). Secondly, the number of coding bits for the residual coefficient in the RB block is determined (the number of coding bits for the residual coefficient in the same RB block is consistent). For example, if the critical limit (CL) of the current RB block is 2 and the current residual coefficient is 1, then 2 bits are required to encode the residual coefficient 1, which is expressed as 01. If the CL of the current RB block is 7, it means encoding an 8-bit residual coefficient and a 1-bit sign bit. The determination of CL is to find the minimum M value that satisfies that all residuals of the current sub-block are within the range of [-2 ^(M-1), 2 ^(M-1)]. If both -2^(M-1) and 2^(M-1) exist at the same time, M should be increased by 1, that is, M+1 bits are required to encode all residuals of the current RB block; if only one of the two boundary values ​​​​of -2^(M-1) and 2^(M-1) exists, a Trailing bit needs to be encoded to determine whether the boundary value is -2^(M-1) or 2^(M-1); if all residuals do not exist in either -2^(M-1) or 2^(M-1), there is no need to encode the Trailing bit.

[0163] In addition, for some special cases, the video encoder can also directly encode the original value of the image instead of the residual value.

[0164] The video encoder 102 and the video decoder 112 may also be implemented in another implementation form, for example, by using a general digital processing system, see Figure 6 , Figure 6 The following is a schematic diagram of the composition of the image coding and decoding device provided in the embodiment of the present application. The image coding and decoding device can be applied to the encoding side (or encoding end) or the decoding side (or decoding end). Figure 6 As shown, the image coding and decoding device includes a processor 41 and a memory 42. The processor 41 is connected to the memory 42 (for example, connected to each other through a bus 43). Optionally, the coding and decoding device may also include a communication interface 44, which connects the processor 41 and the memory 42 and is used to receive / send data.

[0165] The processor 41 is used to execute the instructions stored in the memory 42 to implement the image encoding and decoding method provided in the following embodiments of the present application. The processor 41 can be a central processing unit (CPU), a general-purpose processor network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 41 can also be any other device with processing functions, such as a circuit, a device, or a software module, which is not limited in the embodiments of the present application. In one example, the processor 41 may include one or more CPUs, such as Figure 6 As an optional implementation, the electronic device may include multiple processors, for example, in addition to the processor 41, it may also include a processor 45 ( Figure 6 The dashed line is used as an example.

[0166] Memory 42 is used to store instructions. For example, the instructions may be computer programs. Optionally, the memory 42 may be a read-only memory (ROM) or other types of static storage devices that can store static information and / or instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and / or instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, etc., and the embodiments of the present application are not limited to this.

[0167] It should be noted that the memory 42 may exist independently of the processor 41, or may be integrated with the processor 41. The memory 42 may be located in the image coding and decoding device, or may be located outside the image coding and decoding device, which is not limited in the embodiment of the present application.

[0168] The bus 43 is used to transmit information between the various components included in the codec device. The bus 43 can be an industry standard architecture (ISA) line, a peripheral component interconnect (PCI) line, or an extended industry standard architecture (EISA) line. The bus 43 can be divided into an address line, a data line, a control line, etc. For ease of representation, Figure 6 Although only one solid line is used in the diagram, it does not mean that there is only one line or one type of line.

[0169] The communication interface 44 is used to communicate with other devices or other communication networks. The other communication networks may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 44 may be a module, a circuit, a transceiver, or any device capable of achieving communication. The embodiments of the present application are not limited to this.

[0170] It should be noted that Figure 6 The structure shown in the figure does not constitute a limitation on the image encoding and decoding device, except Figure 6 In addition to the components shown, the image coding and decoding apparatus may include more or fewer components than those shown in the figure, or a combination of certain components, or a different arrangement of components.

[0171] The execution subject of the image coding and decoding method provided in the embodiment of the present application may be the above-mentioned image coding and decoding device, or an application (application, APP) installed in the image coding and decoding device that provides image coding and decoding functions; or a CPU in the image coding and decoding device; or a functional module in the image coding and decoding device for executing the image coding and decoding method. The embodiment of the present application does not limit this. For the sake of simplicity of description, the following description is uniformly taken as an example of an image coding and decoding device as an encoding end or a decoding end.

[0172] The image encoding and decoding method provided by the embodiment of the present application is introduced below with reference to the accompanying drawings.

[0173] Figure 7 The following is a flow chart of an image encoding method provided in an embodiment of the present application. Figure 7 As shown, the image encoding method includes S101 to S203.

[0174] S101: The encoder obtains the status of the code stream buffer.

[0175] The code stream buffer is used to store the code stream obtained after the block to be coded is coded. The state of the code stream buffer is the number of bits in the code stream buffer after the Mth block to be coded before the current block to be coded, where M is the number of delay blocks mentioned above and M is a positive integer. The number of bits in the code stream buffer at a certain moment refers to the difference between the number of bits flowing into the code stream buffer and the number of bits flowing out of the code stream buffer up to that moment.

[0176] For example, assuming that by the time the encoding of the Mth block to be encoded before the current block to be encoded is completed, 100 bits flow into the code stream buffer and 80 bits flow out of the code stream buffer, then the state change amount of the code stream buffer at the time when the encoding of the Mth block to be encoded before the current block to be encoded obtained at the current time is completed is 100-80=20 bits, assuming that the state accumulation amount of the code stream buffer before the Mth block is 20 bits, then the state of the code stream buffer at the time when the encoding of the Mth block to be encoded is completed is the sum of the accumulation amount and the change amount 20+20=40 bits.

[0177] In a possible implementation, the code stream buffer may include a main buffer and an additional buffer. Both the main buffer and the additional buffer are used to store the code stream obtained after encoding the block to be encoded.

[0178] Optionally, the main buffer and the additional buffer may be different physical storage partitions.

[0179] For example, the main buffer may be set in storage device 1 , and the additional buffer may be set in storage device 2 .

[0180] Optionally, the main buffer and the additional buffer may also be different logical storage partitions.

[0181] For example, both the main buffer and the additional buffer may be set in the storage device 1 , that is, the main buffer and the additional buffer are different logical storage partitions in the storage device 1 .

[0182] S102: When the state of the code stream buffer satisfies a preset condition, the encoder performs a first operation on the estimated number of bits of the current block to be encoded.

[0183] The estimated number of bits is the number of bits required for encoding the current block to be encoded predicted in the state of the code stream buffer. The first operation is to reduce the estimated number of bits by the number of penalty bits indicated by the penalty parameter. The number of penalty bits indicated by the penalty parameter can be referred to in the following specific implementation process, which will not be repeated here.

[0184] Optionally, the preset condition may be that the available buffer size determined based on the state of the code stream buffer is smaller than the available space of the additional buffer, that is, the encoding process uses the space of the additional buffer. In this case, the above S102 may specifically include the following three steps:

[0185] Step 1: The encoder determines the available buffer size based on the status of the code stream buffer.

[0186] The available buffer size is used to represent the available size of the code stream buffer after the current block to be encoded is encoded using the estimated number of bits.

[0187] For example, the encoder may determine the available buffer size based on the state of the code stream buffer, the estimated number of bits of the current block to be encoded, and the estimated number of bits of each of the M blocks to be encoded before the current block to be encoded.

[0188] In a possible implementation, the above step 1 may specifically include: the encoder obtains the estimated number of bits of M blocks to be encoded before the current block to be encoded; the encoder determines the available buffer size according to the sum of the bitstream buffer and the reference change value.

[0189] The reference change value is the difference between the fixed outgoing bit number of the M+1 blocks in the code stream buffer and the estimated incoming bit number of the M+1 blocks in the code stream buffer. The fixed outgoing bit number of the M+1 blocks in the code stream buffer is the product of the number of pixels of each block to be encoded, the preset pixel depth, and M+1. The estimated incoming bit number of the M+1 blocks in the code stream buffer is the sum of the estimated bit number of the current block to be encoded and the estimated bit number of each of the M blocks to be encoded before the current block to be encoded.

[0190] Optionally, for any historical block, which is any one of the M blocks to be encoded before the block to be encoded, the encoding end may directly obtain the estimated number of bits obtained when encoding the historical block.

[0191] Optionally, for any one of the historical blocks, the encoding end may also obtain the complexity level of the historical block and the complexity level of the current block to be encoded, and determine the estimated number of bits of the historical block based on the quotient of the complexity level of the historical block and a reference value.

[0192] The reference value is the product of the complexity level of the current block to be encoded and the estimated number of bits of the current block to be encoded.

[0193] Optionally, the encoder may calculate the estimated number of bits of the historical block according to the following formula (1):

[0194] TargetRate t－i = complexity t－i / complexity * TargetRate formula (1)

[0195] In formula (1), TargetRate t－iIndicates the estimated number of bits in the history block. t－i Indicates the complexity level of the historical block. complexity indicates the complexity level of the current block to be encoded. TargetRate indicates the estimated number of bits of the current block to be encoded.

[0196] Step 2: The encoder obtains the additional buffer threshold.

[0197] The additional buffer threshold is used to represent the available space of the additional buffer.

[0198] In a possible implementation, the penalty parameter may include a first penalty parameter. In this case, the above step 2 may specifically include: when the current block to be encoded is the end block of the stripe where the current block to be encoded is located, the encoder determines the additional buffer threshold according to the difference between the initial size of the additional buffer and the reference product.

[0199] Among them, the end block is a block to be encoded that needs to be filled with preset placeholder bits. The end block is a block to be encoded that needs to be filled with placeholder bits when the initial transmission delay function is turned on. The initial transmission delay function can refer to the relevant technology and will not be repeated here. The end block can include multiple. The initial size of the additional buffer can be preset by the administrator in the encoding end, and its value is related to the data type of the image where the block to be encoded is located, the preset BPP, and the number of delayed blocks. For example, for a 10-bit image delayed by 2 blocks, the initial size of the additional buffer can be set to 8192 bits. The embodiment of the present application does not limit the specific value of the initial size of the additional buffer. The reference product is the product of the reference ranking and the first penalty parameter, and the reference ranking is the ranking of the current block to be encoded in the end block. The first penalty parameter is the value of the additional buffer threshold that becomes smaller when the reference ranking increases by one block to be encoded, or the decreasing step size of the additional buffer threshold when the reference ranking increases by one block to be encoded.

[0200] Exemplarily, taking the initial size of the additional buffer as 1000 bits as an example, assuming that the descending step size represented by the first penalty parameter is 50 bits, when the current block to be encoded is the first end block, the additional buffer threshold at this time is 1000-50×1=950 bits, when the current block to be encoded is the second end block, the additional buffer threshold at this time is 1000-50×2=900 bits, when the current block to be encoded is the third end block, the additional buffer threshold at this time is 1000-50×3=850 bits, ..., and so on, until the additional buffer threshold is 0.

[0201] Optionally, the additional buffer threshold may be calculated according to the following formula (1):

[0202] ExtraBufferThreshold=Max(0, ExtraBufferThreshold-(1<<ExtraDecreaseStepLog2))) formula (2)

[0203] In formula (2), ExtraBufferThreshold represents the threshold of the additional buffer, which is initialized to the initial size of the additional buffer (ExtraBufferSize). ExtraDecreaseStepLog2 represents the logarithm of the first penalty parameter with base 2. ExtraDecreaseStepLog2 is used to control the coverage of the additional buffer. The larger its value, the smaller the coverage of the additional buffer. This value only affects the coverage of the additional buffer when encoding and decoding to the end block.

[0204] Optionally, when the current block to be encoded is a block to be encoded other than the end block, the additional buffer is an initial size of the additional buffer.

[0205] Optionally, before the above step 1, the method may further include: the encoding end obtains the number of end blocks; the encoding end determines a first penalty parameter according to the initial size of the additional buffer and the number of end blocks.

[0206] Optionally, the first penalty parameter may be calculated according to the following formula (2):

[0207]

[0208] In formula (3), EndControlBlocks represents the number of end blocks.

[0209] Optionally, the value of ExtraBufferDecreaseStepLog2 may be less than or equal to the value of DecreaseStepLog2, where DecreaseStepLog2 represents the logarithm with base 2 of the step size of the preset placeholder bits filled at the end of the slice in each block to be encoded.

[0210] For example, assuming that the placeholder bits filled in the first end block are 50 bits and the step size is 50 bits, then the placeholder bits filled in the second end block are 50+50=100 bits, the placeholder bits filled in the third end block are 50+50+50=150 bits, and so on.

[0211] When ExtraBufferDecreaseStepLog2=DecreaseStepLog2, that is, the extra buffer is first filled with placeholder bits at the end block of the slice, and the extra buffer disappears after it is filled. At this time, the first operation is no longer performed on the estimated number of bits.

[0212] Optionally, the value of ExtraBufferDecreaseStepLog2 may be greater than the value of DecreaseStepLog2. When ExtraBufferDecreaseStepLog2 is too large, for example, greater than a preset penalty threshold, no extra buffer exists starting from the first end block of the strip, and the first operation is no longer performed on the estimated number of bits.

[0213] The penalty threshold is related to the size of the additional buffer. For example, if the size of the additional buffer is 8192 bits, the penalty threshold may be 1408 bits.

[0214] It should be understood that the end block is usually a number of to-be-encoded blocks (image blocks) that are at the end of the encoding order in a strip. The closer a strip is to encoding completion, the greater the reference ranking of the current to-be-encoded block in the end block, the greater the reference product, the smaller the additional buffer threshold, the less likely it is that the estimated number of bits of the current to-be-encoded block will be used to perform the first operation, and more encoding blocks will be filled into the additional buffer to make full use of encoding resources.

[0215] In another possible implementation, the encoding end may directly determine the initial size of the additional buffer as the additional buffer threshold. In this possible embodiment, the existence of the buffer mechanism can be fully ensured.

[0216] Step 3: When the available buffer size is less than the additional buffer threshold, the encoding end performs a first operation on the estimated number of bits of the current block to be encoded.

[0217] In a possible implementation, the penalty parameter also includes a second penalty parameter, which is the number of blocks to be encoded required to consume the number of bits in the additional buffer. In this case, before the encoder performs a first operation on the estimated number of bits of the current block to be encoded, the method also includes: the encoder obtains the number of remaining blocks to be encoded in the stripe where the current block to be encoded is located; if the number of remaining blocks to be encoded is less than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the number of currently remaining blocks to be encoded and the preset pixel depth.

[0218] In another possible implementation, if the number of remaining blocks to be encoded is greater than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0219] Optionally, the encoding end may calculate the estimated number of bits after performing the first operation according to the following formula (4) to formula (6):

[0220] ExtraBufferPenaltyLog2=MIN(RemainBlksLog2, ExtraBufferPenaltyLog2) formula (4)

[0221] In formula (4), ExtraBufferPenaltyLog2 represents the logarithm of the second penalty parameter with base 2. When the value is 3, it means that the number of blocks to be encoded required to consume the number of bits in the extra buffer is 8. ExtraBufferPenaltyLog2 is used to adjust the penalty intensity. The larger the value, the smaller the penalty intensity, which affects the number of penalty bits of all blocks. RemainBlksLog2 represents the logarithm of the number of remaining blocks to be encoded with base 2, and its minimum value is 0.

[0222] When ExtraBufferPenaltyLog2 is equal to 2,

[0223] TargetRate=Max(0, TargetRate–((ExtraBufferThreshold-availableBuffer)) formula (5)

[0224] When ExtraBufferPenaltyLog2 is less than 2,

[0225] TargetRate=Max(0, TargetRate–((ExtraBufferThreshold-availableBuffer)<<ExtraBufferPenaltyLog2)) Formula (6)

[0226] S103: The encoding end encodes the current block to be encoded based on the estimated number of bits after performing the first operation.

[0227] For example, the encoding end can determine the quantization parameter of the current block to be encoded based on the estimated number of bits after performing the first operation, and encode the current block to be encoded based on the quantization parameter. The specific process can be referred to as described in the above-mentioned encoding and decoding system, and will not be repeated here.

[0228] In some possible embodiments, if LosslessFlag is equal to 1, the quantization parameters Qp[0] and Qp[1] of the luminance coding block and the chrominance coding block of the current coding unit (i.e., the aforementioned current block to be encoded) are both 0; otherwise, the quantization parameter MasterQp of the coding unit is first calculated according to the luminance complexity level ComplexityLevel[0] and the chrominance complexity level ComplexityLevel[1] of the current coding unit, and then the quantization parameters Qp[0] and Qp[1] of the luminance coding block and the chrominance coding block of the current coding unit are calculated according to MasterQp.

[0229] The specific process of calculating the quantization parameter MasterQp of the coding unit according to the brightness complexity level ComplexityLevel[0] and the chrominance complexity level ComplexityLevel[1] of the current coding unit is as follows:

[0230] physicalBufferLevelPrev=PhysicalBufferLevelRecord[CurrPos]

[0231] rcBufferLevelPrev=physicalBufferLevelPrev+(VirtualEndStuff<ExtraBufferSize?0:VirtualEndStuff–ExtraBufferSize)

[0232] avgLosslessBits=AvgLosslessBitsRecord[CurrPos]

[0233] bppAdj=(EndTargetFullness–physicalBufferLevelPrev)<<7

[0234] bppAdj=bppAdj>=0? bppAdj>>(5+RemainBlksLog2):–((–bppAdj)>>(5+RemainBlksLog2))

[0235] bppAdj=Clip3((–512,1536,bppAdj)

[0236] bpp=(bpp<<3)+bppAdj

[0237] shiftCurr=FullnessCalcShift–7

[0238] Fullness=(rcBufferLevelPrev×FullnessCalcMultiplier+((1<<shiftCurr)>>1))>>shiftCurr

[0239] bitsOffset=BitsOffset+(341–((43×bpp)>>7)

[0240] bitsOffset=Max(bitsOffset,0)

[0241] tmp=(avgLosslessBits>>7)–bitsOffset

[0242] tmp=Clip3(1,32,((tmp>>5)+1)>>1)

[0243] infoRatio=(bpp×InverseTable[tmp–1]+256)>>9

[0244] if(image_format==‘000’){ / *YUV400* /

[0245] CuComplexityLevel=ComplexityLevel[0]

[0246] }else if(image_format==‘001’){ / *YUV420* /

[0247] CuComplexityLevel=ComplexityDivide3Table[ComplexityLevel[1]+(ComplexityLevel[0]<<1)]

[0248] }else if(image_format==‘010’){ / *YUV422* /

[0249] CuComplexityLevel=(ComplexityLevel[0]+ComplexityLevel[1])>>1

[0250] }else if(image_format==‘011’||image_format==‘100’){ / *YUV444orRGB444* /

[0251] infoRatio=infoRatio>((77×(2+ChromaSampleRate))>>1)?(77×(2+ChromaSampleRate))>>1:infoRatio

[0252] CuComplexityLevel=ComplexityDivide3Table[ComplexityLevel[0]+(ComplexityLevel[1]<<1)]

[0253] }

[0254] losslessBits=LosslessBitsRecord[CurrPos][CuComplexityLevel]

[0255] relativeLosslessBits=Clip3(0,320,MaxLosslessBits–losslessBits)

[0256] tmp=(BppParam×bpp+512))>>10

[0257] minRate1=(tmp×Min(MaxLosslessBits,losslessBits)+64)>>7

[0258] tmp=160×Fullness

[0259] tmp=((tmp<<7)+1024)>>11

[0260] tmp=(192–tmp)×bpp

[0261] minRate2=((minRate2<<7)+8192)>>14

[0262] tmp=((Fullness–109)×32+8)>>4

[0263] tmp=bpp–tmp

[0264] minRate3=tmp–(relativeLosslessBits×(2+ChromaSampleRate))>>1

[0265] minRate=Max(Max(minRate1,minRate2),minRate3)

[0266] tmp=1121–((((Fullness×1083)<<7)+8192)>>14)

[0267] bppOffset1=tmp–(((ChromaSampleRate+2)×relativeLosslessBits+1)>>1)

[0268] bppOffset2=Max(1536–bpp,512)–relativeLosslessBits

[0269] bppOffset3=1554–((((457×Fullness)<<7)+2048)>>12)

[0270] bppOffset=Min(Min(bppOffset1,bppOffset2),bppOffset3)

[0271] if(CbPosY[0]==CurrSlicePosY){

[0272] bppOffset=Max(bppOffset,–128)

[0273] }

[0274] maxRate=Max(bpp+bppOffset,minRate)

[0275] TargetRate=(infoRatio×Max(0,losslessBits–bitsOffset)+64)>>7

[0276] TargetRate=Clip3(minRate,maxRate,TargetRate)

[0277] availableBuffer=MaxBufferSize–rcBufferLevelPrev+((bpp<<4)+(bpp<<3)–TargetRate–TargetRateRecord[(CurrPos+1)%3]–TargetRateRecord[(CurrPos+2)%3])>>2;

[0278] if(VirtualEndStuff>0){

[0279] ExtraBufferThreshold=Max(0, ExtraBufferThreshold–(1<<ExtraBufferDecreaseStepLog2))

[0280] }

[0281] if(available_buffer<ExtraBufferThreshold){

[0282] TargetRate=Max(0, TargetRate–((ExtraBufferThreshold–availableBuffer)>>(ExtraBufferPenaltyLog2–2)));

[0283] }

[0284] tmp=(TargetRate×InvElem+((1<<InvElemShift)>>1))>>InvElemShift

[0285] MasterQp=(losslessBits–tmp)<<3

[0286] Among them, InverseTable and ComplexityDivide3Table are defined as: InverseTable = {1024, 512, 341, 256, 205, 171, 146, 128, 114, 102, 93, 85, 79, 73, 68, 64, 60, 57, 54, 51, 49, 47, 45, 43, 41, 39, 38, 37, 35, 34, 33, 32}, ComplexityDivide3Table = {0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4}.

[0287] According to the brightness complexity level ComplexityLevel[0] and the chrominance complexity level ComplexityLevel[1] of the current coding unit, BiasInit is obtained by looking up Table 1, and then the brightness quantization parameter Qp[0] and the chrominance quantization parameters Qp[1] and Qp[2] are calculated.

[0288] Bias=(BiasInit×FormatBias)>>1

[0289] tmp=ChromaSampleRate×Bias

[0290] tmp=((tmp<<7)+128)>>8

[0291] Qp[0]=Clip3(0, MaxQp[0], (MasterQp–tmp)>>7)

[0292] Qp[1]=Clip3(0, MaxQp[1], (MasterQp+Bias)>>7)

[0293] Qp[2]=Qp[1]

[0294] Table 1

[0295]

[0296]

[0297] In some other possible embodiments, before encoding each stripe, it is also necessary to initialize the WarmUp array, EndTargetFullness parameter, and ExtraBufferThreshold parameter of the code control. The specific process is as follows:

[0298] for(j=0;j<5;j++){

[0299] WarmUp[j]=4

[0300] }

[0301] EndTargetFullness=((DelayBits–(3×(SubstreamSegmentSize–1)))×3)>>2

[0302] ExtraBufferThreshold=ExtraBufferSize

[0303] Then, according to the bit depth BitDepth[0] of the image where the current block to be encoded is located and the image format ImageFormat, Table 2 is consulted to obtain the initialization values ​​of LosslessBits[i], AvgLosslessBits, BitsOffset, MaxLosslessBits and BppParam.

[0304] Table 2

[0305]

[0306] According to ImageFormat, look up Table 3 to get the initialization values ​​of ChromaSampleRate, InvElem, InvElemShift and FormatBias.

[0307] Table 3

[0308]

[0309]

[0310] Initialize the parameters LosslessBitsRecord[i][j], AvgLosslessBitsRecord[i], PhysicalBufferLevelRecord[i] and CurrPos. The specific process is as follows:

[0311] for(i=0;i<3;i++){

[0312] for(j=0;i<5;j++){

[0313] LosslessBitsRecord[i][j]=LosslessBits[j]

[0314] }

[0315] AvgLosslessBitsRecord[i]=AvgLosslessBits

[0316] PhysicalBufferLevelRecord[i]=DelayBits

[0317] }

[0318] CurrPos=0

[0319] In some embodiments, the method may further include: the encoding end encoding the first penalty parameter and the second penalty parameter into a bit stream of the encoding block.

[0320] In the image encoding method provided by the embodiment of the present application, when the state of the code stream buffer satisfies a preset condition, the encoding end can perform a first operation on the estimated number of bits of the current block to be encoded to reduce the predicted estimated number of bits for encoding the current block to be encoded. The estimated number of bits is inversely proportional to the quantization parameter, and the quantization parameter is inversely proportional to the encoding bit rate. Therefore, when the estimated number of bits is reduced, the bit rate when encoding the current block to be encoded is also reduced, thereby avoiding code stream buffer overflow caused by using the code stream buffer state of the historical block to control the encoding bit rate of the current block to be encoded in the high-speed video transmission interface.

[0321] Correspondingly, an embodiment of the present application also provides an image decoding method. Figure 8 The following is a flow chart of an image decoding method provided in an embodiment of the present application. Figure 8 As shown, the image decoding method includes S201 to S203.

[0322] S201: The decoding end obtains the status of the code stream buffer.

[0323] The code stream buffer is used to store the decoded code stream of the block to be decoded. The state of the code stream buffer is the number of bits of the code stream buffer after decoding the Mth block to be decoded before the current block to be decoded.

[0324] Optionally, the code stream buffer may include a main buffer and an additional buffer. The main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0325] S202: When the state of the code stream buffer satisfies a preset condition, the decoding end performs a first operation on the estimated number of bits of the current block to be decoded.

[0326] The estimated number of bits is the number of bits required for decoding the current block to be decoded predicted in the state of the code stream buffer. The first operation is to reduce the estimated number of bits by the number of penalty bits indicated by the penalty parameter.

[0327] Optionally, the preset condition may be that the available buffer size determined based on the state of the code stream buffer is smaller than the available space of the additional buffer. In this case, the above S202 may specifically include the following three steps:

[0328] Step 1: The decoder determines the available buffer size based on the status of the code stream buffer.

[0329] The available buffer size is used to represent the available size of the code stream buffer after the current to-be-decoded block is decoded using the estimated number of bits.

[0330] For example, the decoding end may determine the available buffer size based on the state of the code stream buffer, the estimated number of bits of the current block to be decoded, and the estimated number of bits of each of the M blocks to be decoded before the current block to be decoded.

[0331] In a possible implementation, the above step 1 may specifically include: the decoding end obtains the estimated number of bits of M blocks to be decoded before the current block to be decoded; the decoding end determines the available buffer size according to the sum of the bit stream buffer and the reference change value.

[0332] The reference change value is the difference between the fixed number of incoming bits of the M+1 blocks in the code stream buffer and the estimated number of outgoing bits of the M+1 blocks in the code stream buffer. The fixed number of outgoing bits of the M+1 blocks in the code stream buffer is the product of the number of pixels of each block to be decoded, the preset pixel depth, and M+1. The estimated number of incoming bits of the M+1 blocks in the code stream buffer is the sum of the estimated number of bits of the current block to be decoded and the estimated number of bits of each of the M blocks to be decoded before the current block to be decoded.

[0333] Optionally, for any historical block, which is any one of the M blocks to be decoded before the block to be decoded, the decoding end may directly obtain the estimated number of bits obtained when decoding the historical block.

[0334] Optionally, for any one of the historical blocks, the decoding end may also obtain the complexity level of the historical block and the complexity level of the current block to be decoded, and determine the estimated number of bits of the historical block based on the quotient of the complexity level of the historical block and a reference value.

[0335] The reference value is the product of the complexity level of the current block to be decoded and the estimated number of bits of the current block to be decoded.

[0336] Step 2: The decoding end obtains the additional buffer threshold.

[0337] The additional buffer threshold is used to represent the available space of the additional buffer.

[0338] In a possible implementation, the penalty parameter may include a first penalty parameter. In this case, the above step 2 may specifically include: when the current block to be decoded is the end block of the stripe where the current block to be decoded is located, the decoding end determines the additional buffer threshold according to the difference between the initial size of the additional buffer and the reference product.

[0339] The end block is a block to be decoded that needs to be filled with preset placeholder bits.

[0340] Optionally, when the current block to be decoded is a decoded block other than the end block, the additional buffer is an initial size of the additional buffer.

[0341] Optionally, before the above step 1, the method may further include: the decoding end parses the bitstream stored in the bitstream buffer to obtain a first penalty parameter.

[0342] In another possible implementation manner, the decoding end may directly determine the initial size of the additional buffer as the additional buffer threshold.

[0343] Step 3: When the available buffer size is less than the additional buffer threshold, the decoding end performs a first operation on the estimated number of bits of the current block to be decoded.

[0344] In a possible implementation, the penalty parameter further includes a second penalty parameter, which is the number of blocks to be decoded required to consume the number of bits in the additional buffer. In this case, before the decoding end performs the first operation on the estimated number of bits of the current block to be decoded, the method further includes: the decoding end parses the second penalty parameter from the bitstream stored in the bitstream buffer; the decoding end obtains the number of remaining blocks to be decoded in the stripe where the current block to be decoded is located; if the number of remaining blocks to be decoded is less than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the number of currently remaining blocks to be decoded and the preset pixel depth.

[0345] In another possible implementation, if the number of blocks currently remaining to be decoded is greater than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0346] S203: The decoding end decodes the current block to be decoded based on the estimated number of bits obtained after performing the first operation.

[0347] For example, the decoding end can determine the quantization parameter of the current block to be decoded based on the estimated number of bits after performing the first operation, and decode the current block to be decoded based on the quantization parameter. The specific process can be referred to as described in the above-mentioned decoding system and will not be repeated here.

[0348] Based on the above Figure 8 According to the understanding, an image decoding method provided in an embodiment of the present application may specifically include the following steps:

[0349] a) The decoder parses the status of the bitstream buffer (or the relevant information of the additional buffer);

[0350] b) Decoding complexity information at the decoding end

[0351] c) Calculation of quantization parameters at the decoding end

[0352] Steps a) to c) may refer to the above decoding method and will not be described in detail.

[0353] d) The decoding end analyzes the residual coefficients between the prediction mode and the quantization parameters;

[0354] e) The decoding end obtains the predicted value of each pixel of the current block to be decoded (or decoding unit) based on the prediction mode;

[0355] f) The decoding end obtains the residual value of each pixel of the current block to be decoded (or decoding unit) based on the residual coefficient decoding;

[0356] g) The decoding end obtains the reconstructed value based on the predicted value and residual value of each pixel point of the current block to be decoded (or decoding unit).

[0357] In some possible embodiments, the bit rate may be reduced by modifying the quantization parameter. Fig. 9 A flowchart of another image encoding method provided in an embodiment of the present application is shown below. Fig. 9 As shown, the image encoding method includes S301 to S303.

[0358] S301: The encoder obtains the status of the code stream buffer.

[0359] The code stream buffer is used to store the code stream obtained after encoding the block to be encoded; the state of the code stream buffer is the number of bits after the Mth block to be encoded before the current block to be encoded is encoded in the code stream buffer.

[0360] In a possible implementation, the code stream buffer may include a main buffer and an additional buffer. The main buffer and the additional buffer are both used to store the code stream obtained after encoding the block to be encoded. The main buffer and the additional buffer may be different physical storage partitions, or different logical storage partitions.

[0361] S302: When the filling of the code stream buffer satisfies a preset condition, the encoder performs a second operation on the estimated quantization parameter of the current block to be encoded to obtain a target quantization parameter of the current block to be encoded.

[0362] The estimated quantization parameter is obtained based on the estimated number of bits, which is the number of bits required to encode the current block to be encoded predicted in the state of the code stream buffer. The second operation refers to adding the estimated quantization parameter to the reference quantization parameter, which is determined based on the number of penalty bits indicated by the penalty parameter.

[0363] Optionally, S302 may specifically include the following three steps:

[0364] Step 1: The encoder determines the available buffer size based on the status of the code stream buffer.

[0365] Step 2: The encoder obtains the additional buffer threshold.

[0366] It should be noted that the specific steps 1 and 2 included in S302 can refer to the specific steps 1 and 2 included in S101 above, and will not be repeated here.

[0367] Step 3: When the available buffer size is less than the additional buffer threshold, the encoder performs a second operation on the estimated quantization parameter of the current block to be encoded.

[0368] In a possible implementation, the penalty parameter further includes a second penalty parameter, which is the number of blocks to be encoded required to consume the number of bits in the additional buffer. In this case, before the encoder performs the second operation on the estimated quantization parameter of the current block to be encoded, the method further includes: the encoder obtains the number of remaining blocks to be encoded in the stripe where the current block to be encoded is located; if the number of remaining blocks to be encoded is less than the second penalty parameter, then in the second operation performed on the estimated quantization parameter of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the number of currently remaining blocks to be encoded and the preset pixel depth.

[0369] In another possible implementation, if the number of remaining blocks to be encoded is greater than the second penalty parameter, then in the second operation performed on the estimated quantization parameter of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0370] Optionally, the encoding end performs a second operation on the estimated quantization parameter of the current block to be encoded, which may include: the encoding end determines the number of penalty bits based on the product of the second penalty parameter and a preset pixel depth; the encoding end determines the reference quantization parameter based on the number of penalty bits; the encoding end determines the target quantization parameter of the current block to be encoded based on the sum of the estimated quantization parameter and the reference quantization parameter.

[0371] The process of determining the reference quantization parameter according to the number of penalty bits at the encoder may refer to the process of determining the quantization parameter in some possible embodiments at S103 above, which will not be described in detail here.

[0372] Optionally, the encoding end may calculate the number of penalty bits according to the following formula (7):

[0373] DeltaRate=(ExtraBufferThreshold-availableBuffer)>>(ExtraBufferPenaltyLog2-2) formula (7)

[0374] In formula (7), DeltaRate represents the number of penalty bits.

[0375] Optionally, the process of the encoder calculating the reference quantization parameter according to the number of penalty bits is as follows:

[0376] tmp=(DeltaRate×InvElem+((1<<InvElemShift)>>1))>>InvElemShift

[0377] MasterQp=(losslessBits–tmp)<<3

[0378] Among them, InverseTable and ComplexityDivide3Table are defined as: InverseTable = {1024, 512, 341, 256, 205, 171, 146, 128, 114, 102, 93, 85, 79, 73, 68, 64, 60, 57, 54, 51, 49, 47, 45, 43, 41, 39, 38, 37, 35, 34, 33, 32}, ComplexityDivide3Table = {0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4}.

[0379] According to the brightness complexity level ComplexityLevel[0] and the chrominance complexity level ComplexityLevel[1] of the current coding unit, BiasInit is obtained by looking up Table 1, and then the reference brightness quantization parameter AdjQp[0] and the reference chrominance quantization parameters AdjQp[1] and AdjQp[2] are calculated.

[0380] Bias=(BiasInit×FormatBias)>>1

[0381] tmp=ChromaSampleRate×Bias

[0382] tmp=((tmp<<7)+128)>>8

[0383] AdjQp[0]=Clip3(0, MaxQp[0], (MasterQp–tmp)>>7)

[0384] AdjQp[1]=Clip3(0, MaxQp[1], (MasterQp+Bias)>>7)

[0385] AdjQp[2]=Qp[1]

[0386] Optionally, the encoding end may calculate the target quantization parameter of the current block to be encoded according to the following formula (8).

[0387] QP[i]=Clip3(0, MaxQp[i], BaseQP[i]+AdjQP[i]) Formula (8)

[0388] In formula (8), QP[i] represents the target quantization parameter of the current block to be encoded. BaseQP[i] represents the estimated quantization parameter of the current block to be encoded. AdjQP[i] represents the reference quantization parameter, and i is 0, 1, or 2.

[0389] S303: The encoder encodes the current block to be encoded based on the target quantization parameter obtained after performing the second operation.

[0390] The specific process of S303 may refer to the description of the above-mentioned encoding and decoding system, which will not be repeated here.

[0391] In the image encoding method provided by the embodiment of the present application, when the state of the code stream buffer satisfies a preset condition, the encoding end can perform a second operation on the estimated quantization parameter of the current block to be encoded to obtain a target quantization parameter after adding the reference quantization parameter. The quantization parameter is inversely proportional to the encoding bit rate. Therefore, as the quantization parameter increases, the bit rate when encoding the current block to be encoded also decreases, thereby avoiding code stream buffer overflow caused by using the code stream buffer state of the historical block to control the encoding bit rate of the current block to be encoded in the high-speed video transmission interface.

[0392] Correspondingly, an embodiment of the present application also provides an image decoding method. Fig.10 A flowchart of another image decoding method provided in an embodiment of the present application is shown below. Fig.10 As shown, the image decoding method includes S401 to S403.

[0393] S401: The decoding end obtains the status of the code stream buffer.

[0394] The code stream buffer is used to store the code stream of the block to be decoded. The state of the code stream buffer is the number of bits after decoding the Mth block to be decoded before the current block to be decoded in the code stream buffer.

[0395] Optionally, the code stream buffer may include a main buffer and an additional buffer. The main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0396] S402: When the state of the code stream buffer satisfies a preset condition, the decoding end performs a second operation on the estimated quantization parameter of the current block to be decoded to obtain a target quantization parameter of the current block to be decoded.

[0397] The estimated number of bits is the number of bits required for decoding the current block to be decoded predicted in the state of the code stream buffer. The first operation is to reduce the estimated number of bits by the number of penalty bits indicated by the penalty parameter.

[0398] Optionally, the above S402 may specifically include the following three steps:

[0399] Step 1: The decoder determines the available buffer size based on the status of the code stream buffer.

[0400] Step 2: The decoding end obtains the additional buffer threshold.

[0401] It should be noted that the specific steps 1 and 2 included in S402 can refer to the specific steps 1 and 2 included in the above S202, and will not be repeated here.

[0402] Step 3: When the available buffer size is less than the additional buffer threshold, the decoding end performs a second operation on the estimated quantization parameter of the current block to be decoded to obtain a target quantization parameter of the current block to be decoded.

[0403] In a possible implementation, the penalty parameter further includes a second penalty parameter, which is the number of blocks to be decoded required to consume the number of bits in the additional buffer. In this case, before the decoder performs the second operation on the estimated quantization parameter of the current block to be decoded, the method further includes: the decoder parses the second penalty parameter from the bitstream stored in the bitstream buffer; the decoder obtains the number of remaining blocks to be decoded in the stripe where the current block to be decoded is located; if the number of remaining blocks to be decoded is less than the second penalty parameter, then in the second operation performed on the estimated quantization parameter of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the number of currently remaining blocks to be decoded and the preset pixel depth.

[0404] In another possible implementation, if the number of blocks currently remaining to be decoded is greater than the second penalty parameter, in the second operation performed on the estimated quantization parameter of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0405] Optionally, the decoding end performs a second operation on the estimated quantization parameter of the current block to be decoded, which may include: the decoding end determines the number of penalty bits based on the product of the second penalty parameter and a preset pixel depth; the decoding end determines the reference quantization parameter based on the number of penalty bits; the decoding end determines the target quantization parameter of the current block to be decoded based on the sum of the estimated quantization parameter and the reference quantization parameter.

[0406] Among them, the process of determining the number of penalty bits at the decoding end according to the product of the second penalty parameter and the preset pixel depth can refer to the above formula (7), which will not be repeated here. The process of determining the reference quantization parameter at the decoding end according to the number of penalty bits can refer to the process of determining the quantization parameter in some possible embodiments at S103 above, which will not be repeated here. The process of determining the reference quantization parameter at the decoding end according to the number of penalty bits can refer to the code at step 3 in S302 above, which will not be repeated here. The decoding end determines the target quantization parameter of the current block to be decoded according to the sum of the estimated quantization parameter and the reference quantization parameter, which can refer to the above formula (8), which will not be repeated here.

[0407] S403: The decoding end decodes the current block to be decoded based on the target quantization parameter obtained after performing the second operation.

[0408] The specific process of S403 can be referred to the above-mentioned decoding system, which will not be repeated here.

[0409] In some possible embodiments, the encoding end can turn on the sub-stream parallel function. When the sub-stream parallel function is turned on, the encoding end can use multiple entropy encoders (modules) to encode different channels of the current block to be encoded to obtain multiple sub-streams, and fill the multiple sub-streams into their respective corresponding sub-stream buffers, and interleave the sub-streams in the sub-stream buffer into a code stream according to a preset interleaving rule.

[0410] It should be noted that, considering the dependency between substreams, different substream buffers are filled with substreams at different speeds. At the same time, the substream buffer with a faster filling speed has more substream bits than the substream buffer with a slower filling speed. In order to ensure the integrity of the filled data, all substream buffers need to be set larger, which increases the hardware cost. In order to save hardware costs, an anti-expansion mode is proposed. The anti-expansion mode represents the mode with the highest bit cost among all prediction modes. Usually, the mode with the highest bit cost refers to the coding mode that directly encodes the original pixel value of the current block to be coded.

[0411] Optionally, for an N-channel image, the bit depths (bitDepth) of the N channels may be different. When encoding the N-channel image, if an anti-expansion mode is selected for a block to be encoded after rate-distortion optimization (the anti-expansion mode participates in rate-distortion optimization), or the bit cost of the optimal mode selected for a block to be encoded after rate-distortion optimization is higher than the bit cost of the anti-expansion mode (the anti-expansion mode does not participate in rate-distortion optimization), in order to save hardware costs and prevent the sub-stream buffer from expanding, the encoding end will select the anti-expansion mode. At this time, the bit cost of the anti-expansion mode is the cost obtained by encoding the N channels according to the bit depth of the channel with the smallest bit depth among the N channels, that is, the encoding of some channels is lossy.

[0412] Based on the above understanding, an embodiment of the present application also provides another image encoding method, which includes: the encoding end encodes the brightness channel, the first chroma channel, and the second chroma channel according to the bit depth of the brightness channel, and encodes the to-be-encoded blocks of the brightness channel, the to-be-encoded blocks of the first chroma channel, and the to-be-encoded blocks of the second chroma channel according to the anti-expansion mode.

[0413] It should be understood that converting RGB to the YCoCg format will cause the bit depth of the Co channel and the Cg channel to be 1 bit more than the Y channel. At this time, the embodiment of the present application stipulates that the anti-expansion mode encodes the to-be-encoded blocks of the Y channel, the to-be-encoded blocks of the Co channel, and the to-be-encoded blocks of the Cg channel at the bit depth of the Y channel. The overall estimated number of bits of the encoding blocks of the three channels is small, thereby reducing the overflow of the code stream buffer caused by using the code stream buffer status of the historical block to control the encoding bit rate of the current block to be encoded in the high-speed video transmission interface.

[0414] Correspondingly, an embodiment of the present application also provides another image decoding method, which includes: a decoding end parses a mode flag from a code stream of a block to be decoded; the mode flag is used to indicate the encoding mode adopted when encoding the block to be encoded; when the encoding mode indicated by the mode flag is an anti-expansion mode, the decoding end decodes the current block to be decoded according to the bit depth of the luminance channel and the anti-expansion mode.

[0415] Optionally, also when the substream parallel function is turned on, the substreams of each channel are transmitted through different substream buffers, and the quantization parameter is derived and does not need to be transmitted. In this case, the luminance channel transmits the complexity information of the luminance, the first chrominance channel transmits the first relationship mark of the complexity information of the first chrominance and the luminance, the first relationship mark is used to indicate the size relationship between the complexity information of the first chrominance channel and the complexity information of the luminance, and the second chrominance channel transmits the second relationship mark of the complexity information of the second chrominance and the luminance, the second relationship mark is used to indicate the size relationship between the complexity information of the second chrominance and the complexity information of the luminance.

[0416] For example, the complexity information of brightness includes five categories: 0, 1, 2, 3, or 4. Then the first relationship mark transmitted in the first chromaticity channel may include 0, 1, or 2, where 0 indicates that the complexity information of the first chromaticity is consistent with the complexity information of brightness, 1 indicates that the complexity information of the first chromaticity is smaller than the complexity information of brightness, and 2 indicates that the complexity information of the first chromaticity is larger than the complexity information of brightness. The second relationship mark transmitted in the second chromaticity channel may include 0, 1, or 2, where 0 indicates that the complexity information of the second chromaticity is consistent with the complexity information of brightness, 1 indicates that the complexity information of the second chromaticity is smaller than the complexity information of brightness, and 2 indicates that the complexity information of the second chromaticity is larger than the complexity information of brightness.

[0417] Changes in complexity information affect rate control in two ways:

[0418] 1. Determine the complexity level of the current block to be encoded (or also called coding unit).

[0419] 2. Determine the quantization parameters of each channel.

[0420] Based on the above understanding, the embodiment of the present application further provides another image encoding method, which includes the following steps:

[0421] Step 1: The encoder obtains the current block to be encoded.

[0422] The current block to be encoded can be understood as an encoding unit, and the encoding unit includes encoding blocks of three YUV channels.

[0423] Step 2: The encoder obtains the complexity information of the coding block of each of the three channels.

[0424] The complexity information is used to characterize the degree of difference in pixel values ​​of the coding blocks of each channel. For example, the complexity information of the coding block of the luminance channel is used to characterize the degree of difference in pixel values ​​of the coding block of the luminance channel; the complexity information of the coding block of the first chrominance channel is used to characterize the degree of difference in pixel values ​​of the coding block of the first chrominance channel; the complexity information of the coding block of the second chrominance channel is used to characterize the degree of difference in pixel values ​​of the coding block of the second chrominance channel;

[0425] Step 3: The encoding end determines the complexity level of the current block to be encoded according to the complexity level of the encoding block of each channel.

[0426] In a possible implementation, the encoder may directly use the complexity level of the coding block of the luminance channel as the complexity level of the coding unit, that is, CuComplexityLevel == ComplexityLevel[0].

[0427] In another possible implementation, if the complexity level of the coding block of the luminance channel is 1, the complexity level of the coding block of the first chrominance channel is 1-1=0; otherwise, if the complexity level of the coding block of the luminance channel is 2, the complexity level of the coding block of the first chrominance channel is 2+1=3; otherwise, the complexity level of the coding block of the first chrominance channel is equal to the complexity level of the coding block of the luminance channel. The complexity level of the coding block of the second chrominance channel can refer to the complexity level of the coding block of the first chrominance channel described above, and will not be repeated. After obtaining the complexity levels of the three channels, the encoding end takes a weighted average to obtain the complexity level of the coding unit, and the average of the complexity levels of the first chrominance and the second chrominance is used as the complexity level of the chrominance.

[0428] In some possible embodiments, the method may further include: the encoding end correcting quantization parameters of three YUV channels according to complexity information of the first channel.

[0429] The first channel is any one of the three YUV channels.

[0430] For example, if the complexity information of the luminance channel is 1, the quantization parameter of the first chrominance channel is the difference between the quantization parameter of the luminance channel and the empirical value; otherwise, if the complexity information of the luminance channel is 2, the quantization parameter of the first chrominance channel is the sum of the quantization parameter of the luminance channel and the empirical value. The quantization parameter of the second chrominance channel can refer to the determination process of the quantization parameter of the first chrominance channel, which will not be repeated.

[0431] For example, the experience value can be 1.

[0432] Correspondingly, the embodiment of the present application also provides another image decoding method, which includes the following steps:

[0433] Step 1: Analyze the complexity information of the brightness channel in the brightness channel of the decoding end.

[0434] Step 2: The decoding end parses the first relationship tag from the first chrominance channel.

[0435] Step 3: The decoding end parses the second relationship tag from the second chrominance channel.

[0436] Step 4: The decoding end determines the complexity level of the luminance channel, the complexity level of the first chrominance channel, and the complexity level of the second chrominance channel based on the complexity information of the luminance channel, the first relationship mark, and the second relationship mark.

[0437] Step 5: The decoding end determines the complexity level of the current block to be decoded based on the complexity level of the luminance channel, the complexity level of the first chrominance channel, and the complexity level of the second chrominance channel.

[0438] Optionally, the method may also include: the decoding end determines the complexity information of the first chroma channel and the complexity information of the second chroma channel based on the complexity information of the luminance channel, the first relationship mark, and the second relationship mark, and determines the quantization parameter of the luminance channel, the quantization parameter of the first chroma channel, and the quantization parameter of the second chroma channel based on the complexity information of the luminance channel, the complexity information of the first chroma channel, and the complexity information of the second chroma channel.

[0439] In some embodiments, the embodiments of the present application also provide a displacement quantization method to reduce hardware costs and improve the competitiveness of programmable logic controller (PLC) standards.

[0440] The existing quantization scheme of the PLC standard is 8-point fractional quantization, the quantization step is Qstep=2^(QP / 8), and the value range of QP is 0-8×bitdepth-1.

[0441] The 8-point score quantification mainly includes the following three changes:

[0442] Change 1: Align the QP exported by code control to a multiple of 8.

[0443] The specific process of change 1 is:

[0444] luma_qp=((luma_qp+3)>>3)<<3;

[0445] chroma_qp=((chroma_qp+3)>>3)<<3;

[0446] Among them, luma_qp and chroma_qp are the luminance and chromaticity QP derived by the code control, luma_qp is also lumaQP and Qp[0], chroma_qp is also chromaQP, Qp[1], and Qp[2]. The rounding offset is 3, which means that the alignment method is rounded according to Qstep. For details, see Table 4.

[0447] Table 4

[0448]

[0449]

[0450] Change 2: Adaptively modify the parameters of the stationary point quantization technology.

[0451] The adjustment method of QP by point-by-point quantization technology includes the following two parts:

[0452] a) Determine the values ​​of parameters jndQp, adjustMaxQp and resiThresjndQp according to BitDepth[component]. The specific process is as follows:

[0453] bdIdx=Clip3(0, 8, BitDepth[component]–8)

[0454] jndQp=(bdIdx<<2)+16((bdIdx>>1)<<3)+16

[0455] adjustMaxQp=((bdIdx>>1)<<3)+32

[0456] resiThres=10<<(bdIdx>>1)

[0457] b) Determine the quantization parameter PixelQp of the residual sample. The specific process is as follows:

[0458] 1) If Qp[component] is greater than jndQp and Qp[component] is less than or equal to adjustMaxQp, and PredResiTmp is less than or equal to resiThres, then:

[0459] PixelQp=Max(Qp[component]–4 8,jndQp)

[0460] 2) Otherwise, let PixelQp equal Qp[component].

[0461] Change 3: The rounding operation when masterQP exports luminance and chrominance QP is changed from floor to round.

[0462] The specific process of change 3 is as follows:

[0463] Bias=(BiasInit×FormatBias)>>1

[0464] tmp=ChromaSampleRate×Bias

[0465] tmp=((tmp<<7)+128)>>8

[0466] Qp[0]=Clip3(0, MaxQp[0], (MasterQp–tmp+64)>>7)

[0467] Qp[1]=Clip3(0, MaxQp[1], (MasterQp+Bias+64)>>7)

[0468] Qp[2]=Qp[1]

[0469] On this basis, the present application provides two improved solutions for displacement quantization:

[0470] Improvement plan 1: Allow the following two methods to align the bright chroma QP to a multiple of 8.

[0471] Method 1: According to the Qstep rounding method, please refer to the above 8-point score quantization for details, which will not be repeated here.

[0472] Method 2: Align the luma QP downward and the chroma QP upward. This method is only used to replace the situation in which both the luma and chroma QPs are aligned upward in method 1.

[0473] The selection principle of method 1 or method 2 is: ensure that the absolute value of the total QP change is the smallest; if the absolute values ​​are equal, select the method with a negative change (distortion priority); if they are still equal, select method 1. The specific selection process is shown in Table 5.

[0474] Table 5

[0475]

[0476]

[0477] Improvement scheme 2: first round the chroma QP, then compensate the rounding error of the chroma QP to the brightness QP, and then round the brightness QP, so as to reduce the rounding error of the entire block to be encoded (or coding unit). The specific process is shown in Table 6 below.

[0478] Table 6

[0479]

[0480] lumaQP=masteQP-sampleRate*bias formula (9)

[0481] chromaQP=masterQP+bias formula (10)

[0482] lumaQP+sampleRate*chromaQP=(1+sampleRate)*masteQP formula (11)

[0483] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy to realize that the technical goals in this field are combined with the units and algorithm steps of each example described in the embodiments disclosed in this article, and the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical goals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0484] In an exemplary embodiment, the present application also provides an image encoding device. Fig.11 The following is a schematic diagram of the composition of the image encoding device provided in the embodiment of the present application. Fig.11 As shown, the device includes: an acquisition module 1101 and a processing module 1102.

[0485] The acquisition module 1101 is used to obtain the state of the code stream buffer; the code stream buffer is used to store the code stream obtained after the block to be encoded is encoded; the state of the code stream buffer is the number of bits of the code stream buffer after the Mth block to be encoded before the current block to be encoded is encoded; M is a positive integer.

[0486] The processing module 1102 is used to perform a first operation on the estimated number of bits of the current block to be encoded when the state of the code stream buffer satisfies a preset condition; the estimated number of bits is the number of bits required to encode the current block to be encoded predicted under the state of the code stream buffer; the first operation refers to reducing the estimated number of bits by the number of penalty bits indicated by the penalty parameter; and encoding the current block to be encoded based on the estimated number of bits after performing the first operation.

[0487] In some possible embodiments, the code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0488] In some other possible embodiments, the processing module 1102 is specifically used to determine the available buffer size based on the state of the code stream buffer; the available buffer size is used to characterize the available size of the code stream buffer after the current block to be encoded is encoded using the estimated number of bits; obtain an additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; when the available buffer size is less than the additional buffer threshold, perform a first operation on the estimated number of bits of the current block to be encoded.

[0489] In some other possible embodiments, the processing module 1102 is specifically used to determine the additional buffer threshold according to the difference between the initial size of the additional buffer and the reference product when the current block to be encoded is the end block of the strip where the current block to be encoded is located; the end block is a block to be encoded that needs to be filled with preset placeholder bits; the reference product is the product of the reference ranking and the first penalty parameter; the reference ranking is the ranking of the current block to be encoded in the end block; the first penalty parameter is the value by which the additional buffer threshold becomes smaller when one more block to be encoded is added to the reference ranking.

[0490] In some other possible embodiments, when the current block to be encoded is a block to be encoded other than the end block, the additional buffer threshold is an initial size of the additional buffer.

[0491] In some other possible embodiments, before obtaining the additional buffer threshold, the acquisition module 1101 is further used to obtain the number of end blocks; the processing module 1102 is further used to determine the first penalty parameter according to the initial size of the additional buffer and the number of end blocks.

[0492] In some other possible embodiments, the processing module 1102 is specifically configured to determine an initial size of the additional buffer as the additional buffer threshold.

[0493] In some other possible embodiments, the processing module 1102 is specifically used to obtain the estimated number of bits of the M blocks to be encoded before the current block to be encoded; determine the available buffer size according to the state of the code stream buffer and the sum of the reference change value; the reference change value is the difference between the fixed number of outgoing bits of the M+1 blocks in the code stream buffer and the estimated number of incoming bits of the M+1 blocks in the code stream buffer; the fixed number of outgoing bits of the M+1 blocks in the code stream buffer is the product of the number of pixels of each block to be encoded, the preset pixel depth, and M+1; the estimated number of incoming bits of the M+1 blocks in the code stream buffer is the sum of the estimated number of bits of the current block to be encoded and the estimated number of bits of each of the M blocks to be encoded before the current block to be encoded.

[0494] In some other possible embodiments, the processing module 1102 is specifically used to obtain the complexity level of the historical block and the complexity level of the current block to be encoded; determine the estimated number of bits of the historical block based on the quotient of the complexity level of the historical block and a reference value; the reference value is the product of the complexity level of the current block to be encoded and the estimated number of bits of the current block to be encoded.

[0495] In some other possible embodiments, the penalty parameter also includes a second penalty parameter; the second penalty parameter is the number of blocks to be encoded required to consume the number of bits in the additional buffer; before performing the first operation on the estimated number of bits of the current block to be encoded, the acquisition module 1101 is also used to obtain the number of remaining blocks to be encoded in the strip where the current block to be encoded is located; if the number of remaining blocks to be encoded is less than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the current number of remaining blocks to be encoded and the preset pixel depth.

[0496] In some other possible embodiments, if the number of blocks currently remaining to be encoded is greater than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0497] In an exemplary embodiment, the present application also provides an image decoding device. Fig.12 The following is a schematic diagram of the composition of the image decoding device provided in the embodiment of the present application. Fig.12 As shown, the device includes: an acquisition module 1201 and a processing module 1202.

[0498] The acquisition module 1201 is used to acquire the state of the code stream buffer; the code stream buffer is used to store the decoded code stream of the to-be-decoded block; the state of the code stream buffer is the number of bits of the code stream buffer after decoding the Mth to-be-decoded block before the current to-be-decoded block; M is a positive integer.

[0499] The processing module 12021 is used to perform a first operation on the estimated number of bits of the current block to be decoded when the state of the code stream buffer satisfies a preset condition; the estimated number of bits is the number of bits required for decoding the current block to be decoded predicted under the state of the code stream buffer; the first operation refers to reducing the estimated number of bits by the number of penalty bits indicated by the penalty parameter; based on the estimated number of bits obtained after performing the first operation, the current block to be decoded is decoded.

[0500] In some possible embodiments, the code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0501] In some other possible embodiments, the processing module 12021 is specifically used to determine the available buffer size based on the state of the code stream buffer; the available buffer size is used to characterize the available size of the code stream buffer after the current block to be decoded is decoded using the estimated number of bits; obtain an additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; when the available buffer size is less than the additional buffer threshold, perform a first operation on the estimated number of bits of the current block to be decoded.

[0502] In some other possible embodiments, the processing module 12021 is specifically used to determine the additional buffer threshold according to the difference between the initial size of the additional buffer and the reference product when the current block to be decoded is the end block of the strip where the current block to be decoded is located; the end block is a block to be decoded filled with preset placeholder bits; the reference product is the product of the reference ranking and the first penalty parameter; the reference ranking is the ranking of the current block to be decoded in the end blocks; the first penalty parameter is the value by which the size of the additional buffer decreases for each additional block to be decoded in the reference ranking.

[0503] In some other possible embodiments, when the current block to be decoded is a decoded block other than the end block, the additional buffer threshold is an initial size of the additional buffer.

[0504] In some other possible embodiments, before obtaining the additional buffer threshold, the processing module 12021 is further configured to parse the first penalty parameter from the bitstream stored in the bitstream buffer.

[0505] In some other possible embodiments, the processing module 12021 is specifically configured to determine an initial size of the additional buffer as an additional buffer threshold.

[0506] In some other possible embodiments, the processing module 12021 is specifically used to obtain the estimated number of bits of the M blocks to be decoded before the current block to be decoded; determine the available buffer size according to the state of the code stream buffer and the sum of the reference change value; the reference change value is the difference between the fixed number of incoming bits of the M+1 blocks in the code stream buffer and the estimated number of outgoing bits of the M+1 blocks in the code stream buffer; the fixed number of outgoing bits of the M+1 blocks in the code stream buffer is the product of the number of pixels of each block to be decoded, the preset pixel depth, and M+1; the estimated number of incoming bits of the M+1 blocks in the code stream buffer is the sum of the estimated number of bits of the current block to be decoded and the estimated number of bits of each of the M blocks to be decoded before the current block to be decoded.

[0507] In some other possible embodiments, before determining the available buffer size according to the state of the code stream buffer and the sum of the reference change values, the acquisition module 1201 is further used to obtain the complexity level of the historical block and the complexity level of the current block to be decoded; the historical block is any one of the M blocks to be decoded before the current block to be decoded; the processing module 1202 is further used to determine the estimated number of bits of the historical block based on the quotient of the complexity level of the historical block and the reference value; the reference value is the product of the complexity level of the current block to be decoded and the estimated number of bits of the current block to be decoded.

[0508] In some other possible embodiments, the penalty parameter also includes a second penalty parameter; the second penalty parameter is the number of blocks to be decoded required to consume the number of bits in the additional buffer; before performing the first operation on the estimated number of bits of the current block to be decoded, the processing module 1202 is further used to parse the second penalty parameter from the bitstream stored in the bitstream buffer; obtain the number of remaining blocks to be decoded in the stripe where the current block to be decoded is located; if the number of remaining blocks to be decoded is less than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the current number of remaining blocks to be decoded and the preset pixel depth.

[0509] In some other possible embodiments, if the number of blocks currently remaining to be decoded is greater than the second penalty parameter, then in the first operation performed on the estimated number of bits of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0510] In an exemplary embodiment, the present application also provides another image encoding device. Fig.13 A schematic diagram of another image encoding device provided in an embodiment of the present application. Fig.13 As shown, the device includes: an acquisition module 1301 and a processing module 1302.

[0511] The acquisition module 1301 is used to acquire the state of the code stream buffer; the code stream buffer is used to store the code stream obtained after the block to be encoded is encoded; the state of the code stream buffer is the number of bits of the code stream buffer after the Mth block to be encoded before the current block to be encoded is encoded; M is a positive integer;

[0512] The processing module 1302 is used to perform a second operation on the estimated quantization parameter of the current block to be encoded when the state of the code stream buffer satisfies a preset condition, so as to obtain a target quantization parameter of the current block to be encoded; the estimated quantization parameter is obtained based on the estimated number of bits; the estimated number of bits is the number of bits required for encoding the current block to be encoded predicted under the state of the code stream buffer; the second operation refers to adding a reference quantization parameter to the estimated quantization parameter; the reference quantization parameter is determined based on the number of penalty bits indicated by the penalty parameter; and based on the target quantization parameter obtained after performing the second operation, the current block to be encoded is encoded.

[0513] In some possible embodiments, the code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0514] In some other possible embodiments, the processing module 1302 is specifically used to determine the available buffer size based on the state of the code stream buffer; the available buffer size is used to characterize the available size of the code stream buffer after the current block to be encoded is encoded using the estimated number of bits; obtain an additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; when the size of the available buffer is less than the additional buffer threshold, perform a second operation on the estimated quantization parameter of the current block to be encoded.

[0515] In some other possible embodiments, the penalty parameter includes a first penalty parameter; the processing module 1302 is specifically used to determine the additional buffer threshold according to the difference between the initial size of the additional buffer and the reference product when the current block to be encoded is the end block of the strip where the current block to be encoded is located; the end block is a block to be encoded that needs to be filled with preset placeholder bits; the reference product is the product of the reference ranking and the first penalty parameter; the reference ranking is the ranking of the current block to be encoded in the end block; the first penalty parameter is the value by which the additional buffer threshold becomes smaller when one more block to be encoded is added to the reference ranking.

[0516] In some other possible embodiments, when the current block to be encoded is a block to be encoded other than the end block, the additional buffer threshold is an initial size of the additional buffer.

[0517] In some other possible embodiments, before obtaining the additional buffer threshold, the acquisition module 1301 is further used to obtain the number of end blocks; the processing module 1302 is further used to determine the first penalty parameter according to the initial size of the additional buffer and the number of end blocks.

[0518] In some other possible embodiments, the processing module 1302 is specifically configured to determine an initial size of the additional buffer as the additional buffer threshold.

[0519] In some other possible embodiments, the processing module 1302 is specifically used to obtain the estimated number of bits of the M blocks to be encoded before the current block to be encoded; determine the available buffer size according to the state of the code stream buffer and the sum of the reference change value; the reference change value is the difference between the fixed outflow bit number of the M+1 blocks in the code stream buffer and the estimated inflow bit number of the M+1 blocks in the code stream buffer; the fixed outflow bit number of the M+1 blocks in the code stream buffer is the product of the number of pixels of each block to be encoded, the preset pixel depth, and M+1; the estimated inflow bit number of the M+1 blocks in the code stream buffer is the sum of the estimated bit number of the current block to be encoded and the estimated bit number of each of the M blocks to be encoded before the current block to be encoded.

[0520] In some other possible embodiments, for the historical block, the historical block is any one of the M blocks to be encoded before the current block to be encoded, and the processing module 1302 is specifically used to obtain the complexity level of the historical block and the complexity level of the current block to be encoded; based on the quotient of the complexity level of the historical block and the reference value, determine the estimated number of bits of the historical block; the reference value is the product of the complexity level of the current block to be encoded and the estimated number of bits of the current block to be encoded.

[0521] In some other possible embodiments, the penalty parameter also includes a second penalty parameter; the second penalty parameter is the number of blocks to be encoded required to consume the number of bits in the additional buffer; before performing the second operation on the estimated quantization parameter of the current block to be encoded, the acquisition module 1301 is also used to obtain the number of remaining blocks to be encoded in the strip where the current block to be encoded is located; if the number of remaining blocks to be encoded is less than the second penalty parameter, then in the second operation performed on the estimated quantization parameter of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the current number of remaining blocks to be encoded and the preset pixel depth.

[0522] In some other possible embodiments, it is characterized in that if the number of currently remaining blocks to be encoded is greater than the second penalty parameter, then in the second operation performed on the estimated quantization parameter of the current block to be encoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and the preset pixel depth.

[0523] In an exemplary embodiment, the present application also provides another image decoding device. Fig.14 A schematic diagram of another image decoding device provided in an embodiment of the present application. Fig.14 As shown, the device includes: an acquisition module 1401 and a processing module 1402.

[0524] The acquisition module 1401 is used to acquire the state of the code stream buffer; the code stream buffer is used to store the decoded code stream of the to-be-decoded block; the state of the code stream buffer is the number of bits of the code stream buffer after decoding the Mth to-be-decoded block before the current to-be-decoded block; M is a positive integer.

[0525] The processing module 1402 is used to perform a second operation on the estimated quantization parameter of the current block to be decoded when the state of the code stream buffer satisfies a preset condition, so as to obtain a target quantization parameter of the current block to be decoded; the estimated quantization parameter is obtained based on the estimated number of bits; the estimated number of bits is the number of bits required for decoding the current block to be decoded predicted under the state of the code stream buffer; the second operation refers to adding a reference quantization parameter to the estimated quantization parameter; the reference quantization parameter is determined based on the number of penalty bits indicated by the penalty parameter; and based on the target quantization parameter obtained after performing the second operation, the current block to be decoded is decoded.

[0526] In some possible embodiments, the code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions.

[0527] In some other possible embodiments, the processing module 1402 is specifically used to determine the available buffer size based on the state of the code stream buffer; the available buffer size is used to characterize the available size of the code stream buffer after the current block to be decoded is decoded using the estimated number of bits; obtain an additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; when the size of the available buffer is less than the additional buffer threshold, perform a second operation on the estimated quantization parameter of the current block to be decoded.

[0528] In some other possible embodiments, the processing module 1402 is specifically used to determine the additional buffer threshold according to the difference between the initial size of the additional buffer and the reference product when the current block to be decoded is the end block of the strip where the current block to be decoded is located; the end block is a block to be decoded that needs to be filled with preset placeholder bits; the reference product is the product of the reference ranking and the first penalty parameter; the reference ranking is the ranking of the current block to be decoded in the end blocks; the first penalty parameter is the value by which the additional buffer threshold becomes smaller when one more block to be decoded is added to the reference ranking.

[0529] In some other possible embodiments, when the current block to be decoded is a block to be decoded other than the end block, the additional buffer threshold is an initial size of the additional buffer.

[0530] In some other possible embodiments, before obtaining the additional buffer threshold, the processing module 1402 is further configured to parse the first penalty parameter from the bitstream stored in the bitstream buffer.

[0531] In some other possible embodiments, the processing module 1402 is specifically configured to determine an initial size of the additional buffer as the additional buffer threshold.

[0532] In some other possible embodiments, the processing module 1402 is specifically used to obtain the estimated number of bits of the M blocks to be decoded before the current block to be decoded; determine the available buffer size according to the state of the code stream buffer and the sum of the reference change value; the reference change value is the difference between the fixed number of incoming bits of the M+1 blocks in the code stream buffer and the fixed number of outgoing bits of the M+1 blocks in the code stream buffer; the fixed number of outgoing bits of the M+1 blocks in the code stream buffer is the product of the number of pixels of each block to be decoded, the preset pixel depth, and M+1; the estimated number of incoming bits of the M+1 blocks in the code stream buffer is the sum of the estimated number of bits of the current block to be decoded and the estimated number of bits of each of the M blocks to be decoded before the current block to be decoded.

[0533] In some other possible embodiments, for the historical block, the historical block is any one of the M blocks to be decoded before the current block to be decoded, and the processing module 1402 is specifically used to obtain the complexity level of the historical block and the complexity level of the current block to be decoded; based on the quotient of the complexity level of the historical block and the reference value, determine the estimated number of bits of the historical block; the reference value is the product of the complexity level of the current block to be decoded and the estimated number of bits of the current block to be decoded.

[0534] In some other possible embodiments, the penalty parameter also includes a second penalty parameter; the second penalty parameter is the number of blocks to be decoded required to consume the number of bits in the additional buffer; before performing the second operation on the estimated quantization parameter of the current block to be decoded, the processing module 1402 is also used to parse the second penalty parameter from the bitstream stored in the bitstream buffer; obtain the number of remaining blocks to be decoded in the strip where the current block to be decoded is located; if the number of remaining blocks to be decoded is less than the second penalty parameter, then in the second operation performed on the estimated quantization parameter of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the current number of remaining blocks to be decoded and the preset pixel depth.

[0535] In some other possible embodiments, if the number of currently remaining blocks to be decoded is greater than the second penalty parameter, then in the second operation performed on the estimated quantization parameter of the current block to be decoded, the number of penalty bits indicated by the penalty parameter is the product of the second penalty parameter and a preset pixel depth.

[0536] It should be noted that Fig.11 , Fig.12 , Fig.13 ,as well as Fig.14The division of modules in the example is schematic and is only a logical function division. There may be other division methods in actual implementation. For example, two or more functions may be integrated into one processing module. The above integrated modules may be implemented in the form of hardware or software function modules.

[0537] In an exemplary embodiment, the present application also provides a readable storage medium, including execution instructions, which, when executed on an image coding and decoding device, enable the image coding and decoding device to execute any one of the methods provided in the above embodiments.

[0538] In an exemplary embodiment, the present application also provides a computer program product including execution instructions, which, when executed on an image coding and decoding device, enables the image coding and decoding device to execute any one of the methods provided in the above embodiments.

[0539] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer-executable instructions. When loading and executing computer-executable instructions on a computer, a process or function according to an embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer-executable instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more servers that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0540] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other changes to the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0541] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0542] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image coding method, characterized in that: The method comprises: Obtain the state of a code stream buffer; the code stream buffer is used to store the code stream obtained after encoding the block to be encoded; the state of the code stream buffer is the number of bits of the code stream buffer after encoding the Mth block to be encoded before the current block to be encoded; M is a positive integer; When the state of the code stream buffer satisfies a preset condition, performing a first operation on the estimated number of bits of the current block to be encoded; the estimated number of bits is the number of bits required to encode the current block to be encoded predicted under the state of the code stream buffer; the first operation refers to reducing the estimated number of bits by a penalty bit number; the penalty bit number is obtained based on a penalty parameter; Determine a quantization parameter of a current block to be encoded based on the estimated number of bits after performing the first operation, and encode the current block to be encoded according to the quantization parameter; The code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions; When the state of the code stream buffer satisfies a preset condition, performing a first operation on the estimated number of bits of the current block to be encoded includes: Obtaining the estimated number of bits of M blocks to be encoded before the current block to be encoded; The available buffer size is determined according to the state of the code stream buffer and the sum of the reference change value; the reference change value is the difference between the fixed outflow bit number of the M+1 blocks of the code stream buffer and the estimated inflow bit number of the M+1 blocks of the code stream buffer; the fixed outflow bit number of the M+1 blocks of the code stream buffer is the product of the number of pixels of each block to be encoded, the preset pixel depth, and M+1; the estimated inflow bit number of the M+1 blocks of the code stream buffer is the sum of the estimated bit number of the current block to be encoded and the estimated bit number of each of the M blocks to be encoded before the current block to be encoded; the available buffer size is used to represent the available size of the code stream buffer after the current block to be encoded is encoded by the estimated bit number; Obtaining an additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; When the available buffer size is less than the additional buffer threshold, the first operation is performed on the estimated number of bits of the current block to be encoded.

2. The method according to claim 1, characterized in that For a historical block, the historical block is any one of the M blocks to be encoded before the current block to be encoded, and obtaining the estimated number of bits of the M blocks to be encoded before the current block to be encoded includes: Obtaining the complexity level of the historical block and the complexity level of the current block to be encoded; The estimated number of bits of the historical block is determined based on the quotient of the complexity level of the historical block and a reference value; the reference value is the product of the complexity level of the current block to be encoded and the estimated number of bits of the current block to be encoded.

3. The method according to claim 2, characterized in that The obtaining the complexity level of the current block to be encoded includes: Obtain a current block to be encoded; the current block to be encoded includes three channels; Obtaining complexity information of the coding block of each of the three channels; the complexity information is used to characterize the degree of difference of pixel values ​​of the coding block of each channel; The complexity level of the current block to be encoded is determined according to the complexity level of the encoding block of each channel.

4. The method according to claim 3, characterized in that The method further comprises: The quantization parameters of the three channels are modified according to the complexity information of the first channel; the first channel is any one of the three channels.

5. The method according to any one of claims 1 to 4, characterized in that: The encoding the current block to be encoded according to the quantization parameter includes: Analyze residual coefficients based on prediction mode and quantization parameters; Obtain the predicted value of each pixel of the current block to be decoded based on the prediction mode; Obtaining residual values ​​of each pixel of the current block to be encoded based on residual coefficient encoding; The current block to be encoded is encoded based on the prediction value and the residual value of each pixel point of the current block to be encoded.

6. The method according to claim 5, characterized in that The prediction mode is an anti-expansion mode, and the anti-expansion mode is a mode with the largest bit cost among all prediction modes.

7. The method according to claim 6, characterized in that The method further comprises: The luminance channel, the first chrominance channel, and the second chrominance channel are encoded according to the bit depth of the luminance channel, and the to-be-encoded block of the luminance channel, the to-be-encoded block of the first chrominance channel, and the to-be-encoded block of the second chrominance channel are encoded according to the anti-expansion mode.

8. The method according to any one of claims 1 to 4, characterized in that: Encoding the current block to be encoded based on the anti-expansion mode includes: Before encoding the slice where the current block to be encoded is located, the hot start array of the code control, the target fullness parameter at the end, and the additional buffer threshold parameter are initialized.

9. An image decoding method, characterized in that: The method comprises: Obtaining the state of a code stream buffer; the code stream buffer is used to store the code stream after the block to be decoded is decoded; the state of the code stream buffer is the number of bits after the Mth block to be decoded before the current block to be decoded is decoded in the code stream buffer; M is a positive integer; When the state of the code stream buffer satisfies a preset condition, performing a first operation on the estimated number of bits of the current block to be decoded; the estimated number of bits is the number of bits required for decoding the current block to be decoded predicted under the state of the code stream buffer; the first operation refers to reducing the estimated number of bits by a penalty bit number; the penalty bit number is obtained based on a penalty parameter; Determine a quantization parameter of a current block to be decoded based on the estimated number of bits obtained after performing the first operation, and decode the current block to be decoded according to the quantization parameter; The code stream buffer includes a main buffer and an additional buffer; the main buffer and the additional buffer are different physical storage partitions, or different logical storage partitions; When the state of the code stream buffer satisfies a preset condition, performing a first operation on the estimated number of bits of the current block to be decoded includes: Obtaining the estimated number of bits of M blocks to be decoded before the current block to be decoded; The available buffer size is determined according to the state of the code stream buffer and the sum of the reference change value; the reference change value is the difference between the fixed number of inflow bits of the M+1 blocks of the code stream buffer and the estimated number of outflow bits of the M+1 blocks of the code stream buffer; the fixed number of outflow bits of the M+1 blocks of the code stream buffer is the product of the number of pixels of each block to be decoded, the preset pixel depth, and M+1; the estimated number of inflow bits of the M+1 blocks of the code stream buffer is the sum of the estimated number of bits of the current block to be decoded and the estimated number of bits of each of the M blocks to be decoded before the current block to be decoded; the available buffer size is used to represent the predicted available size of the code stream buffer after the current block to be decoded is decoded using the estimated number of bits; Acquire the additional buffer threshold; the additional buffer threshold is used to characterize the available space of the additional buffer; When the available buffer size is less than the additional buffer threshold, the first operation is performed on the estimated number of bits of the current block to be decoded.

10. The method according to claim 9, characterized in that For a historical block, the historical block is any one of the M blocks to be decoded before the current block to be decoded, and obtaining the estimated number of bits of the M blocks to be decoded before the current block to be decoded includes: Obtaining the complexity level of the historical block and the complexity level of the current block to be decoded; The estimated number of bits of the historical block is determined based on the quotient of the complexity level of the historical block and a reference value; the reference value is the product of the complexity level of the current block to be decoded and the estimated number of bits of the current block to be decoded.

11. The method according to claim 10, characterized in that The obtaining the complexity level of the current block to be decoded includes: Obtain a current block to be decoded; the current block to be decoded includes three channels; Obtaining complexity information of a decoding block of each of the three channels; the complexity information is used to characterize the degree of difference of pixel values ​​of the decoding block of each channel; The complexity level of the current block to be decoded is determined according to the complexity level of the decoding block of each channel.

12. The method according to claim 11, characterized in that The method further comprises: The quantization parameters of the three channels are modified according to the complexity information of the first channel; the first channel is any one of the three channels.

13. The method according to any one of claims 9 to 12, characterized in that: The decoding the current block to be decoded according to the quantization parameter includes: Analyze residual coefficients based on prediction mode and quantization parameters; Obtain the predicted value of each pixel of the current block to be decoded based on the prediction mode; Obtaining residual values ​​of each pixel of the current block to be decoded based on residual coefficient decoding; The current block to be decoded is decoded based on the prediction value and the residual value of each pixel point of the current block to be decoded.

14. The method according to claim 13, characterized in that The prediction mode is an anti-expansion mode, and the anti-expansion mode is a mode with the largest bit cost among all prediction modes.

15. The method according to claim 14, characterized in that The method comprises: The luminance channel, the first chrominance channel, and the second chrominance channel are decoded according to the bit depth of the luminance channel, and the to-be-decoded blocks of the luminance channel, the to-be-decoded blocks of the first chrominance channel, and the to-be-decoded blocks of the second chrominance channel are decoded according to the anti-expansion mode.

16. The method according to any one of claims 9 to 12, characterized in that: Decoding the current block to be decoded based on the anti-expansion mode includes: Before decoding the stripe where the current block to be decoded is located, the hot start array, the target fullness parameter at the end, and the additional buffer threshold parameter of the code control are initialized.

17. An image encoding device, characterized in that: The device comprises: a processor and a memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the image encoding device implements the method according to any one of claims 1 to 8.

18. An image decoding device, characterized in that: The device comprises: a processor and a memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the image decoding device implements the method according to any one of claims 9 to 16.

19. A readable storage medium, characterized in that: The readable storage medium includes: computer program instructions; When the computer program instructions are executed in an image coding and decoding device, the image coding and decoding device implements the method according to any one of claims 1 to 16.

Images