Image Encoding, Decoding Method and Device

Abstract

The present invention relates to the field of video encoding and decoding, and provides an image encoding, decoding method and apparatus that can be used to prevent overflow of a bitstream. The specific method includes: determining whether a fallback mode is used for an image block corresponding to a block to be decoded based on the bitstream of the block to be decoded, where the code length of the block to be decoded of the image block is less than or equal to the maximum code length of the block to be decoded, and the maximum code length is the maximum bitstream length that is allowed to be cached in the bitstream buffer; when the fallback mode is used for the image block corresponding to the block to be decoded, obtaining a first fixed code length, where the first fixed code length is the code length of one sub-block to be decoded, and the block to be decoded includes one or more sub-blocks to be decoded; and analyzing the bitstream based on the first fixed code length to decode the block to be decoded.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of video encoding and decoding, and in particular, to image encoding, decoding methods and apparatuses.

Background Art

[0002] Video encoding technology is also called video compression technology, and video compression technology is used to reduce the data bandwidth of video signals. Here, video is a continuous sequence of images and consists of continuous image frames. Due to the visual persistence effect of the human eye, when the frame sequence is played back at a certain speed, what the human eye sees is continuous video. Encoding and decoding video is to encode and decode the images of each frame of the video. Taking one frame of image as an example, on the encoding side, the image encoder encodes the image, obtains the bitstream corresponding to the image, and sends the bitstream to the decoding side. On the decoding side, the image decoder analyzes the bitstream to reconstruct the image.

[0003] In the process of image encoding and decoding, in order to improve the compression efficiency, general video compression technology adopts variable-length coding. In variable-length coding, the bitstream obtained after a video sequence is compressed and encoded has a varying rate and is not suitable for real-time transmission over a fixed-bitrate channel. Therefore, usually, the variable-bitrate data generated by the encoder is output to a buffer with a specific storage space, and then output from the buffer at a fixed bitrate. The size of the storage space of the buffer is fixed. If the bitstream data input to the buffer is too large and the data that needs to be temporarily stored in the buffer exceeds the size of its storage space, "overflow" of the bitstream occurs and subsequent image information is lost. Also, if the data of the bitstream input to the buffer is too small and the data temporarily stored in the buffer is "insufficient" and cannot meet the fixed output bitrate of the buffer, "underflow" of the bitstream occurs.

[0004] Therefore, how to avoid the "overflow" or "underflow" of the bitstream in the image encoding and decoding processes is a problem to be solved.

Summary of the Invention

[0005] Embodiments of the present invention provide an image encoding, decoding method, and apparatus that can be used to prevent the overflow of a bitstream.

[0006] To achieve the above object, embodiments of the present invention adopt the following technical solutions.

[0007] In a first aspect, an embodiment of the present invention is an image decoding method executed by a decoding side, including: determining whether a fallback mode is used for an image block corresponding to the block to be decoded based on a bitstream of the block to be decoded, where a code length of the block to be decoded is less than or equal to a maximum code length of the block to be decoded, and the maximum code length is a maximum bitstream length that is allowed to be cached in a bitstream buffer; when the fallback mode is used for the image block corresponding to the block to be decoded, obtaining a first fixed code length, where the first fixed code length is a code length of one sub-block to be decoded, and the block to be decoded includes one or more sub-blocks to be decoded; and parsing the bitstream based on the first fixed code length to decode the block to be decoded.

[0008] In a second aspect, an embodiment of the present invention is an image encoding method executed by an encoding side, comprising: obtaining a maximum code length of a block to be encoded, where the maximum code length is the maximum bitstream length that is allowed to be cached in a bitstream buffer; pre-encoding the block to be encoded based on a first mode to obtain a first code length of the block to be encoded, where the first mode is one of a plurality of prediction modes on the encoding side, and the first code length is the length of the bitstream obtained after encoding the block to be encoded; and when the first code length is greater than or equal to the maximum code length, encoding the block to be encoded in a fallback mode, where the code length obtained by encoding the block to be encoded based on the fallback mode is less than or equal to the maximum code length.

[0009] In a third aspect, an embodiment of the present invention is an image decoding method executed by a decoding side, comprising: determining whether a fallback mode is used during encoding for an image block corresponding to a block to be decoded based on the bitstream of the block to be decoded, where the code length obtained when the image block is encoded based on the fallback mode is greater than or equal to the minimum code length of the block to be decoded, and the minimum code length is the minimum bitstream length that is allowed to be cached in a bitstream buffer; when the fallback mode is used during encoding for the image block corresponding to the block to be decoded, obtaining a first fixed code length based on the bitstream, where the first fixed code length is the code length of one sub-block to be decoded, and the block to be decoded includes one or more sub-blocks to be decoded; and parsing the bitstream based on the first fixed code length to decode the block to be decoded.

[0010] In a fourth aspect, an embodiment of the present invention is an image encoding method applied to an encoding side, the method including: obtaining a minimum code length of a block to be encoded, where the minimum code length is a minimum bitstream length that is allowed to be cached in a bitstream buffer; pre-encoding the block to be encoded based on a first mode to obtain a first code length of the block to be encoded, where the first mode is one of a plurality of prediction modes on the encoding side, and the first code length is a length of a bitstream obtained after encoding the block to be encoded; and when the first code length is less than or equal to the minimum code length, encoding the block to be encoded in a fallback mode, where a code length obtained by encoding the block to be encoded based on the fallback mode is greater than or equal to the minimum code length.

[0011] In a fifth aspect, an embodiment of the present invention is an image decoding method applied to a decoding side, the method including: analyzing a bitstream of a block to be decoded to determine whether a skip residual mode is used; when the skip residual mode is used, determining a second mode of the block to be decoded, where the second mode is one of a plurality of prediction modes on the decoding side; determining a predicted value of the block to be decoded based on the second mode; and determining the predicted value of the block to be decoded as a reconstructed value of the block to be decoded.

[0012] In a sixth aspect, an embodiment of the present invention is an image encoding method applied to an encoding side, including: obtaining a maximum code length of a block to be encoded, where the maximum code length is a maximum bitstream length that is allowed to be cached in a bitstream buffer; pre-encoding the block to be encoded based on a first mode to obtain a first code length of the block to be encoded, where the first mode is one of a plurality of prediction modes on the encoding side, and the first code length is a length of a bitstream obtained after encoding the block to be encoded; and when the first code length is greater than or equal to the maximum code length, encoding a block to be encoded in a second mode based on a skip residual mode, where a code length obtained by encoding the block to be encoded in the second mode based on the skip residual mode is less than or equal to the maximum code length, and the second mode is one of a plurality of prediction modes on the encoding side.

[0013] In a seventh aspect, an embodiment of the present invention is an image decoding method applied to a decoding side, comprising: determining whether a fallback mode is used during decoding for an image block corresponding to the block to be decoded based on a bitstream of the block to be decoded, wherein a code length of the block to be decoded is less than or equal to a maximum code length of the block to be decoded, and the maximum code length is a maximum bitstream length that is allowed to be cached in a bitstream buffer; when the fallback mode is used during decoding for the image block corresponding to the block to be decoded, obtaining a first fixed code length, wherein the first fixed code length is a code length of one sub-block to be decoded, the block to be decoded includes one or more sub-blocks to be decoded, the first fixed code length is a target bit per pixel of the block to be decoded, and the target bit per pixel is used to indicate a code length required to decode each pixel of the block to be decoded at a target compression ratio; dividing the block to be decoded into a plurality of sub-blocks to be decoded based on the code length of the block to be decoded and the first fixed code length; and analyzing a bitstream of each sub-block to be decoded of the block to be decoded based on the first fixed code length to decode the block to be decoded.

[0014] In some embodiments, the block to be decoded includes a first chrominance component, a second chrominance component, and a luminance component, and the step of determining whether the fallback mode is used during decoding for the image block corresponding to the block to be decoded based on the bitstream of the block to be decoded includes analyzing the luminance component to determine whether the fallback mode is used during decoding for the first chrominance component, the second chrominance component, and the luminance component.

[0015] In some embodiments, a sum of a code length of the first chrominance component, a code length of the second chrominance component, and a code length of the luminance component is the code length of the block to be decoded, and the code length of the luminance component is greater than or equal to the code length of the first chrominance component, or the code length of the luminance component is greater than or equal to the code length of the second chrominance component.

[0016] In some embodiments, based on the first fixed code length, the step of decoding the block to be decoded by analyzing the bit stream of each sub-block to be decoded of the block to be decoded includes: based on the first fixed code length, analyzing the bit stream of each sub-block to be decoded of the block to be decoded to obtain a residual value of each sub-block to be decoded; and based on the predicted value and the residual value of each sub-block to be decoded, reconstructing each sub-block to be decoded to obtain a reconstructed block of each sub-block to be decoded.

[0017] In some embodiments, the method further includes: when a codeword for indicating a prediction mode in the bit stream of the block to be decoded is a first codeword, and a fallback mode is not used during decoding for the image block corresponding to the block to be decoded, reconstructing the image block of the block to be decoded based on the original value mode, where the first codeword is used to indicate the fallback mode or the original value mode.

[0018] In an eighth aspect, an embodiment of the present invention is an image encoding method applied to an encoding side, the method including: obtaining a maximum code length of a block to be encoded, where the maximum code length is a maximum bitstream length that is allowed to be cached in a bitstream buffer; pre-encoding the block to be encoded based on a first mode to obtain a first code length of the block to be encoded, where the first mode is one of a plurality of prediction modes on the encoding side, and the first code length is a length of a bitstream obtained after encoding the block to be encoded; and when the first code length is greater than or equal to the maximum code length, encoding the block to be encoded based on a fixed code length in a fallback mode, where a code length obtained by encoding the block to be encoded based on the fallback mode is less than or equal to the maximum code length, the fixed code length is a target bit per pixel of the block to be encoded, and the target bit per pixel is used to indicate a code length required when encoding each pixel of the block to be encoded at a target compression ratio.

[0019] In a 9 of certain aspect, the present invention provides an image decoding apparatus. The decoding apparatus may be a video decoder or a device including a video decoder. The decoding apparatus may be in accordance with the first aspect, the third aspect , the and the fifth aspect , or the seventh aspect It includes each module for implementing any one of the decoding methods. The decoding device can implement the operations in the examples of the related method. The operations may be implemented by hardware or may be implemented by the hardware executing the corresponding software. The hardware or software includes one or more modules corresponding to the above operations.

[0020] The 10 of In an aspect, the present invention provides an image encoding device. The encoding device may be a video encoder or a device including a video encoder. The encoding device includes each module for implementing any one of the encoding methods in the second aspect, the fourth aspect , the Aspect 6 , or the eighth aspect It includes each module for implementing any one of the encoding methods. The encoding device can implement the operations in the examples of the related method. The operations may be implemented by hardware or may be implemented by the hardware executing the corresponding software. The hardware or software includes one or more modules corresponding to the above operations.

[0021] The 11 of In an aspect, the present invention provides an electronic device including a processor and a memory. The memory is used to store computer instructions, and the processor calls and executes the computer instructions from the memory to implement the method described in any one of the first aspect to the 8 of aspect. For example, the electronic device may be a video encoder or a device including a video encoder. In another example, the electronic device may be a video decoder or a device including a video decoder.

[0022] The 12 In an aspect, the present invention provides a computer-readable storage medium storing a computer program or instructions. When the computer program or instructions are executed by a computing device or a processor, the first aspect to the 8 of Provide a computer-readable storage medium in which the method according to any one of the aspects is implemented.

[0023] First 3 of In one aspect, the present invention is a computer program product including instructions, and when the computer program product is executed on a computing device or a processor, the computing device or the processor is caused to execute the instructions to implement the method according to any one of the first aspect to the 8 of Provide a computer program product that causes the method according to any one of the aspects to be implemented.

[0024] First 4 of In one aspect, the present invention is a chip including a memory and a processor, where the memory is used to store computer instructions, and the processor calls and executes the computer instructions from the memory to implement the method according to any one of the first aspect to the 8 of Provide a chip used to implement the method according to any one of the aspects.

[0025] First 5 of In one aspect, the present invention is an image coding system including an encoding side and a decoding side, where the decoding side is used to implement the corresponding decoding method provided in the first aspect, the third aspect , the Aspect 5 , or the seventh aspect and the encoding side is used to implement the corresponding encoding method. Provide an image coding system.

[0026] The present invention may be further combined to provide more implementation manners based on the implementation manners provided in the above aspects. Or, any possible implementation manner of any of the above aspects may be applied to other aspects to obtain new embodiments as long as there is no contradiction. For example, the first aspect, the third aspect , the Aspect 5 , and the seventh aspect Any of the image decoding methods provided in may be combined in pairs or in combinations of three aspects so as not to contradict each other, thereby obtaining a new image decoding method.

[0027] According to the image encoding / decoding method and apparatus provided by the present invention, when it is determined that an overflow of the bitstream occurs, by using a fallback mode to encode the block to be encoded, the occurrence of an overflow of the bitstream can be prevented, and the loss of image information of the block to be encoded can be prevented.

Brief Description of the Drawings

[0028]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Embodiments for Carrying Out the Invention

[0029] Hereinafter, with reference to the drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments that can be obtained by those skilled in the art without creative efforts all belong to the protection scope of the present invention.

[0030] In the description of the present invention, unless otherwise specified, " / " means "or". For example, A / B can represent A or B. The "and / or" in this specification is for explaining the relationship of relevant objects, indicating that there may be three types of relationships. For example, A and / or B can represent three cases: when A exists alone, when A and B exist simultaneously, and when B exists alone. "At least one" means one or more, and "a plurality" means two or more. Terms such as "first" and "second" do not limit the number or execution order, and are not necessarily limited to being different.

[0031] In addition, in the present invention, terms such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or technical solution described as "exemplary" or "for example" in the present invention should not be construed as being more preferable or advantageous than other embodiments or technical solutions. Strictly speaking, the use of terms such as "exemplary" or "for example" is intended to show related concepts in a specific manner.

[0032] Hereinafter, the system architecture applicable to the embodiments of the present invention will be described.

[0033] Referring to FIG. 1, FIG. 1 shows a schematic architecture diagram of an encoding and decoding system 1 to which the embodiments of the present invention are applicable. As shown in FIG. 1, the encoding and decoding system 1 may include an encoding side 10 and a decoding side 20. Here, the encoding side 10 generates encoded video data. Therefore, the encoding side 10 may be called a video encoding device. The decoding side 20 can decode the encoded video data generated by the encoding side 10. Therefore, the decoding side 20 may be called a video decoding device.

[0034] The specific forms of the encoding side 10 and the decoding side 20 may be various devices including a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a mobile phone such as a "smart" phone, a television, a camera, a display device, a digital media player, a video game console, an in-vehicle computer, or the like.

[0035] Optionally, the encoding side 10 and the decoding side 20 of FIG. 1 may be two separate devices. Or, the encoding side 10 and the decoding side 20 may be the same device, that is, the encoding side 10 or its corresponding function and the decoding side 20 or its corresponding function may be integrated into the same device.

[0036] The encoding side 10 and the decoding side 20 are communicable with each other. For example, the decoding side 20 may receive the encoded video data from the encoding side 10 via the link 30. The link 30 may include one or more media or devices capable of transmitting the encoded video data from the encoding side 10 to the decoding side 20. In one example, the link 30 may include one or more communication media that enable the encoding side 10 to directly transmit the encoded video data to the decoding side 20 in real time. In this example, the encoding side 10 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to the decoding side 20. The one or more communication media may include wireless and / or wired communication media such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include a router, a switch, a base station, or another device that facilitates communication from the encoding side 10 to the decoding side 20.

[0037] Optionally, the encoding side 10 may output the encoded video data from the output interface 140 to a built-in or external storage device. The storage device may include any one of a plurality of distributed or locally accessible data storage media such as a hard disk drive, a Blu-ray disc, a digital video disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing the encoded video data.

[0038] As shown in FIG. 1, the encoding side 10 includes a video source 120, an encoder 100, and an output interface 140. In some examples, the output interface 140 may include a modulator / demodulator (modem) and / or a transmitter. The video source 120 may include a video capture device (e.g., a camera), a video archive including previously captured video data, a video feed interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or any combination of the above sources of video data. The encoder 100 can encode the video data from the video source 120. In some examples, the encoding side 10 directly transmits the encoded video data to the decoding side 20 via the output interface 140. In other examples, the encoded video data may be stored in a storage device for the decoding side 20 to access for decoding and / or playback.

[0039] As shown in FIG. 1, the decoding side 20 includes an input interface 240, a decoder 200, and a display device 220. In some examples, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive encoded video data via the link 30 and / or from a storage device. The display device 220 may be integrated with the decoding side 20 or provided outside the decoding side 20. Generally, the display device 220 displays the decoded video data. The display device 220 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0040] Optionally, the encoding side 10 may further include a bitstream buffer (not shown in FIG. 1), which receives the bitstream data generated by encoding and is used to output fixed bitrate bitstream data. The size of the bitstream data generated after different blocks to be encoded are encoded may vary greatly based on the nature of the blocks to be encoded. Therefore, the bitstream obtained after a video sequence is compressed and encoded has a varying rate and is not suitable for real-time transmission on a fixed bitrate channel. Through the bitstream buffer, the rate change in the compressed video can be smoothed. The larger the storage space of the bitstream buffer, the more it can withstand bitrate fluctuations.

[0041] Optionally, the decoding side 20 may include another buffer for receiving fixed bitrate bitstream data and outputting it from the buffer to the bitstream analysis unit 201 of the decoder 200.

[0042] In an example of the present invention, the encoding side and the decoding side can always record the state of the bitstream buffer throughout the encoding / decoding process. Therefore, when encoding / decoding an image block, both the encoding side and the decoding side can obtain the maximum bitstream length that can be cached in the bitstream buffer. Regarding the process in which the encoding side and the decoding side record the state of the bitstream buffer, as an option, if only the encoding side includes the bitstream buffer, the encoding side and the decoding side may always record the state of the encoding-side bitstream buffer. If only the decoding side includes the bitstream buffer, reference may be made to the case of the encoding-side bitstream buffer above. If both the encoding side and the decoding side include the bitstream buffer, the encoding side and the decoding side may record the state of the encoding-side bitstream buffer simultaneously, may record the state of the decoding-side bitstream buffer simultaneously, may record the states of their respective bitstream buffers, may record the states of each other's bitstream buffers, or may record the states of both of the two bitstream buffers simultaneously. The present invention does not limit the process of recording the state of the bitstream buffer as long as the encoding side and the decoding side can obtain the maximum bitstream length that can be cached in the bitstream buffer of the encoding side and / or the decoding side when encoding / decoding an image block.

[0043] Although not shown in FIG. 1, in some embodiments, the encoder 100 and the decoder 200 may each be integrated with an audio encoder and decoder and may include a suitable multiplexer-demultiplexer unit or other hardware and software to handle the encoding of both audio and video in a common data stream or separate data streams.

[0044] The encoding and decoding system 1 shown in FIG. 1 is merely an example, and it should be understood that the technology of the present invention is applicable to video coding settings (e.g., video encoding or video decoding) that do not necessarily require any data communication between the encoding device and the decoding device. In other examples, it is also possible to obtain data from local memory or stream it via a network. The video encoding device can encode data and store it in memory, and / or the video decoding device can read data from memory and decode it. In many examples, encoding and / or decoding may be performed by devices that only encode data into memory and / or retrieve and decode it from memory without communicating with each other.

[0045] Hereinafter, with reference to the drawings, the specific structures of the encoder 100 and the decoder 200 in FIG. 1 will be briefly described.

[0046] Referring to FIG. 2, FIG. 2 shows a schematic block diagram of an example of an encoder 100 for implementing the method of an embodiment of the present invention. As shown in FIG. 2, the encoder 100 includes a prediction processing unit 101, a residual calculation unit 102, a residual transformation unit 103, a quantization unit 104, an encoding processing unit 105, an inverse quantization unit 106, a residual inverse transformation unit 107, a reconstruction unit 108, and a filter unit 109.

[0047] In one example, the input to the encoder 100 is an image block of the image to be encoded (i.e., the block to be encoded or the encoding unit).

[0048] In another example, when the input to the encoder 100 is an image to be encoded, the encoder 100 may further include a splitting unit (not shown in FIG. 2) for splitting the image to be encoded into a plurality of image blocks. The encoder 100 is used to encode the plurality of image blocks of the image to be encoded block by block to complete the encoding of the image to be encoded. For example, the encoder 100 executes an encoding process for each image block to complete the encoding of the image to be encoded.

[0049] Exemplarily, a method of dividing an image to be encoded into a plurality of image blocks may include the following steps.

[0050] In step 11, an image of one frame is divided into one or more non-overlapping parallel encoding units. Each parallel encoding unit is independent of each other and can be encoded and decoded completely in parallel / independently.

[0051] In step 12, for each parallel encoding unit, on the encoding side, it may be further divided into one or more non-overlapping independent encoding units. Each independent encoding unit may not be dependent on each other, but may share some header information of the parallel encoding unit.

[0052] In step 13, for each independent encoding unit, on the encoding side, it may be further divided into one or more non-overlapping encoding units. When dividing an independent encoding unit into a plurality of non-overlapping encoding units, the division method may be horizontal equal division, vertical equal division, or horizontal and vertical equal division, etc. Of course, the specific implementation is not limited to these. Each encoding unit within the independent encoding unit may be dependent on each other, that is, they may refer to each other in the process of executing prediction processing.

[0053] The width of the encoding unit is w_cu, and the height is h_cu. Optionally, its width is larger than the height (except for some edge regions where its width may be less than or equal to the height). Usually, the encoding unit may be a fixed w_cu×h_cu, and both w_cu and h_cu are powers of 2 (N is 0 or more), for example, 16×4, 8×4, 16×2, 8×2, 4×2, 8×1, 4×1, etc.

[0054] The symbolization unit may include three components of luminance Y, chrominance Cb, and chrominance Cr (or three components of red R, green G, and blue B, or three components of luminance Y, chrominance U, and chrominance V), or may include only any one of these components. When including three components, the sizes of these components may be exactly the same or different, specifically related to the input format of the image.

[0055] Figure 3 is a schematic diagram of the correspondence relationship between an image, a parallel encoding unit, an independent encoding unit, and an encoding unit. In Figure 3, taking the example of dividing one image into parallel encoding unit 1 and parallel encoding unit 2 according to 3:1, and one independent encoding unit including four encoding units for explanation.

[0056] Optionally, the splitting unit may split the image to be encoded into a plurality of image blocks, and these image blocks may be further split into smaller blocks such as image blocks based on, for example, a quadtree structure or a binary tree structure. Also, this splitting may include splitting into slices, tiles, or other larger units. The slice may be split into a plurality of image blocks (or split into a set of image blocks called tiles).

[0057] The prediction processing unit 101 receives or obtains the original value of the block to be encoded and the reconstructed image data, predicts the block to be encoded based on the relevant data in the reconstructed image data, and is used to obtain the predicted block of the block to be encoded.

[0058] Optionally, the prediction processing unit 101 may include an inter-frame predictor and an intra-frame predictor. The inter-frame predictor determines an inter-frame prediction mode for encoding the block to be encoded, and based on the determined inter-frame prediction mode, predicts the motion information of one or more sub-blocks within the current image block, for example, a motion vector (MV), and is used to obtain or generate a prediction block of the current image block using the motion information (e.g., motion vector) of one or more sub-blocks within the current image block. The intra-frame predictor may determine an intra-frame prediction mode for encoding the block to be encoded.

[0059] Intra-frame prediction is to use the spatial correlation of the video to predict using the encoded block (coding block, CB) of the current block in order to reduce the spatial redundancy of the video. Exemplarily, intra-frame prediction specifies a plurality of prediction modes, and each prediction mode corresponds to one texture direction (excluding the DC mode). For example, when the texture of the image is horizontal, the horizontal prediction mode can predict the image information better.

[0060] Inter-frame prediction is to predict the pixels of the current image using the pixels of the neighboring encoded images based on the temporal correlation of the video because the video sequence contains strong temporal correlation, and can effectively reduce the temporal redundancy of the video. The inter-frame prediction part of the video coding standard uses block-based motion compensation technology, and its main principle is to find one optimal matching block in the previous encoded image for each pixel block of the current image, and this process is called motion estimation (ME).

[0061] Also, in the inter-frame prediction mode, the motion vector can represent the relative displacement between the current encoded block and the optimal matching block in its reference image. Each divided block has a corresponding motion vector transmitted to the decoding side. When encoding and transmitting the motion vectors of each block independently, especially when the blocks are divided into small sizes, a significant number of bits are consumed. To reduce the number of bits for encoding the motion vectors, the spatial correlation between adjacent image blocks is utilized to predict the motion vector of the current block to be encoded based on the motion vectors of adjacent encoded blocks, and the prediction difference is encoded. Thereby, the number of bits representing the motion vector can be effectively reduced. In the process of encoding the motion vector of the current block, first, the motion vector of the current block is predicted using the motion vectors of adjacent encoded blocks, and then, by encoding the difference (motion vector difference, MVD) between the predicted value of the motion vector (motion vector prediction, MVP) and the actual estimated value of the motion vector, the number of encoded bits of the MV is effectively reduced.

[0062] The residual calculation unit 102 is used to calculate the residual value between the original value of the block to be encoded and the predicted block of the block to be encoded to obtain a residual block. For example, for each pixel, the pixel value of the predicted block is subtracted from the original pixel value of the block to be encoded.

[0063] In one example, the residual transformation unit 103 is used to determine residual coefficients based on a residual block. Optionally, the process may include performing a transformation such as a discrete cosine transform (DCT) or a discrete sine transform (DST) on the residual block to obtain transform coefficients in the transform domain. The transform coefficients may be referred to as transform residual coefficients or residual coefficients, and the residual coefficients can represent the residual block in the transform domain. Of course, the process of encoding the block to be encoded by the encoder 100 may not include the step of residual transformation.

[0064] The quantization unit 104 is used to apply scalar quantization or vector quantization to quantize the transform coefficients or the residual values to obtain quantized residual coefficients (or quantized residual values). The quantization process can reduce the bit depth for some or all of the transform coefficients. For example, during quantization, a p-bit transform coefficient may be truncated to a q-bit transform coefficient, where p is greater than q. The degree of quantization can be changed by adjusting the quantization parameter (QP). For example, in the case of scalar quantization, fine quantization or coarse quantization can be achieved by applying different scales. A small quantization step corresponds to fine quantization, and a large quantization step corresponds to coarse quantization. An appropriate quantization step may be indicated by the QP.

[0065] Here, in the image encoding process, in order to achieve compression of an image, generally, quantization is performed on the residual block of the block to be encoded, or quantization is performed on the residual coefficient block obtained by performing specific processing on the residual block, so that the quantized residual block or residual coefficient block can be encoded with fewer bits. As can be understood, the residual block is a residual value block obtained based on the original pixel block and the prediction block of the block to be encoded, and the residual coefficient block is a coefficient block obtained by performing specific processing and transformation on the residual block.

[0066] Exemplarily, taking the case where the encoder 100 quantizes the residual block as an example, the encoder 100 may divide each residual value in the residual block of the block to be encoded by a quantization coefficient in order to reduce the residual value in the residual block. In this way, the quantized and reduced residual value can be encoded with fewer bits compared to the non-quantized residual value, and lossy compression encoding of the image is achieved.

[0067] The encoding processing unit 105 is used to encode the quantized residual coefficient (or quantized residual value) and output the encoded image data in the form of an encoded bitstream (i.e., the encoding result of the currently block to be encoded). Then, the encoded bitstream may be transmitted to the decoder, or stored for later transmission to the decoder or for use in searching. The encoding processing unit 105 may further be used to encode the syntax elements of the block to be encoded, such as encoding the prediction mode used for the block to be encoded into the bitstream.

[0068] In one example, one possible way for the encoding processing unit 105 to encode the residual coefficients is the semi-fixed length encoding method. First, define the maximum value of the absolute value of the residuals within one residual block (RB) as the modified maximum (mm). Determine the number of encoding bits for the residual coefficients within the RB (the number of encoding bits for the residual coefficients within the same RB is the same). For example, if the fixed length code encoding length (CL) of the current RB is 2 and the current residual coefficient is 1, 2 bits are required to encode the residual coefficient 1, which is represented as 01. When the CL of the current RB is 7, the range of residuals that can be represented is [-2 6 , -2 6 -1] or [-2 6 +1, -2 6 . The determination of CL is to find the minimum M value that satisfies the condition that all the residuals of the current sub-block are within the range of [-2 (M-1) , 2 (M-1) . If within this range, two boundary values of -2 (M-1) and 2 (M-1) exist simultaneously, it is necessary to increase the value M corresponding to CL by 1, that is, M + 1 bits are required to encode all the residuals of the current RB. If only one of the two boundary values of -2 (M-1) and 2 (M-1) exists, it is necessary to encode one Trailing bit to determine whether the boundary value is -2 (M-1) or 2 (M-1) . If neither -2 (M-1) nor 2 (M-1) exists for all the residuals, there is no need to encode the Trailing bit.

[0069] Of course, other residual coefficient encoding methods, such as the exponential Golomb encoding method, the Golomb-Rice encoding method, the truncated unary encoding method, the run-length encoding method, the method of directly encoding the original residual value, etc. may also be used.

[0070] In some special cases, the original value may be directly encoded instead of the residual value.

[0071] The inverse quantization unit 106 is used to inverse-quantize the quantized residual coefficient (or quantized residual value) to obtain an inverse-quantized residual coefficient (inverse-quantized residual value). The inverse quantization is the inverse application of the quantization unit 104. For example, based on or using the same quantization step as the quantization unit 104, an inverse quantization scheme corresponding to the quantization scheme applied by the quantization unit 104 is applied.

[0072] The residual inverse transformation unit 107 is used to inverse-transform the inverse-quantized residual coefficient to obtain a reconstructed residual block. Optionally, the inverse transformation may include an Inverse Discrete Cosine Transform (IDCT) or an Inverse Discrete Sine Transform (IDST). In this way, the inverse-transformed value obtained by inverse-transforming the inverse-quantized residual coefficient is a residual value reconstructed in the pixel domain (or called the sample domain). That is, after the inverse-quantized residual coefficient block is inverse-transformed by the residual inverse transformation unit 107, the obtained block is a reconstructed residual block. Of course, if the encoder 100 does not include the above residual transformation unit 103, the inverse transformation step may not be included.

[0073] The reconstruction unit 108 is used to add the reconstructed residual block to the prediction block to obtain a reconstruction block in the sample domain. The reconstruction unit 108 may be an adder. For example, the reconstruction unit 108 adds the residual value in the reconstructed residual block to the predicted value of the corresponding pixel in the prediction block to obtain the reconstructed value of the corresponding pixel. The reconstruction block output by the reconstruction unit 108 may be used to predict other image blocks to be encoded in subsequent steps.

[0074] The filter unit 109 (or abbreviated as "filter") is used to filter the reconstruction block to obtain a filtered block in order to perform pixel conversion smoothly or to improve the quality of the image.

[0075] In one example, the encoding process performed by the encoder 100 may include the following steps.

[0076] In step 21, the prediction processing unit 101 determines a prediction mode, predicts the block to be encoded based on the determined prediction mode and the reconstruction block of the encoded image block, and obtains a prediction block of the block to be encoded.

[0077] Here, the reconstruction block of the encoded image block is obtained by the inverse quantization unit 106, the residual inverse transformation unit 107, and the reconstruction unit 108 processing the quantized residual coefficient block of the encoded image block in sequence.

[0078] In step 22, the residual calculation unit 102 obtains a residual block of the block to be encoded based on the prediction block and the original pixel values of the block to be encoded.

[0079] In step 23, the residual transformation unit 103 transforms the residual block to obtain a residual coefficient block.

[0080] In step 24, the quantization unit 104 quantizes the residual coefficient block to obtain a quantized residual coefficient block.

[0081] In step 25, the encoding processing unit 105 encodes the quantized residual coefficient block, encodes related syntax elements (e.g., prediction mode, encoding mode, etc.), and obtains a bitstream of the block to be encoded.

[0082] Referring to FIG. 4, FIG. 4 shows a schematic block diagram of an example of a decoder 200 for implementing the method of an embodiment of the present invention. The decoder 200 is used to receive, for example, image data encoded by an encoder 100 (i.e., an encoded bitstream, for example, an encoded bitstream of an image block and related syntax elements), and to obtain a decoded image block.

[0083] As shown in FIG. 4, the decoder 200 includes a bitstream analysis unit 201, an inverse quantization unit 202, a residual inverse transformation unit 203, a prediction processing unit 204, a reconstruction unit 205, and a filter unit 206. In some examples, the decoder 200 may perform a decoding process that is substantially the reverse of the encoding process described for the encoder 100 in FIG. 2.

[0084] The bitstream analysis unit 201 is used to decode the encoded bitstream to obtain quantized residual coefficients (or quantized residual values) and / or decoding parameters (decoded encoded parameters, for example, the decoding parameters may include any one or more of inter-frame prediction parameters, intra-frame prediction parameters, filter parameters, and / or other syntax elements executed on the encoding side). The bitstream analysis unit 201 is further used to transfer the above decoding parameters to the prediction processing unit 204 so that the prediction processing unit 204 can execute a prediction process based on the decoding parameters.

[0085] The function of the inverse quantization unit 202 may be the same as that of the inverse quantization unit 106 of the encoder 100, and is used to inverse quantize the quantized residual coefficients or quantized residual values decoded and output by the bitstream analysis unit 201.

[0086] Here, inverse quantization is the inverse process of quantization. Inverse quantization means mapping the quantized value (e.g., coefficient or residual value) to a reconstructed signal in the input signal space, and the reconstructed signal is an approximation of the input signal. To reconstruct an image block from the compressed and encoded bitstream, the inverse quantization unit 202 of the decoder 200 may inverse-quantize the quantized residual block or residual coefficient block analyzed from the bitstream, thereby reconstructing an unquantized residual block or residual coefficient block corresponding to the image block, and thereby the decoder 200 reconstructs the image block based on the reconstructed residual block or residual coefficient block to obtain a reconstructed block of the image.

[0087] As an example, taking the case where the decoder 200 analyzes the quantized residual block of the block to be decoded from the bitstream, the decoder 200 may inverse-quantize the residual block via the inverse quantization unit 202. Specifically, the inverse quantization unit 202 may multiply each residual value in the analyzed residual block by a quantization coefficient to reconstruct the residual values in the unquantized residual block corresponding to the block to be decoded, and obtain a reconstructed residual block. Here, the quantization coefficient is the quantization coefficient when the encoder quantizes the residual block of the block to be decoded when encoding the block to be decoded. In this way, the decoder 200 can realize the reconstruction of the block to be decoded based on the reconstructed residual block after inverse quantization and obtain a reconstructed block of the block to be decoded.

[0088] The function of the residual inverse transformation unit 203 may be the same as that of the residual inverse transformation unit 107 of the encoder 100, and is used to perform an inverse transformation (e.g., inverse DCT, inverse integer transformation, or a conceptually similar inverse transformation process) on the inverse-quantized residual coefficients to obtain the reconstructed residual values. The block obtained by the inverse transformation is the residual block in the pixel region of the block to be reconstructed and decoded.

[0089] The function of the reconstruction unit 205 (e.g., an adder) may be the same as that of the reconstruction unit 108 of the encoder 100.

[0090] The prediction processing unit 204 is used to receive or acquire the encoded image data (e.g., the prediction mode of the current image block) and the reconstructed image data. The prediction processing unit 204 further receives or acquires, for example, the relevant parameters of the prediction mode and / or the information regarding the selected prediction mode (i.e., the above-mentioned decoding parameters) from the bitstream analysis unit 201, and predicts the current image block based on the relevant data and the decoding parameters in the reconstructed image data to obtain the predicted block of the current image block.

[0091] The reconstruction unit 205 is used to add the reconstructed residual block to the predicted block to obtain the reconstructed block of the image to be decoded in the sample area. For example, the reconstruction unit 205 adds the residual value in the reconstructed residual block to the predicted value in the predicted block.

[0092] The filter unit 206 is used to filter the reconstructed block to obtain the filtered block, and the filtered block is the decoded image block.

[0093] Specifically, in the embodiments of the present invention, the decoder 200 is used to implement the decoding method described in the following embodiments.

[0094] In the encoder 100 and the decoder 200 of the embodiments of the present invention, the processing result of a certain process may be further processed before being output to the next process. For example, after processes such as interpolation filtering, motion vector derivation, or filtering, further operations such as Clip or shift are performed on the processing result of the corresponding process.

[0095] In one example, the decoding process performed by the decoder 200 may include the following steps.

[0096] In step 31, the bit stream analysis unit 201 analyzes the prediction mode and the residual coding mode.

[0097] In step 32, the bit stream analysis unit 201 analyzes quantization related values (for example, near value or QP value, etc.) based on the prediction mode and the residual coding mode.

[0098] In step 33, the inverse quantization unit 202 inverse quantizes the residual coefficients based on the prediction mode and the quantization related values.

[0099] In step 34, the prediction processing unit 204 obtains the predicted value of each pixel of the current image block based on the prediction mode.

[0100] In step 35, the residual inverse transformation unit 203 inverse transforms the residual coefficients to reconstruct the residual value of each pixel of the current image block.

[0101] In step 36, the reconstruction unit 205 obtains the reconstructed value based on the predicted value and the residual value of each pixel of the current image block.

[0102] FIGS. 1 to 4 are merely examples provided by the embodiments of the present invention. In some examples, the encoder 100, the decoder 200, and the encoding and decoding system 1 may include more or fewer components or units.

[0103] Hereinafter, with reference to the drawings, the following image encoding and decoding method provided by the embodiments of the present invention will be described.

[0104] FIG. 5 is a schematic flowchart of the image encoding method provided by the present invention. Optionally, the image encoding method may be applied to the encoding and decoding system 1 shown in FIG. 1, and the image encoding method may be executed by the encoding side 10. Specifically, the encoding method may be executed by the encoder 100 included in the encoding side 10. As shown in FIG. 5, the image encoding method provided by the embodiment of the present invention includes the following steps.

[0105] In S101, the encoding side obtains the maximum code length of the block to be encoded.

[0106] Here, the block to be encoded may include an encoding unit. Usually, when the encoding side encodes an image frame in the video to be encoded, it can encode in units of encoding units.

[0107] Also, the maximum code length is used to indicate the maximum code length that can be encoded for the block to be encoded. In order to prevent overflow of the bitstream, the maximum code length may be determined based on the size of the storage space of the bitstream buffer on the encoding side in the encoding and decoding system 1 shown in FIG. 1. Therefore, the maximum code length is the maximum bitstream length that is currently allowed to be cached for the storage space for the encoding side to cache the encoded bitstream.

[0108] Optionally, the maximum code length is the sum of the length of the bitstream that can be stored in the remaining space of the storage space and the length of the bitstream that the storage space outputs per unit time.

[0109] Optionally, the maximum code length is smaller than the sum of the length of the bitstream that can be stored in the remaining space of the storage space and the length of the bitstream that the storage space outputs per unit time.

[0110] Exemplarily, the maximum code length may be obtained by subtracting the header information of the block to be encoded from the sum of the length of the bitstream that can be stored in the remaining space of the memory space and the length of the bitstream output by the memory space per unit time. The header information is data having a specific length at the beginning of the bitstream of each block to be encoded, and this data may be used to indicate the image parameters of the current block to be encoded. Exemplarily, the image parameters include one or more of the image complexity of the current block to be encoded, the prediction mode, the target bits per pixel, and the texture complexity.

[0111] Note that when the bitstream buffer receives the bitstream of the current block encoded by the encoder, it simultaneously outputs a certain amount of bitstream to the channel. That is, when the bitstream buffer receives the bitstream of the current block, a certain amount of bitstream flows out. That is, the bitstream input to the bitstream buffer may occupy the memory space occupied by the outflowing bitstream before it flows out. Therefore, the maximum code length may be the sum of the code length flowing out of the bitstream buffer per unit time and the code length that can be stored in the current remaining space of the bitstream buffer.

[0112] Also, the code length flowing out of the bitstream buffer per unit time may be determined based on the channel bandwidth through which the encoding side transmits the encoded bitstream.

[0113] In S102, the encoding side pre-encodes the block to be encoded based on the first mode to obtain the first code length of the block to be encoded.

[0114] The first mode may be a preset prediction mode on the encoding side.

[0115] As an option, the preset prediction modes on the encoding side may include a point-by-point prediction mode, an intra-frame prediction mode, a block copy mode, an original value mode, and the like.

[0116] Here, the intra-frame prediction mode is a prediction mode that uses the reconstructed values of pixels in adjacent blocks around the block to be predicted as the predicted values. The block copy prediction mode is a prediction mode that uses the reconstructed values of pixels in surrounding encoded (decoded) blocks (not necessarily adjacent) as the predicted values. At the time of reconstruction, the original value mode is a reconstructed value mode that directly decodes a fixed bit width, that is, a prediction mode without reference. The point-by-point prediction mode is a prediction mode that uses the reconstructed values of adjacent pixels around the pixel to be predicted as the predicted value of the pixel to be predicted.

[0117] Here, the point-by-point prediction mode may include one or a combination of multiple prediction methods such as vertical prediction, horizontal prediction, vertical average value prediction, and horizontal average value prediction.

[0118] For vertical prediction, the predicted value (PointPredData) of the pixel to be predicted is obtained by using the reconstructed value of the pixel above the pixel to be predicted (which may be the adjacent upper side or the non-adjacent but nearby upper side). For horizontal prediction, the predicted value of the pixel to be predicted is obtained by using the reconstructed value of the pixel on the left side of the pixel to be predicted (which may be the adjacent left side or the non-adjacent but nearby left side). For vertical average value prediction, the predicted value of the pixel to be predicted is obtained by using the reconstructed values of the pixels above and below the pixel to be predicted. For horizontal average value prediction, the predicted value of the pixel to be predicted is obtained by using the reconstructed values of the pixels on both the left and right sides of the pixel to be predicted.

[0119] Specifically, the encoding side may predict the block to be encoded using each of the above preset different prediction modes, and determine the first mode based on the encoding performance after predicting the block to be encoded in different prediction modes. The encoding performance may be a compression ratio, an encoding efficiency, an encoding loss rate, or the like.

[0120] Exemplarily, the encoding side predicts a block to be encoded using each of different prediction modes, and after obtaining a predicted block by predicting based on different prediction modes, the encoding side may execute steps 22 to 25 described above to obtain a bitstream of the block to be encoded in different prediction modes. Exemplarily, the encoding side may determine the time for obtaining the bitstream of the block to be encoded in different prediction modes, and determine the prediction mode with the shortest time as the first mode. That is, the encoding side may determine the prediction mode with the highest encoding efficiency as the first mode.

[0121] Therefore, the above first code length may be the bitstream length of the block to be encoded based on the first mode, which is obtained by the encoding side predicting the block to be encoded in the first mode and then executing steps 22 to 25 described above after obtaining the predicted block.

[0122] In S103, when the first code length is greater than or equal to the maximum code length, the encoding side encodes the block to be encoded in the fallback mode, and the code length obtained by encoding the block to be encoded based on the fallback mode is smaller than the maximum code length.

[0123] Here, encoding the block to be encoded in the fallback mode, that is, encoding the block to be encoded in a fixed code length encoding method based on a predetermined target code length, and the code length obtained by encoding the block to be encoded is the target code length. The target code length is smaller than the maximum code length.

[0124] Note that the size of the storage space of the bitstream buffer is fixed. When the bitstream data input to the bitstream buffer is too large and the data that needs to be temporarily stored in the bitstream buffer exceeds the size of the storage space, "overflow" of the bitstream occurs. In this way, the data of the exceeded part is lost. Therefore, the image information in the image frame of the video to be encoded is lost, and the image frame cannot be completely decoded based on the bitstream data output by the bitstream buffer.

[0125] When the first code length is greater than or equal to the maximum code length, that is, when the bitstream of the block to be encoded obtained based on the first mode is transmitted through the bitstream buffer, "overflow" of the bitstream occurs in the bitstream buffer. To prevent "overflow" of the bitstream, the encoding side may encode the block to be encoded in the fallback mode so that the code length obtained by encoding the block to be encoded is smaller than the maximum code length.

[0126] In some embodiments, when the current block to be encoded is encoded in the fallback mode, since the target code length of the block to be encoded is fixed, the complexity information in the header information of the block to be encoded may not be updated. The complexity information may be used to calculate the quantization parameter of the block to be encoded.

[0127] In some embodiments, the encoding side may select the MidPoint Prediction Fallback mode (MPPF Mode) to predict the block to be encoded. After obtaining the predicted block, the block to be encoded may be encoded based on the fixed code length encoding method to obtain the bitstream of the block to be encoded.

[0128] Here, based on the midpoint prediction fallback mode, the encoding side may first divide the block to be encoded into a plurality of 2×2 pixel sub-image blocks. For each 2×2 sub-image block, it is necessary to calculate one median value. The calculation method of the median middle is as shown in the following formula (1). middle = 1 << (bitDepth - 1) Formula (1)

[0129] Here, bitDepth is the image bit width of the color component channel of the current sub-block to be encoded, and 1 << represents a left shift of 1 bit.

[0130] Furthermore, the encoding side may determine the average value mean of the sub-block to be encoded.

[0131] In one case, when the current sub-block cannot obtain the surrounding reconstructed image, the mean value is the above middle value.

[0132] In another case, the mean value is the average value of the reconstructed values of the 2×1 pixel sub-blocks in the previous row of the sub-block. When the reconstructed values of the 2×1 pixel sub-blocks in the previous row cannot be obtained, the average value of the reconstructed values of the 2×2 sub-blocks reconstructed one before the sub-block is taken.

[0133] Furthermore, the encoding side may determine a bias value for restricting the value of the mean of the sub-block. The encoding side may use the bias value and the middle value to clamp the mean to limit the mean value within the range of the values consisting of the bias value and the middle value. Therefore, the mean value adjusted by the bias value and the middle value is determined as the predicted value of the current 2×2 pixel sub-block to be encoded.

[0134] The bias value may be determined based on the following formula (2). bias = 1 << (mppStepSize[k] - 1) Formula (2)

[0135] Here, mppStepSize is used to indicate the quantization step in the quantization process of one component of the block to be encoded, [k] is used to indicate the k-th component of the block to be encoded, and 1<< represents a left shift by 1 bit.

[0136] Also, based on the MPPF Mode, as shown in FIG. 6, the residual value of the 2×2 pixel sub-block may be calculated based on the order of the pixel A in the upper left corner, pixel B in the upper right corner, pixel C in the lower left corner, and pixel D in the lower right corner.

[0137] Optionally, in the process of reconstructing the sub-block to be encoded, the reconstructed value of each pixel may be sequentially determined according to the order of pixel A in the upper left corner, pixel B in the upper right corner, pixel C in the lower left corner, and pixel D in the lower right corner.

[0138] In some embodiments, the block to be encoded may include one or more components.

[0139] Optionally, the block to be encoded may include only the luminance component. Or, the block to be encoded may include three components, for example, luminance Y, chrominance Cb, chrominance Cr (or three components of red R, green G, and blue B, or three components of luminance Y, chrominance U, and chrominance V). Or, in addition to the above three components, the block to be encoded may further include an α component, that is, the block to be encoded may include a total of four components. Here, the α component is a pixel transparency component. When the value of the α component is 0, the pixel corresponding to the block to be encoded is transparent, and the image bit width of the α component is different from that of the other three components.

[0140] In some embodiments, the encoding side may determine the maximum code length that can be encoded for each component, and based on the determined maximum code length, determine whether to use the fallback mode for each component.

[0141] As an option, the block to be encoded may include three components: a first chrominance component, a second chrominance component, and a luminance component. Exemplarily, the first chrominance component, the second chrominance component, and the luminance component may be three components of luminance Y, chrominance Cb, and chrominance Cr, respectively, or three components of luminance Y, chrominance U, and chrominance V.

[0142] Therefore, the encoding side may respectively determine whether it is necessary to encode each of the three components of the first chrominance component, the second chrominance component, and the luminance component in the fallback mode.

[0143] In some embodiments, the encoding side may determine the first chrominance maximum code length, the second chrominance maximum code length, and the luminance maximum code length according to a preset ratio based on the maximum code length.

[0144] The first chrominance maximum code length is the maximum code length that can be encoded for the first chrominance component of the block to be encoded. The second chrominance maximum code length is the maximum code length that can be encoded for the second chrominance component of the block to be encoded. The luminance maximum code length is the maximum code length that can be encoded for the luminance component of the block to be encoded. Note that the sum of the first chrominance maximum code length, the second chrominance maximum code length, and the luminance maximum code length is not more than the maximum code length that can be encoded for the block to be encoded.

[0145] Here, the sum of the first chrominance maximum code length, the second chrominance maximum code length, and the luminance maximum code length may be the maximum code length of the block to be encoded or the code length flowing out of the memory space per unit time.

[0146] Note that since the code length flowing out of the bitstream buffer per unit time is smaller than the maximum code length, the sum of the first chrominance maximum code length, the second chrominance maximum code length, and the luminance maximum code length may be the code length flowing out of the memory space per unit time. Also, by determining the first chrominance maximum code length, the second chrominance maximum code length, and the luminance maximum code length based on the code length flowing out of the memory space per unit time, the code length of the block to be encoded becomes smaller, the compression ratio becomes larger, and the memory occupancy of the bitstream buffer becomes smaller.

[0147] As an option, the preset ratio of the first chrominance maximum code length: the second chrominance maximum code length: the luminance maximum code length may be 1:1:1, 2:1:1, or 4:1:1.

[0148] In some embodiments, when the block to be encoded includes a plurality of components and the plurality of components of the block to be encoded share a fallback mode (sharing means that the fallback mode is used for any of the plurality of components or not used for any of them), the encoding side may only determine whether to use the fallback mode for one of the components. When the fallback mode is used for one of the components, the fallback mode is also used for the other components. Or, when some of the plurality of components of the block to be encoded share a fallback mode, the encoding side may only determine whether to use the fallback mode for one of the some components and whether to use the fallback mode for the other components excluding the some components. Or, when all of the plurality of components of the block to be encoded do not share a fallback mode, the encoding side may determine whether to use the fallback mode for each component respectively.

[0149] Optionally, the encoding side may determine whether to encode the first chrominance component, the second chrominance component, and the luminance component in the fallback mode based on one or more of the first maximum chrominance code length, the second maximum chrominance code length, and the maximum luminance code length. Some possible implementation manners are shown below.

[0150] Implementation Manner 1: The encoding side determines whether to encode the first chrominance component, the second chrominance component, and the luminance component in the fallback mode based on any one of the first maximum chrominance code length, the second maximum chrominance code length, and the maximum luminance code length.

[0151] Optionally, the encoding side first determines whether to encode any one of the first chrominance component, the second chrominance component, and the luminance component in the fallback mode, and if it is determined to use the fallback mode for this component, it may also determine to encode the other two components in the fallback mode.

[0152] Exemplarily, the encoding side first determines whether to encode the first chrominance component in the fallback mode based on the first maximum chrominance code length, and if it is determined to encode the first chrominance component in the fallback mode, it may also determine to encode the second chrominance component and the luminance component in the fallback mode.

[0153] Similarly, the encoding side may first determine whether to encode the second chrominance component in the fallback mode based on the second maximum chrominance code length, or may determine whether to encode the luminance component in the fallback mode based on the maximum luminance code length.

[0154] As an option, the process by which the encoding side determines whether to encode the first chrominance component in the fallback mode based on the first maximum chrominance code length may specifically include that the encoding side pre-encodes the first chrominance component based on the first mode to obtain the first chrominance code length of the first chrominance component. Next, if the first chrominance code length is greater than or equal to the first maximum chrominance code length, it is determined to encode the first chrominance component in the fallback mode.

[0155] Similarly, the process by which the encoding side determines whether to encode the second chrominance component in the fallback mode based on the second maximum chrominance code length may specifically include that the encoding side pre-encodes the second chrominance component based on the first mode to obtain the second chrominance code length of the second chrominance component. Next, if the second chrominance code length is greater than or equal to the second maximum chrominance code length, it is determined to encode the second chrominance component in the fallback mode.

[0156] Similarly, the process by which the encoding side determines whether to encode the luminance component in the fallback mode based on the maximum luminance code length may specifically include that the encoding side pre-encodes the luminance component based on the first mode to obtain the luminance code length of the luminance component. Next, if the luminance code length is greater than or equal to the maximum luminance code length, it is determined to encode the luminance component in the fallback mode.

[0157] Here, the process of performing pre-encoding on the first chrominance component, the second chrominance component, or the luminance component based on the first mode may refer to the related description of the pre-encoding based on the first mode in step S102, and the description is omitted here.

[0158] Implementation method 2: The encoding side determines whether to encode the first chrominance component and the second chrominance component in the fallback mode based on the first maximum chrominance code length or the second maximum chrominance code length, and determines whether to encode the luminance component in the fallback mode based on the maximum luminance code length.

[0159] As an option, for the first chrominance component and the second chrominance component, on the encoding side, first, it may be determined whether to encode the first chrominance component in the fallback mode. When it is determined to use the fallback mode for the first chrominance component, it may also be determined to encode the second chrominance component in the fallback mode. Or, on the encoding side, first, it may be determined whether to encode the second chrominance component in the fallback mode. When it is determined to use the fallback mode for the second chrominance component, it may also be determined to encode the first chrominance component in the fallback mode.

[0160] Here, for the process in which the encoding side determines whether to encode the first chrominance component in the fallback mode based on the first chrominance maximum code length, and for the process in which the encoding side determines whether to encode the second chrominance component in the fallback mode based on the second chrominance maximum code length, the relevant descriptions in the above implementation manner 1 may be specifically referred to.

[0161] In one possible implementation manner, for the luminance component, the encoding side may determine whether to encode the luminance component in the fallback mode based on the luminance maximum code length. For the process in which the encoding side determines whether to encode the luminance component in the fallback mode based on the luminance maximum code length, the relevant descriptions in the above implementation manner 1 may be specifically referred to.

[0162] In another possible implementation manner, for the luminance component, when the encoding side determines to encode the first chrominance component and the second chrominance component in the fallback mode, the encoding side may determine the code lengths for encoding the first chrominance component and the second chrominance component in the fallback mode, and based on the first mode, pre-encode the luminance component to obtain the luminance code length of the luminance component. Next, when the sum of the luminance code length and the code lengths for encoding the first chrominance component and the second chrominance component in the fallback mode is greater than or equal to the maximum code length of the block to be encoded, the encoding side may determine to encode the luminance component in the fallback mode.

[0163] Implementation Method 3: The encoding side determines whether to encode the first chrominance component in the fallback mode based on the first maximum chrominance code length, determines whether to encode the second chrominance component in the fallback mode based on the second maximum chrominance code length, and determines whether to encode the luminance component in the fallback mode based on the maximum luminance code length.

[0164] Here, for the process in which the encoding side determines whether to encode the first chrominance component in the fallback mode based on the first maximum chrominance code length, the process in which the encoding side determines whether to encode the second chrominance component in the fallback mode based on the second maximum chrominance code length, and the process in which the encoding side determines whether to encode the luminance component in the fallback mode based on the maximum luminance code length, the relevant descriptions in the above Implementation Method 1 may be specifically referred to.

[0165] In some embodiments, when the block to be encoded includes three components, namely the first chrominance component, the second chrominance component, and the luminance component, the encoding side may divide the target code length of the block to be encoded into three sub-code lengths according to the preset ratio, and one component corresponds to one sub-code length. Next, based on the sub-code length of each component, the components corresponding to each sub-code length are encoded in the fallback mode respectively.

[0166] Optionally, for the block to be encoded in the YCoCg image, the Y component: Co component: Cg component may be 2:1:1. For the block to be encoded in the YUV444 image, the Y component: U component: V component may be 2:1:1. For the block to be encoded in the YUV422 image, the Y component: U component: V component may be 2:1:1. For the block to be encoded in the YUV420 image, the Y component: U component: V component may be 4:1:1. For the block to be encoded in the YUV400 image, the block to be encoded includes only the luminance component Y.

[0167] As an option, for the block to be encoded in the RGB image, based on the fact that the R component: G component: B component is 1:1:1, the encoding side may divide the target code length of the block to be encoded into three sub-code lengths, and one component corresponds to one sub-code length.

[0168] In addition, in the encoding process, for an image block in RGB format, since the human eye is sensitive to image luminance, in order to improve the image effect seen by the human eye, usually, the image block in RGB format is converted into an image block in YCoCg format and then encoded. However, if the encoding side determines that it is necessary to use the fallback mode in the encoding process for any of the components of the converted image block in YCoCg format, it means that the code length required for encoding the converted image block in YCoCg format is long, and the encoding side may encode the image block in RGB format. In some embodiments, for the component for which the fallback mode is selected, since the sub-code length of the component is determined, it is not necessary to update the quantization parameter in the header information of the current component, and it is not necessary to update the information for instructing the bitstream of the component such as the complexity of the current component.

[0169] In some embodiments, when the first code length is smaller than the maximum code length, the encoding side may encode the original pixel value of the block to be encoded in the original value mode. As an option, the encoding side may use the original value mode to encode the quantized and transformed value of the original pixel value of the block to be encoded. In this way, the encoding side can improve the effect of reconstructing the image of the block to be encoded by encoding the block to be encoded in the original value mode.

[0170] In some embodiments, when the first code length is smaller than the maximum code length, the encoding side may encode the block to be encoded in the residual encoding mode.

[0171] In one example, the codeword in the original value mode and the codeword in the fallback mode are the same, where the codeword in the original value mode is a number for indicating the original value mode, and the codeword in the fallback mode is a number for indicating the fallback mode.

[0172] In another example, the codeword in the fallback mode is unique. In this case, the codeword in the original value mode is different from the codeword in the fallback mode.

[0173] In some embodiments, the above-mentioned fallback mode may be used when there is no overflow in the block to be encoded (i.e., the encoding side also uses the fallback mode when the first code length is smaller than the maximum code length). Hereinafter, this case will be described.

[0174] Optionally, when the occupied memory in the bitstream buffer is less than or equal to a first threshold of the total memory of the bitstream buffer, the encoding side may encode the block to be encoded with another code length value greater than the target code length. Exemplarily, the first threshold may be 30% of the total memory of the bitstream buffer. Therefore, the encoding side may encode the block to be encoded with a code length value greater than the target code length, for example, 1.5 times the target code length value. Optionally, when the occupied memory in the bitstream buffer is greater than or equal to a second threshold of the total memory of the bitstream buffer, the encoding side may encode the block to be encoded with another code length value smaller than the target code length. Exemplarily, the second threshold may be 85% of the total memory of the bitstream buffer. Therefore, the encoding side may encode the block to be encoded with a code length value smaller than the target code length, for example, 0.5 times the target code length value. When the occupied memory in the bitstream buffer is smaller than the second threshold and greater than the first threshold, the encoding side may encode the block to be encoded with the target code length.

[0175] Correspondingly, when there is no overflow in the bitstream buffer on the decoding side and the decoding is performed in the fallback mode, the decoding side may determine the code length of the block to be decoded based on the size of the occupied memory in the bitstream buffer when encoding the block to be decoded.

[0176] As described above, both the encoding / decoding sides record the state of the bitstream buffer. After encoding one block to be decoded, the encoding side updates the state of the bitstream buffer, and after analyzing this block to be decoded, the decoding side similarly updates the state of the bitstream buffer. The update of the bitstream buffer state is related to the target compression ratio and the size of the bitstream of the encoded / decoded block, and the target compression ratio is written into the image header bitstream. Therefore, if the update methods are the same when the encoding side and the decoding side update the state of the bitstream buffer, it can be guaranteed that the states of the bitstream buffers for encoding / decoding match.

[0177] Therefore, when both the encoding side and the decoding side include a bitstream buffer, the decoding / encoding side may know the memory occupancy status of the bitstream buffer during encoding / decoding by recording the states of each other's bitstream buffers. When only the encoding side or only the decoding side includes a bitstream buffer, the encoding side may know the memory occupancy status of the bitstream buffer during decoding by recording the state of the decoding side's bitstream buffer, and the decoding side may also know the memory occupancy status of the bitstream buffer during encoding by recording the state of the encoding side's bitstream buffer.

[0178] The image encoding method provided by the embodiments of the present invention has at least the following beneficial effects. First, the block to be encoded is pre-encoded, the bitstream length obtained by the pre-encoding is compared with the maximum code length allowed to be occupied by the block to be encoded, and based on the comparison result, it can be determined whether an overflow of the bitstream occurs. Next, if it is determined that an overflow of the bitstream occurs, the block to be encoded is encoded in the fallback mode to prevent the loss of the image information of the block to be encoded.

[0179] In some embodiments, as shown in FIG. 7, the process in which the encoding side encodes the block to be encoded based on the fixed code length encoding method based on the fallback mode may be specifically implemented as the following steps.

[0180] In S201, the encoding side divides the block to be encoded into one or more sub-blocks to be encoded.

[0181] Optionally, in the process of encoding the block to be encoded based on the fallback mode, usually, each sub-block to be encoded of the block to be encoded is determined and encoded by the fixed code length encoding method.

[0182] Optionally, the fixed code length and the number of sub-blocks to be encoded in the block to be encoded satisfy the following formula (3). subblock num ×fixed len ≦fixed bit Formula (3)

[0183] Here, subblock num is the number of sub-blocks to be encoded in the block to be encoded. fixed len is the fixed code length. fixed bit is the target code length of the block to be encoded.

[0184] Optionally, the target code length of the block to be encoded may be determined based on the target bits per pixel (BPP) of the block to be encoded.

[0185] Here, the target bits per pixel is used to indicate the code length required when encoding each pixel of the block to be encoded at the target compression ratio. The target bits per pixel may be determined based on the target compression ratio of the block to be encoded.

[0186] Optionally, the encoding side may obtain the target bits per pixel of the block to be encoded, and based on the target bits per pixel and the number of pixels of the block to be encoded, determine the product of the target bits per pixel and the number of pixels of the block to be encoded as the target code length of the block to be encoded.

[0187] Optionally, the target code length may be smaller than the product of the target bits per pixel and the number of pixels of the block to be encoded.

[0188] Exemplarily, the target code length may be the product of the target bits per pixel and the number of pixels of the block to be encoded minus the encoding cost required to encode the fallback mode. Here, the analysis result of the encoding cost is used to indicate that the encoding mode is the fallback mode. For example, the analysis result of the encoding cost may be an identifier of the fallback mode (a codeword of the fallback mode).

[0189] In some embodiments, when the block to be encoded includes a plurality of components, the encoding side may first divide the block to be encoded into a plurality of components, and then divide each component into one or more sub-blocks to be encoded.

[0190] Exemplarily, based on the related description in step S103 above, when the block to be encoded may include three components: a first chrominance component, a second chrominance component, and a luminance component, the encoding side may divide the target code length of the block to be encoded into three sub-code lengths according to a preset ratio, and one component corresponds to one sub-code length. Next, the encoding side may divide the component into one or more sub-encoding blocks based on the sub-code length corresponding to each component.

[0191] Taking the preset ratio of the Y component: Co component: Cg component being 2:1:1 as an example, the sub-code length of the Co component may be determined based on the following formula (4). Sub-code length of Co component = [(target code length / 2) - 3 × encoding cost in fallback mode] / 2 Formula (4)

[0192] The ratio of the sub-code length of the Cg component to the target code length is the same as that of the Co component, and the sub-code length of the Cg component may also be determined based on formula (4).

[0193] Furthermore, the sub-code length of the Y component may be the result of subtracting the sub-code lengths of the Co component and the Cg component from the target code length.

[0194] In some other embodiments, when the block to be encoded includes multiple components, the encoding side may first divide the block to be encoded into one or more sub-encoding blocks, and then for each sub-encoding block, divide the sub-encoding block into sub-blocks of each component according to the ratio between the components.

[0195] Hereinafter, taking the example of first dividing the block to be encoded into one or more sub-encoding blocks, the division process of the block to be encoded will be described.

[0196] In one possible implementation, the encoding side divides the block to be encoded into one or more sub-blocks to be encoded with a preset size.

[0197] Here, the preset size of the sub-block to be encoded may be 2×1 pixels, 2×2 pixels, or the like.

[0198] Specifically, the encoding side determines the number of sub-blocks to be encoded in the block to be encoded based on the size of the sub-block to be encoded, and may divide the block to be encoded into one or more sub-blocks to be encoded based on the number of sub-blocks to be encoded.

[0199] Exemplarily, the block to be encoded includes two rows of pixels A1 to A16, B1 to B16 as shown in FIG. 8. When the size of one sub-block to be encoded is 2×2 pixels, the encoding side may determine that the number of sub-blocks to be encoded is 8. For example, the sub-blocks to be encoded may be an image block composed of A1, A2, B1, B2, an image block composed of A3, A4, B3, B4, or the like.

[0200] Optionally, after the encoding side determines the number of sub-blocks to be encoded, a first fixed code length for encoding the sub-blocks to be encoded may be determined based on the above formula (3).

[0201] In another possible implementation, the encoding side divides the block to be encoded into one or more sub-blocks to be encoded with a preset first fixed code length.

[0202] Optionally, the first fixed code length may be a preset code length value for encoding the sub-blocks to be encoded.

[0203] Optionally, when the sub-block to be encoded includes only one pixel, the first fixed code length may be the above target bits per pixel.

[0204] Specifically, the encoding side may determine the number of sub - blocks to be encoded for the block to be encoded based on the target code length of the block to be encoded and the first fixed code length for encoding the sub - blocks to be encoded according to the above formula (3). Next, based on the number of sub - blocks to be encoded, the block to be encoded is divided into one or more sub - blocks to be encoded.

[0205] Optionally, according to the above formula (3), the number of sub - blocks to be encoded may be an integer value obtained by dividing the target code length of the block to be encoded by the first fixed code length and rounding down.

[0206] Exemplarily, the size of the block to be encoded is 16×2 pixels, the first fixed code length is 4, and the target code length of the block to be encoded is 23. The number of sub - blocks to be encoded is the integer value 5 obtained by rounding down 23 / 4. When the number of sub - blocks is odd, 16×2 is divided into 5 sub - blocks to be encoded, and the sizes of the sub - blocks to be encoded are 3×2 pixels, 3×2 pixels, 3×2 pixels, 3×2 pixels, and 4×2 pixels respectively.

[0207] When the size of the block to be encoded is 16×2 pixels, the first fixed code length is 4, and the target code length of the block to be encoded is 25, according to the above formula (3), the number of sub - blocks to be encoded is the integer value 6 obtained by rounding down 25 / 4. When the number of sub - blocks is even, first, one 16×2 image block may be divided into two 16×1 pixel image blocks, and then each 16×1 pixel image block may be divided into 5×1 pixel image blocks, 5×1 pixel image blocks, and 6×1 pixel image blocks.

[0208] Note that since the predicted value of the sub-block to be encoded may be determined based on the reconstructed pixels of the previous line, when the sub-block to be encoded contains pixels of two lines, the encoding side may control the number of divisions of the sub-block to be encoded to be even in order to ensure that each sub-block to be encoded is related to the pixels of the first line of the block to be encoded. In this way, since the predicted value of the pixels of the first line may be determined based on the reconstructed pixels of the previous line, the predicted value of the pixels of the second line of the sub-block to be encoded may also be determined based on the reconstructed pixels of the first line. Note that when the sub-block to be encoded contains pixels of Z lines, the encoding side may control the number of divisions of the sub-block to be encoded to be an integer multiple of Z in order to ensure that each encoded sub-block is related to the pixels of the first line of the encoded block.

[0209] As an option, the first fixed code length is a non-negative integer. Taking the case where the first fixed code length is the target bits per pixel as an example, when the block to be encoded is a 16×2 pixel image block, the encoding side divides it into sub-blocks to be encoded of two 16×1 pixel image blocks. In one example, when the target bits per pixel is 3, the first fixed code length of both of the two 16×1 pixel image blocks is 3. In another example, when the target bits per pixel is 2.5, the encoding side may determine that the first fixed code length of the first sub-block to be encoded is 3 and the first fixed code length of the second sub-block is 2.

[0210] In S202, the encoding side encodes a plurality of sub-blocks to be encoded in a fixed code length encoding method based on the first fixed code length.

[0211] Here, the bit stream length obtained by encoding each sub-block to be encoded among the plurality of sub-blocks to be encoded is the first fixed code length.

[0212] For any sub-block to be encoded, the encoding side determines the original value and the predicted value of the pixels of the sub-block to be encoded.

[0213] In some embodiments, the encoding side may determine the average value of the reconstructed values of the pixels in the previous row adjacent to the sub-block to be encoded as the predicted value of the sub-block to be encoded.

[0214] Optionally, the predicted value of the sub-block to be encoded may be determined based on the following formula (5). Predicted value = (Sum of the reconstructed values of each reference pixel + 0.5 × S) / S Formula (5)

[0215] Here, S is the number of reference pixels, and S is a positive integer.

[0216] Note that in the process of calculating the predicted value, the purpose of adding 0.5 × S to the sum of the reconstructed values of each reference pixel is to round the average value of the reconstructed values.

[0217] Exemplarily, as shown in FIG. 9(a) for the current encoding block, the size of the block to be encoded is 16×2 pixels, including the pixels A1~A16 and B1~B16 in two rows, and the reconstructed pixels in the previous row include C0~C16. When the sub-block to be encoded is an image block consisting of A1, A2, A3, B1, B2, B3, the reference pixels in the previous row corresponding to the sub-block to be encoded are C1, C2, C3. Therefore, the predicted value of the sub-block to be encoded is (Reconstructed value of C1 + Reconstructed value of C2 + Reconstructed value of C3 + 1.5) / 3.

[0218] As an option, when the reconstructed value of the pixels in the row before the sub-block to be encoded cannot be obtained, the encoding side may fill the pixels in the row before the sub-block to be encoded with the reconstructed pixels in the previous column. Exemplarily, as shown in FIG. 9(b), the encoding side fills all the pixels in the previous row with the reconstructed value of pixel A0 in the previous column. Therefore, when the sub-block to be encoded is an image block consisting of A1, A2, A3, B1, B2, B3, the reference pixels in the previous row corresponding to the sub-block to be encoded are the reconstructed values of three A0s, and the encoding side may determine the predicted value of the sub-block to be encoded based on the reconstructed values of these three A0s.

[0219] Or, as shown in FIG. 9(c), the encoding side fills all the pixels in the previous row with the reconstructed value of pixel B0 in the previous column. Similarly, when the sub-block to be encoded is an image block consisting of A1, A2, A3, B1, B2, B3, the reference pixels in the previous row corresponding to the sub-block to be encoded are the reconstructed values of three B0s, and the encoding side may determine the predicted value of the sub-block to be encoded based on the reconstructed values of these three B0s.

[0220] Or, as shown in FIG. 9(d), the encoding side fills all the pixels in the previous row alternately with the reconstructed values of pixels A0 and B0 in the previous column.

[0221] Furthermore, the encoding side may determine the average value of the reconstructed values after filling in the previous row adjacent to the sub-block to be encoded as the predicted value of the sub-block to be encoded.

[0222] In some other embodiments, the encoding side may determine the average value of the reconstructed values of the pixels in the previous column adjacent to the sub-block to be encoded as the predicted value of the sub-block to be encoded.

[0223] Exemplarily, as shown in FIG. 9(a), the size of the block to be encoded is 16×2 pixels, including two rows of pixels A1 to A16, B1 to B16, and the reconstructed pixels in the previous column include C0, A0, and B0. When the sub-block to be encoded is an image block consisting of A1, A2, A3, B1, B2, B3, the reference pixels in the previous column corresponding to the sub-block to be encoded are A0 and B0. Therefore, the predicted value of the sub-block to be encoded is (the reconstructed value of A0 + the reconstructed value of B0 + 1) / 2.

[0224] Here, the method for determining the predicted value of the sub-block to be encoded is only an example, and the encoding side may determine the predicted value of the sub-block to be encoded in any of the prediction modes described in step S102, or other possible prediction modes. Optionally, the encoding side may determine the predicted value of the sub-block to be encoded in a preset prediction mode.

[0225] In one possible implementation, the encoding side quantizes the residual value of the sub-block to be encoded, encodes the quantized residual value, and obtains the bitstream of the sub-block to be encoded. Here, the residual value is the difference between the original value and the predicted value of the sub-block to be encoded.

[0226] Optionally, before quantizing the residual value of the sub-block to be encoded, a transformation process may be performed on the residual value.

[0227] Furthermore, in order to enable the sub-block to be encoded to be used as a reference for the predicted values of other sub-blocks to be encoded, the sub-block to be encoded may be reconstructed. When reconstructing the sub-block to be encoded, the encoding side may inverse-quantize the residual value to obtain the inverse-quantized residual value, and reconstruct the sub-block to be encoded based on the inverse-quantized residual value and the predicted value.

[0228] In another possible implementation, the encoding side may quantize the high-order bit value of the image bit width of the sub-block to be encoded, and encode the quantized high-order bit value of the image bit width. Here, the high-order bit value of the image bit width is the value of the first N bits at the beginning of the binary number corresponding to the image bit width, and N is a positive integer. Exemplarily, N may be a possible number such as 4 or 5. Optionally, the high-order bit value may be a value for indicating the color of the sub-block to be encoded.

[0229] Furthermore, in order to enable the sub-block to be encoded to be used as a reference for the predicted value of another sub-block to be encoded, the sub-block to be encoded may be reconstructed. When reconstructing the sub-block to be encoded, the encoding side analyzes the bit stream of the image block corresponding to the sub-block to be encoded to obtain the high-order bit value of the image bit width corresponding to the sub-block to be encoded, inverse quantizes the high-order bit value of the image bit width to obtain the inverse quantized high-order bit value of the image bit width, and combines it with the lower-order bit value of the predicted value to reconstruct the sub-block to be encoded.

[0230] The lower-order bit value of the predicted value is used to indicate the value of the last M bits of the binary number corresponding to the predicted value, and M is a positive integer. Exemplarily, M may be a possible number such as 4 or 5. Optionally, the lower-order bit value may be a value for indicating the texture in the predicted value. Here, the sum of N and M is the image bit width of the sub-block to be decoded, and N is the first fixed code length.

[0231] As an example of the lower-order bit value of the predicted value, when the image bit width (the code length of the original value) is 8 and the first fixed code length N of the sub-block to be encoded is 4, both M and N are 4. When the predicted value of the sub-block to be encoded is 127 and the binary code is 01111111, the lower-order bit value of its last M bits (the last 4 bits) is 1111, that is, 15, that is, the decimal number corresponding to the last 4 bits of the predicted value is 15.

[0232] In another possible implementation, the encoding side may quantize the lower bit value of the image bit width of the sub-block to be encoded and encode the quantized lower bit value of the image bit width. Here, the lower bit value of the image bit width is the value of the last N bits of the binary number corresponding to the image bit width, and N is a positive integer. Optionally, the lower bit value may be a value for indicating the texture of the sub-block to be encoded.

[0233] Furthermore, in order to enable the sub-block to be encoded to be used as a reference for the predicted value of another sub-block to be encoded, the sub-block to be encoded may be reconstructed. When reconstructing the sub-block to be encoded, the encoding side analyzes the bit stream of the image block corresponding to the sub-block to be encoded to obtain the lower bit value of the image bit width corresponding to the sub-block to be encoded, inverse quantizes the lower bit value of the image bit width to obtain the inverse quantized lower bit value of the image bit width, and combines it with the value obtained by padding zeros to the upper bit value of the predicted value to reconstruct the sub-block to be encoded.

[0234] The upper bit value of the predicted value is used to indicate the value of the first M bits of the binary number corresponding to the predicted value, and M is a positive integer. Optionally, the upper bit value may be a value for indicating the color in the predicted value. Here, the sum of N and M is the image bit width of the sub-block to be encoded, and N is the first fixed code length.

[0235] As an example of the upper bit value of the predicted value, when the image bit width (the code length of the original value) is 8 and the first fixed code length N of the sub-block to be encoded is 4, both M and N are 4. When the predicted value of the sub-block to be encoded is 127 and the corresponding binary code is 01111111, the upper bit value of the first M bits (the first 4 bits) is 0111, and the binary code with zeros filled in the subsequent 8 - M bits is 01110000. That is, the decimal number corresponding to the upper bit value with zeros filled in is 112.

[0236] Optionally, for each sub-block to be encoded, when the coefficient value (such as the residual value, the original value, etc.) of the sub-block to be encoded is at the boundary of the representable range of the first fixed code length, in the transmission process of the bit stream of the sub-block, a suffix for indicating the coefficient value of the sub-block to be encoded may be added. For example, when the first fixed code length of the sub-block to be encoded is 3, since the coefficient value range representable by 3 - bit binary data is [0, 7], when the current coefficient value is in the range [1, 8], the encoding side may add a suffix 0 to the bit stream of the sub-block to be encoded. Here, the suffix may be determined according to a preset suffix setting rule. Exemplarily, the preset suffix setting rule may include possible rules such as suffix 0 indicating a negative number, suffix 1 indicating an integer, and suffix 0 indicating that the representable coefficient value range increases by 1 to the right (for example, when the above range [0, 7] increases by 1 to the right, it becomes [1, 8]). Based on this embodiment, first, the block to be encoded may be divided into a plurality of sub-blocks to be encoded, and based on the fixed code length encoding method, each sub-block to be encoded may be encoded with the first fixed code length, whereby the bit stream after the block to be encoded is encoded can be made smaller than the maximum code length that can be cached by the bit stream buffer, preventing the overflow of the bit stream.

[0237] In some embodiments, the present invention further provides an image decoding method, which corresponds to the encoding method shown in FIG. 5. As shown in FIG. 10, FIG. 10 is a schematic flowchart of the image decoding method provided by the present invention. The image decoding method may be executed by a decoder 200 or may be executed by a decoding side (e.g., the decoding side 20 shown in FIG. 1) that supports the functions of the decoder 200. Here, the case where the decoder 200 executes the decoding method is taken as an example for explanation. The image decoding method includes the following steps.

[0238] In S301, the decoding side determines whether a fallback mode is used for an image block corresponding to a block to be decoded based on the bitstream of the block to be decoded.

[0239] Here, when the image block is encoded based on the fallback mode, the code length obtained is less than or equal to the maximum code length of the block to be decoded. For the related description of the maximum code length, reference may be made to the related description in step S101 above, and the description is omitted here.

[0240] Optionally, based on the bitstream of the block to be decoded, the decoding side may analyze the complexity information of the block to be decoded from the header information of the block to be decoded and determine the quantization step of the block to be decoded. Or, the decoding side may analyze the quantization step of the block to be decoded from the header information of the block to be decoded. Or, the decoding side may determine the quantization step of the block to be decoded based on the target bits per pixel.

[0241] In some embodiments, the block to be decoded may include one or more components.

[0242] As an option, the block to be decoded may contain only the luminance component. Or, the block to be decoded may contain three components, for example, luminance Y, chrominance Cb, chrominance Cr (or three components of red R, green G, and blue B, or three components of luminance Y, chrominance U, and chrominance V). Or, in addition to the above three components, the block to be decoded may further contain an α component, that is, the block to be decoded may contain a total of four components. Here, the α component is a pixel transparency component. When the value of the α component is 0, the pixel corresponding to the block to be decoded is transparent, and the image bit width of the α component is different from that of the other three components.

[0243] In some embodiments, when the block to be decoded contains multiple components and the multiple components of the block to be decoded share a fallback mode (sharing a fallback mode means that the fallback mode is used for any of the multiple components or not used for any of them), the decoding side may only determine whether the fallback mode is used for one of the components. If the fallback mode is used for one of the components, the fallback mode is also used for the other components. Or, when some of the multiple components of the block to be decoded share a fallback mode, the decoding side may only determine whether the fallback mode is used for one of the some components and whether the fallback mode is used for the other components excluding the some components. Or, when all of the multiple components of the block to be decoded do not share a fallback mode, the decoding side may determine whether the fallback mode is used for each component respectively.

[0244] Specifically, the bitstream of the block to be decoded may contain an identifier for indicating the sharing status of the above fallback mode. Therefore, the decoding side may determine the components sharing the fallback mode based on the identifier and determine whether the fallback mode is used for each component.

[0245] Exemplarily, when the block to be decoded includes a first chrominance component, a second chrominance component, and a luminance component, the decoding side may respectively determine whether the fallback mode is used for the first chrominance component, the second chrominance component, and the luminance component during encoding.

[0246] In one example, corresponding to implementation method 1 in step S103 above, when encoding the block to be decoded, if it is determined in the above implementation method 1 whether to use the fallback mode for each component, the decoding side may analyze the bitstream of any one of the first chrominance component, the second chrominance component, or the luminance component, and determine whether the fallback mode is used for the first chrominance component, the second chrominance component, and the luminance component during encoding.

[0247] Specifically, when the decoding side analyzes the bitstream of any one of the first chrominance component, the second chrominance component, or the luminance component and determines that the fallback mode is used for this component during encoding, the decoding side may also determine that the fallback mode is used for all of the first chrominance component, the second chrominance component, and the luminance component during encoding, and there is no need to determine whether the fallback mode is used for other components based on the bitstreams of other components.

[0248] In another example, corresponding to implementation method 2 in step S103 above, when encoding the block to be decoded, if it is determined in the above implementation method 2 whether to use the fallback mode for each component, the decoding side may analyze the bitstream of the first chrominance component or the second chrominance component to determine whether the fallback mode is used for the first chrominance component and the second chrominance component during encoding.

[0249] Specifically, when the decoding side analyzes the bitstream of any one of the first chrominance component or the second chrominance component and determines that the fallback mode is used during encoding for this component, the decoding side may determine that the fallback mode is used during encoding for both the first chrominance component and the second chrominance component. Also, the decoding side needs to analyze the bitstream of the luminance component and determine that the fallback mode is used during encoding for the luminance component.

[0250] In another example, corresponding to implementation method 3 in step S103 above, when determining whether to use the fallback mode for each component when encoding the block to be decoded according to implementation method 3 above, the decoding side analyzes the bitstreams of the first chrominance component, the second chrominance component, and the luminance component, and respectively determines whether the fallback mode is used during encoding for the first chrominance component, the second chrominance component, and the luminance component.

[0251] In some embodiments, the codeword of the original value mode and the codeword of the fallback mode are the same. For example, if the codewords of both the original value mode and the fallback mode are the first codeword, when the decoding side analyzes the bitstream of the block to be decoded and obtains the first codeword, the decoding side may determine that the fallback mode or the original value mode is used for the block to be decoded.

[0252] Furthermore, the decoding side may also determine whether the fallback mode is used for the block to be decoded based on the size of the occupied memory space in the bitstream buffer.

[0253] When the decoder determines that the occupied memory space in the bitstream buffer is larger than a first preset memory value when encoding the image block to be decoded, that is, when an overflow occurs, the decoder determines that the fallback mode is used for the image block corresponding to the block to be decoded. Otherwise, the decoder determines that the original value mode is used for the image block corresponding to the block to be decoded.

[0254] The size of the first preset memory value may be determined based on the actual memory capacity of the bitstream buffer, and is not particularly limited in this regard.

[0255] Optionally, the decoder determines that the bitstream length of the block to be encoded is the sum of the products of the pixel numbers and the image bit widths of all components, and the decoder may analyze the bitstream of the block to be decoded and reconstruct the image block of the block to be decoded based on the original value mode.

[0256] In some embodiments, before executing the following step S302, when the fallback mode is used for the image block corresponding to the block to be decoded, the decoder may determine the prediction mode used for the image block corresponding to the block to be decoded. Here, the prediction mode may be any of the prediction modes described in step S102, or other possible prediction modes.

[0257] In one implementation, when the block to be decoded includes one component, the decoder may analyze the bitstream of the block to be decoded to determine the prediction mode of the block to be decoded. Alternatively, the decoder may determine a preset prediction mode as the prediction mode of the block to be decoded, and the preset prediction mode is the same as the preset prediction mode used by the encoder in the prediction process.

[0258] In another implementation, when the block to be decoded contains multiple components, the same prediction mode may be used for the multiple components of the block to be decoded. Therefore, the decoding side may determine the prediction mode for only one component, and this prediction mode is the prediction mode for all components of the block to be decoded. Or, the same prediction mode may be used for some of the multiple components of the block to be decoded. In this case, the decoding side may determine the prediction mode for only one of those some components and the prediction mode for the other components excluding those some components. Or, the decoding side may determine the prediction mode for each component respectively.

[0259] Here, for the process in which the decoding side determines the prediction mode for one component, reference may be made to the detailed description in the above another implementation, and the description is omitted here.

[0260] In S302, when the fallback mode is used for the image block corresponding to the block to be decoded, the decoding side obtains the first fixed code length.

[0261] Here, the first fixed code length is the code length of one sub-block to be decoded, and one block to be decoded may include one or multiple sub-blocks to be decoded.

[0262] Optionally, the first fixed code length may be a preset code length value or the target bits per pixel during the encoding of the block to be decoded. The target bits per pixel is used to indicate the code length required when the encoding side encodes each pixel of the block to be encoded at the target compression ratio.

[0263] In some embodiments, the decoding side may divide the block to be decoded into multiple sub-blocks to be decoded based on the code length of the block to be decoded and the first fixed code length.

[0264] Here, the number of sub-blocks to be decoded may be determined based on the following formula (6).

[0265] The number of sub - blocks to be decoded in the above - mentioned fixed symbol length and the block to be decoded satisfies the following formula (6). The number of sub - blocks to be decoded×the first fixed symbol length = the symbol length of the block to be decoded Formula (6)

[0266] Next, the decoding side may divide the block to be decoded into a plurality of sub - blocks to be decoded based on the calculated number of sub - blocks to be decoded.

[0267] In some embodiments, the decoding side may divide the block to be decoded into a plurality of sub - blocks to be decoded based on a preset size.

[0268] Here, for the process of the decoding side dividing the block to be decoded into a plurality of sub - blocks to be decoded, reference may be made to the related description of step S201 above, and the description is omitted here.

[0269] In some embodiments, when the block to be decoded contains a plurality of components, the decoding side may first divide the block to be decoded into a plurality of components, and then divide each component into one or more sub - blocks to be decoded.

[0270] Exemplarily, the block to be decoded includes three components: a first chrominance component, a second chrominance component, and a luminance component. The decoding side may divide the symbol length of the block to be decoded into three sub - symbol lengths according to a preset ratio, and one component corresponds to one sub - symbol length. Next, based on the sub - symbol length of each component, for each component, according to the above - possible implementation manner, the component may be divided into one or more sub - blocks to be decoded.

[0271] Optionally, for the block to be decoded in the YCoCg image, the preset ratio of the Y component: Co component: Cg component may be 2:1:1. For the block to be decoded in the YUV444 image, the preset ratio of the Y component: U component: V component may be 2:1:1. For the block to be decoded in the YUV422 image, the preset ratio of the Y component: U component: V component may be 2:1:1. For the block to be decoded in the YUV420 image, the preset ratio of the Y component: U component: V component may be 4:1:1. For the block to be decoded in the YUV400 image, the block to be decoded includes only the luminance component Y.

[0272] Optionally, for the block to be decoded in the RGB image, the decoding side may divide the total code length of the block to be decoded into three sub-code lengths based on the fact that the preset ratio of the R component: G component: B component is 1:1:1, and one component corresponds to one sub-code length.

[0273] In the image encoding and decoding process, for an image block in the RGB format, since the human eye is sensitive to image luminance, in order to improve the image effect seen by the human eye, usually, the image block in the RGB format is converted into an image block in the YCoCg format and then encoded and decoded. However, if the image block corresponding to the block to be decoded obtained is in the RGB format, it means that the encoding side has determined that the code length required for encoding the image block in the converted YCoCg format is long, and the decoding side may directly decode the bit stream of each component of the RGB and obtain an image block in the RGB format.

[0274] Next, the decoding side may determine the first fixed code length of the component based on the number and sub-code length of the sub-blocks to be decoded in each component. For the specific process, reference may be made to step S201 above, and the description is omitted here.

[0275] In some other embodiments, when the block to be decoded contains a plurality of components, the decoding side may first divide the block to be decoded into one or more sub - decoding blocks, and then, for each sub - decoding block, divide the sub - decoding block into sub - blocks of each component according to the ratio between the components.

[0276] Optionally, based on the related description in step S103 above, the fallback mode may be used when there is no overflow in the block to be encoded. Correspondingly, in this case, the decoding side may determine that the code length value of the block to be decoded is another code length value greater than the target code length when the occupied memory in the bit - stream buffer is less than or equal to the first threshold of the total memory of the bit - stream buffer. Exemplarily, the first threshold may be 30% of the total memory of the bit - stream buffer. Therefore, the decoding side may also determine that the code length of the block to be decoded is 1.5 times the target code length. Optionally, when the occupied memory in the bit - stream buffer is greater than or equal to the second threshold of the total memory of the bit - stream buffer, the decoding side may determine that the code length value of the block to be decoded is another code length value less than the target code length. Exemplarily, the second threshold may be 85% of the total memory of the bit - stream buffer. Therefore, the decoding side may also determine that the code length of the block to be decoded is 0.5 times the target code length. When the occupied memory in the bit - stream buffer is less than the second threshold and greater than the first threshold, the decoding side may also determine that the target code length is the code length value of the block to be decoded.

[0277] Optionally, the information on whether there is an overflow may be added to the bit - stream of the block to be decoded. Therefore, the decoding side may also determine whether an overflow occurs when encoding the image block based on the bit - stream of the block to be decoded.

[0278] Optionally, the decoding side may determine the first fixed code length based on the quantization step of the block to be decoded, which is analyzed based on the header information of the block to be decoded, or determined based on per-pixel target bits, or determined in other similar ways.

[0279] In S303, the decoding side analyzes the bitstream of the block to be decoded based on the first fixed code length to decode the block to be decoded.

[0280] In one possible implementation, the decoding side may obtain the first fixed code length and analyze the original pixel values of the image block corresponding to each sub-block to be decoded based on the first fixed code length.

[0281] Optionally, when the encoding side quantizes the original pixel values of the image block, the decoding side may inverse-quantize the value analyzed from the bitstream based on the obtained quantization step.

[0282] Optionally, when the encoding side converts the original pixel values of the image block before quantization, the decoding side may inverse-quantize the value analyzed from the bitstream and then further inverse-convert the inverse-quantized value.

[0283] In one example, the decoding side analyzes the bitstream of the image block corresponding to the sub-block to be decoded to obtain the upper-bit value of the reconstructed pixel value corresponding to the sub-block to be decoded, inverse-quantize the upper-bit value of the reconstructed pixel value to obtain the upper-bit value of the inverse-quantized reconstructed pixel value, combine it with the lower-bit value of the predicted value, and reconstruct the sub-block to be decoded.

[0284] Here, the upper-bit value of the reconstructed pixel value and the lower-bit value of the predicted value may refer to the relevant description in step S202, and the description is omitted here.

[0285] In another example, the decoding side may analyze the bit stream of the image block corresponding to the sub-block to be decoded, obtain the lower bit value of the reconstructed pixel value corresponding to the sub-block to be decoded, inverse quantize the lower bit value of the reconstructed pixel value to obtain the lower bit value of the inverse quantized reconstructed pixel value, combine it with the value obtained by supplementing zeros to the upper bit value of the predicted value, and reconstruct the sub-block to be decoded.

[0286] Here, the lower bit value of the reconstructed pixel value and the upper bit value of the predicted value may refer to the relevant description in step S202 above, and the description is omitted here.

[0287] In another possible implementation, the decoding side may obtain a fixed code length and analyze the residual value of each sub-block to be decoded based on the fixed code length. For any sub-block to be decoded, obtain the predicted value of the sub-block to be decoded.

[0288] Next, based on the predicted value and the residual value of the sub-block to be decoded, reconstruct the image block of the block to be decoded.

[0289] Optionally, when the encoding side quantizes the residual value of the image block, the decoding side may inverse quantize the value analyzed from the bit stream based on the obtained quantization step.

[0290] Optionally, when the encoding side converts the original pixel value of the image block before quantization, the decoding side may inverse quantize the value analyzed from the bit stream and then further inverse transform the inverse quantized value.

[0291] As an option, for each sub-block to be decoded, when the coefficient value (e.g., residual value, original value, etc.) of the sub-block to be decoded analyzed based on the first fixed code length is at the boundary of the representable range of the first fixed code length, in the transmission process of the bit stream of the sub-block, a suffix for indicating the coefficient value of the sub-block to be decoded may be included. For example, when the first fixed code length of the sub-block to be decoded is 3, since the coefficient value range representable by 3-bit binary data is [0, 7], when the suffix 0 is included in the bit stream of the sub-block, it can represent that the current coefficient value is in the range [1, 8]. Note that the suffix may be analyzed according to a preset suffix setting rule. Also, the preset suffix setting rule on the decoding side is the same as the preset suffix setting rule on the encoding side.

[0292] Based on the above embodiments, when the encoding side uses the fallback mode, the decoding side may decode the block to be decoded in the fallback mode, and can prevent the overflow of the bit stream. Thereby, it is possible to prevent the image information of the block to be decoded from being lost.

[0293] In some embodiments, the image decoding process provided by the present invention may be specifically represented as the logical flowchart shown in FIG. 11. As shown in FIG. 11, the decoding process may include the following steps.

[0294] In S1, the decoding side analyzes the header information of the block to be decoded and determines the quantization step of the block to be decoded.

[0295] In S2, the decoding side analyzes the codeword for indicating the prediction mode in the first decoded component of the block to be decoded and determines whether it is the first codeword.

[0296] Here, the first decoding component is any one of a plurality of components of the block to be decoded. Also, when the codeword for indicating the original value mode and the codeword for indicating the fallback mode are the same, and both are the above first codeword, when the decoding side analyzes the bit stream of the block to be decoded to obtain the first codeword, the decoding side may determine that the fallback mode or the original value mode is used for the block to be decoded.

[0297] When the decoding side determines that the codeword for indicating the prediction mode in the first decoding component is the first codeword, the following step S3 is executed.

[0298] Otherwise, when the decoding side determines that the codeword for indicating the prediction mode in the first decoding component is not the first codeword, the following step S13 is executed.

[0299] In S3, the decoding side determines whether the original value mode is used for the first decoding component based on the state of the bit stream buffer.

[0300] Referring to the related description in step S301 above, the decoding side may determine that the fallback mode or the original value mode is used for the block to be decoded based on the size of the occupied memory space in the bit stream buffer, and the description is omitted here.

[0301] When the decoding side determines that the original value mode is used for the first decoding component, the following step S4 is executed.

[0302] Otherwise, when the decoding side determines that the original value mode is not used for the first decoding component, the following step S6 is executed.

[0303] In S4, the decoding side determines the first fixed code length as the image bit width of the image block indicated by the first decoding component, and decodes the first decoding component based on the original value mode.

[0304] Here, for the process of determining the first fixed code length, reference may be made to step S302 above, and the description is omitted here.

[0305] In S5, the decoding side decodes the other components of the block to be decoded based on the original value mode.

[0306] In S6, the decoding side determines that the fallback mode is used for the first decoding component, and determines whether the fallback mode is used for the other components.

[0307] For any component for which the fallback mode is used, the decoding side executes step S7.

[0308] For any component for which the fallback mode is not used, the decoding side executes the following step S13 to decode the component.

[0309] In S7, the decoding side determines the prediction mode used for the component.

[0310] Here, the prediction modes used for the respective components of the block to be decoded may be the same or different. For the specific process, reference may be made to the related description in step S301 above, and the description is omitted here.

[0311] In S8, the decoding side analyzes the relative position information of the prediction block in the prediction mode based on the prediction mode used for the component, and determines the predicted value of the component.

[0312] In S9, the decoding side determines the total code length assigned to the component, divides the component into a plurality of sub-blocks to be decoded, and determines the fixed code length of each sub-block to be decoded of the component.

[0313] On the decoding side, the fixed code length of each sub-block to be decoded may be determined based on the quantization step of the block to be decoded. Alternatively, the decoding side may analyze the bit stream of each sub-block to be decoded to determine the fixed code length of each sub-block to be decoded.

[0314] In S10, the residual value of the component is analyzed based on the fixed code length of each sub-block to be decoded.

[0315] In S11, the residual value is inverse quantized based on the quantization step of the component.

[0316] In S12, a reconstructed value of the component is determined based on the predicted value and the inverse quantized residual value of the component.

[0317] In S13, the prediction mode of the component is determined, other information of the component is normally analyzed, and the reconstruction of the component is completed.

[0318] When the decoding side executes step S13, the decoding process of steps 31 to 36 above may be referred to, and the description is omitted here.

[0319] In some embodiments, the present invention further provides an image encoding method as shown in FIG. 12. The image encoding method may be executed by an encoder 100 or may be executed by an encoding side (e.g., the encoding side 10 shown in FIG. 1) that supports the functions of the encoder 100. Here, it is described by taking the implementation of the encoding method by the encoding side as an example. The image encoding method includes the following steps.

[0320] In S401, the encoding side obtains the minimum code length of the block to be encoded.

[0321] Here, the minimum code length is the minimum bit stream length that is currently allowed to be cached in the storage space for the encoding side to cache the encoded bit stream.

[0322] The block to be coded may be referred to as a coding unit. Usually, when coding an image frame in the video to be coded, the coding side codes using the coding unit as a unit.

[0323] Also, the above minimum code length is used to indicate the minimum code length that can be coded for the block to be coded. Note that in order to prevent an underflow of the bitstream, the minimum code length may be determined based on the size of the storage space of the coding buffer.

[0324] Optionally, the sum of the difference between the bitstream length stored in the storage space and the code length output per unit time by the storage space and the minimum code length is greater than or equal to the code length output per unit time by the storage space.

[0325] Note that the bitstream buffer receives the bitstream of the current block coded by the encoder, and at the same time, a certain amount of bitstream flows out. If the bitstream in the current bitstream buffer is empty, or if the bitstream length in the current bitstream buffer is smaller than the code length flowing out of the buffer per unit time, an "underflow" of the bitstream occurs, and the bitstream cannot be transmitted normally. The minimum code length may be the code length that guarantees that the bitstream in the bitstream buffer is transmitted normally.

[0326] In S402, the coding side pre-codes the block to be coded based on the first mode to obtain the first code length of the block to be coded.

[0327] The first mode may be one of a plurality of prediction modes on the coding side, and the first code length is the length of the bitstream obtained after coding the block to be coded.

[0328] Here, for a specific description of step S402, reference may be made to the related description in step S102 above, and the description is omitted here.

[0329] In S403, when the first code length is less than or equal to the minimum code length, the encoding side encodes the block to be encoded in the fallback mode, and the code length obtained by encoding the block to be encoded based on the fallback mode is greater than or equal to the minimum code length.

[0330] Here, for a specific description of step S403, reference may be made to the related description in step S103 above, and the description is omitted here.

[0331] The image encoding method provided by the embodiments of the present invention has at least the following beneficial effects. First, the block to be encoded is pre-encoded, the length of the bitstream obtained by the pre-encoding is compared with the minimum code length that is allowed to be occupied by the block to be encoded, and based on the comparison result, it is possible to determine whether an underflow of the bitstream occurs. Next, when it is determined that there may be an underflow of the bitstream, the block to be encoded in the fallback mode is encoded to ensure that the bitstream in the bitstream buffer is normally transmitted.

[0332] In some embodiments, the present invention further provides an image decoding method as shown in FIG. 13. The decoding method corresponds to the encoding method shown in FIG. 12. The image decoding method may be executed by a decoder 200 or may be executed by a decoding side (for example, the decoding side 20 shown in FIG. 1) that supports the functions of the decoder 200. Here, taking the decoder 200 implementing the decoding method as an example for explanation, the image decoding method includes the following steps.

[0333] In S501, the decoding side determines whether the fallback mode is used during encoding for the image block corresponding to the block to be decoded based on the bitstream of the block to be decoded.

[0334] Here, when an image block is encoded based on the fallback mode, the code length obtained is not less than the minimum code length of the block to be decoded. The minimum code length is the minimum bitstream length that is allowed to be cached in the storage space for caching the encoded bitstream when the image block corresponding to the block to be decoded is encoded.

[0335] Here, the specific implementation of step S501 may refer to the related description of step S301 above, and the description is omitted here.

[0336] In S502, when the fallback mode is used during encoding for the image block corresponding to the block to be decoded on the decoding side, the decoding side obtains a first fixed code length based on the bitstream.

[0337] The first fixed code length is the code length of one sub-block to be decoded, and one block to be decoded may include a plurality of sub-blocks to be decoded.

[0338] Here, the specific implementation of step S502 may refer to the related description of step S302 above, and the description is omitted here.

[0339] In S503, the decoding side analyzes the bitstream based on the first fixed code length to decode the block to be decoded.

[0340] Here, the specific implementation of step S503 may refer to the related description of step S303 above, and the description is omitted here.

[0341] In some embodiments, the present invention further provides an image encoding method as shown in FIG. 14. The image encoding method may be executed by an encoder 100, or may be executed by an encoding side (e.g., the encoding side 10 shown in FIG. 1) that supports the functions of the encoder 100. Here, the case where the encoding side implements the encoding method is taken as an example for explanation. The image encoding method includes the following steps.

[0342] In S601, the encoding side obtains the maximum code length of the block to be encoded.

[0343] Here, the maximum code length is the maximum bitstream length that is currently allowed to be cached in the storage space for the encoding side to cache the encoded bitstream.

[0344] Here, the specific implementation of step S601 may refer to the related description of step S101 above, and the description is omitted here.

[0345] In S602, the encoding side pre-encodes the block to be encoded based on the first mode to obtain the first code length of the block to be encoded.

[0346] Here, the first mode is one of a plurality of prediction modes of the encoding side, and the first code length is the length of the bitstream obtained after encoding the block to be encoded.

[0347] Here, the specific implementation of step S602 may refer to the related description of step S102 above, and the description is omitted here.

[0348] In S603, when the first code length is greater than or equal to the maximum code length, the encoding side encodes the block to be encoded in the second mode based on the skip residual mode.

[0349] Here, based on the skip residual mode, the code length obtained by encoding the block to be encoded in the second mode is less than or equal to the maximum code length, and the second mode may be one of a plurality of prediction modes on the encoding side.

[0350] Also, in the skip residual mode, there is no need to encode (decode) the residual coefficients. In this case, the residual value of the pixel in the current image block is regarded as 0, that is, the reconstructed value of each pixel is equal to the predicted value of the pixel. Therefore, the encoding side only needs to encode the information indicating the second mode. For example, the identifier of the second mode is encoded to obtain a bitstream of the block to be encoded with the identifier of the second mode added.

[0351] Note that for pixels with large residual values, based on the skip residual mode, the distortion of the encoding and decoding processes increases. Therefore, in practice, when the residual value is large, the encoding side does not select the skip residual mode. However, when the first code length is greater than or equal to the maximum code length, in order to prevent the overflow of the bitstream, regardless of the magnitude of the residual value, the encoding side uses the skip residual mode, that is, the forced skip residual encoding mode.

[0352] Optionally, the second mode may be a target mode. Here, the target mode may be the optimal prediction mode. In this case, better encoding performance can be maintained by encoding the block to be encoded based on the target mode.

[0353] In another possible implementation, the second mode may be the same as the first mode. Optionally, the second mode may be a preset prediction mode.

[0354] As can be understood, the encoding side may preset a prediction mode with better encoding performance based on the skip residual mode as the second mode. When the first code length is greater than or equal to the maximum code length, regardless of which prediction mode the optimal prediction mode is, the block to be encoded is encoded in the preset second mode.

[0355] Optionally, the encoding side may encode the block to be encoded by using a plurality of prediction modes such as the point-by-point prediction mode, the intra-frame prediction mode, the block copy mode, and the original value mode based on the skip residual mode, and determine the optimal prediction mode based on the skip residual mode.

[0356] As can be understood, based on the encoding method, when encoding in the skip residual mode, the occurrence of bitstream overflow can be prevented.

[0357] In some embodiments, the present invention further provides an image decoding method as shown in FIG. 15. The decoding method corresponds to the encoding method shown in FIG. 14. The image decoding method may be executed by the decoder 200 or may be executed by a decoding side (for example, the decoding side 20 shown in FIG. 1) that supports the functions of the decoder 200. Here, the case where the decoder 200 implements the decoding method is taken as an example for description. The image decoding method includes the following steps.

[0358] In S701, the decoding side analyzes the bitstream of the block to be decoded to obtain the indication information of the second mode for predicting the block to be decoded.

[0359] Here, the second mode may be any one of a plurality of prediction modes, and reference may be made to the description of the second mode in the above image encoding method. The indication information of the second mode may be an identifier of the second mode.

[0360] In S702, the decoding side determines the predicted value of the block to be decoded based on the second mode.

[0361] Here, the specific process for the decoding side to determine the predicted value of the block to be decoded may be determined based on the second mode.

[0362] For example, the second mode is a vertical average value prediction mode. Therefore, on the decoding side, the predicted value of the pixel to be predicted may be obtained using the reconstructed values of the pixels above and below the pixel to be predicted. This will not be enumerated here.

[0363] In S703, on the decoding side, the predicted value of the block to be decoded is determined as the reconstructed value of the block to be decoded.

[0364] In some embodiments, the present invention further provides another method for determining the predicted value of an image block, which will be described below.

[0365] On the encoding side, first, the reconstructed pixels around the block to be encoded may be obtained, including the following several possible cases.

[0366] Case 1: The encoding side cannot obtain the reconstructed pixels of the row before the block to be encoded and cannot obtain the reconstructed pixels of the column before the block to be encoded.

[0367] In this case, the encoding side may determine the predicted value of the block to be encoded based on the prediction mode in the Huffman table 1. The prediction mode in the Huffman table 1 includes a point-by-point prediction mode, an original value mode, and a fallback mode. Note that when the encoding side cannot obtain the reconstructed pixels of the row before the block to be encoded and cannot obtain the reconstructed pixels of the column before the block to be encoded, the encoding side may also determine the predicted value of the block to be encoded based on the point-by-point prediction mode, the original value mode, or the fallback mode.

[0368] Here, Huffman coding is an entropy coding algorithm for lossless data compression. The Huffman table is a code table that generates codewords based on probabilities. When the encoding side provides n prediction modes, the Huffman table can generate encoded codewords corresponding to each of these n prediction modes based on the selection probabilities of each prediction mode among these n prediction modes. The codeword of the prediction mode with a large selection probability is short, and the codeword of the prediction mode with a small selection probability is long. In this way, the average bit rate after encoding can be reduced. Note that the Huffman table 1 includes the data symbols before encoding in the point-by-point prediction mode, the original value mode, or the fallback mode, and the encoded codewords corresponding to each mode.

[0369] Case 2: The encoding side can obtain the reconstructed pixels of the row before the block to be encoded, but cannot obtain the reconstructed pixels of the column before the block to be encoded.

[0370] The encoding side may use the reconstructed pixels of the previous row as reference pixels and determine the predicted value of the block to be encoded based on the prediction mode in the Huffman table 2. Here, the prediction mode in the Huffman table 2 includes the point-by-point prediction mode, the original value mode, the fallback mode, the modes among other intra-frame prediction modes that can use the previous row as a reference pixel, and the encoded codewords corresponding to each mode.

[0371] Case 3: The encoding side can obtain the reconstructed pixels of the column before the block to be encoded, but cannot obtain the reconstructed pixels of the row before the block to be encoded.

[0372] The symbolization side may use the reconstructed pixels of the previous column as reference pixels and determine the predicted value of the block to be symbolized based on the prediction mode in the Huffman table 3. Here, the prediction mode in the Huffman table 3 includes the point-by-point prediction mode, the block prediction mode, the original value mode, the fallback mode, and the intra-frame prediction mode, which are prediction modes that use the previous column as a reference pixel and the encoded codewords corresponding to each mode.

[0373] Case 4: The symbolization side can obtain the reconstructed pixels of the previous column and the reconstructed pixels of the previous row of the block to be symbolized.

[0374] The symbolization side may use the reconstructed pixels of the previous column and / or the previous row as reference pixels and determine the predicted value of the block to be symbolized based on the prediction mode in the Huffman table 4. Here, the prediction mode in the Huffman table 4 may include all possible prediction modes. For the enumeration and description of the prediction modes, reference may be made to the relevant description in step S102 above, and the description is omitted here.

[0375] Correspondingly, when determining the predicted value of the block to be decoded, the decoding side may first obtain the reconstructed pixels around the block to be decoded. Similarly, there are four possible cases corresponding to the above symbolization side.

[0376] Specifically, when corresponding to Case 1, if the decoding side cannot obtain the reconstructed pixels of the row before the block to be decoded and cannot obtain the reconstructed pixels of the column before the block to be decoded, the decoding side may query the Huffman table 1 based on the codeword for indicating the prediction mode in the block to be decoded to determine the prediction mode, and determine the predicted value of the block to be decoded. When corresponding to Case 2, if the decoding side can obtain the reconstructed pixels of the row before the block to be decoded but cannot obtain the reconstructed pixels of the column before the block to be decoded, the decoding side may query the Huffman table 2 based on the codeword for indicating the prediction mode in the block to be decoded to determine the prediction mode, and determine the predicted value of the block to be decoded. When corresponding to Case 3, if the decoding side can obtain the reconstructed pixels of the column before the block to be decoded but cannot obtain the reconstructed pixels of the row before the block to be decoded, the decoding side may query the Huffman table 3 based on the codeword for indicating the prediction mode in the block to be decoded to determine the prediction mode, and determine the predicted value of the block to be decoded. When corresponding to Case 4, if the decoding side can obtain the reconstructed pixels of the column and the row before the block to be decoded, the decoding side may query the Huffman table 3 based on the codeword for indicating the prediction mode in the block to be decoded to determine the prediction mode, and determine the predicted value of the block to be decoded.

[0377] In some embodiments, a semi-fixed length coding method for coding the sub-blocks to be coded in each of the above embodiments is further provided. As long as there is no contradiction, the semi-fixed length coding method may be used in the methods provided by the above embodiments.

[0378] Here, the semi-fixed length encoding method means that for one or a plurality of sub-blocks to be encoded in a block to be encoded, the encoding side can encode each sub-block to be encoded based on different code lengths. For a sub-block to be encoded including a plurality of pixels, the encoding side can encode each pixel with one fixed code length and arrange the codeword encoded with the fixed code length at the prefix of the bit stream of the sub-block to be encoded.

[0379] Correspondingly, the decoding side may analyze the prefix of each sub-block to be decoded to analyze each fixed code length of each sub-block to be decoded. For one sub-block to be decoded, the decoding side may decode each pixel based on the fixed code length value obtained by analyzing the sub-block to be decoded.

[0380] Hereinafter, the encoding device and the decoding device provided by the embodiments of the present invention will be described.

[0381] In one example, any decoding device provided by the embodiments of the present invention may be the decoding side 20 or the decoder 200 in FIG. 1. In another example, any encoding device provided hereinafter may be the encoding side 10 or the encoder 100 in FIG. 1. Hereinafter, the description thereof will be omitted.

[0382] FIG. 16 is a schematic structural diagram of an image encoding apparatus 1600 provided by the present invention, and any of the above-described encoding method embodiments can be executed by the encoding apparatus 1600. The image encoding apparatus 1600 includes an acquisition module 1601, a processing module 1602, and an encoding module 1603. The acquisition module 1601 is used to obtain the maximum code length of a block to be encoded, and the maximum code length is the maximum bit stream length that is currently allowed to be cached in the storage space for the encoding side to cache the encoded bit stream. The processing module 1602 is used to pre-encode a block to be encoded based on a first mode to obtain a first code length of the block to be encoded. The first mode is one of a plurality of prediction modes on the encoding side, and the first code length is the length of the bit stream obtained after encoding the block to be encoded. The encoding module 1603 is used to encode the block to be encoded in a fallback mode when the first code length is greater than or equal to the maximum code length. The code length obtained by encoding the block to be encoded based on the fallback mode is less than or equal to the maximum code length.

[0383] For more detailed descriptions of the above acquisition module 1601, processing module 1602, and encoding module 1603, more detailed descriptions of their respective technical features, and descriptions of beneficial effects, etc., reference can be made to the corresponding method embodiment parts above, and the description is omitted here.

[0384] FIG. 17 is a schematic structural diagram of an image decoding apparatus 1700 provided by the present invention, and any of the above-described decoding method embodiments can be executed by the decoding apparatus 1700. The image decoding apparatus 1700 includes a determination module 1701 and an analysis module 1702. The determination module 1701 is used to determine whether a fallback mode is used for an image block corresponding to a block to be decoded based on the bit stream of the block to be decoded. The determination module 1701 is further used to obtain a first fixed code length when the fallback mode is used for the image block corresponding to the block to be decoded. The analysis module 1702 is used to analyze the bit stream of the block to be decoded and decode the block to be decoded based on the first fixed code length.

[0385] For more detailed descriptions of the above determination module 1701 and analysis module 1702, more detailed descriptions of their respective technical features, and descriptions of beneficial effects, etc., reference can be made to the corresponding method embodiment parts above, and the description is omitted here.

[0386] FIG. 18 is a schematic structural diagram of an image encoding apparatus 1800 provided by the present invention, and any of the above-described encoding method embodiments can be executed by the encoding apparatus 1800. The image encoding apparatus 1800 includes an acquisition module 1801, a processing module 1802, and an encoding module 1803. The acquisition module 1801 is used to obtain the minimum code length of a block to be encoded, and the minimum code length is the minimum bit stream length that is currently allowed to be cached in the storage space for the encoding side to cache the encoded bit stream. The processing module 1802 is used to pre-encode a block to be encoded based on a first mode to obtain a first code length of the block to be encoded. The first mode is one of a plurality of prediction modes on the encoding side, and the first code length is the length of the bit stream obtained after encoding the block to be encoded. The encoding module 1803 is used to encode the block to be encoded in a fallback mode when the first code length is less than or equal to the minimum code length. The code length obtained by encoding the block to be encoded based on the fallback mode is greater than or equal to the minimum code length.

[0387] For more detailed descriptions of the above acquisition module 1801, processing module 1802, and encoding module 1803, more detailed descriptions of their respective technical features, and descriptions of beneficial effects, etc., reference may be made to the corresponding method embodiment parts above, and the description is omitted here.

[0388] FIG. 19 is a schematic structural diagram of an image decoding apparatus 1900 provided by the present invention. Any of the above-described decoding method embodiments can be executed by the decoding apparatus 1900. The image decoding apparatus 1900 includes a determination module 1901 and an analysis module 1902. The determination module 1901 is used to determine whether a fallback mode is used during encoding for an image block corresponding to a block to be decoded, based on the bitstream of the block to be decoded. When the image block is encoded based on the fallback mode, the code length obtained is greater than or equal to the minimum code length of the block to be decoded. The minimum code length is the minimum bitstream length that is allowed to be cached in the storage space for caching the encoded bitstream when the image block is encoded. The determination module 1901 is further used to obtain a first fixed code length based on the bitstream when the fallback mode is used during encoding for the image block corresponding to the block to be decoded. The first fixed code length is the code length of one sub-block to be decoded, and one block to be decoded may include a plurality of sub-blocks to be decoded. The analysis module 1902 is used to analyze the bitstream of the block to be decoded based on the first fixed code length to decode the block to be decoded.

[0389] For more detailed descriptions of the above determination module 1901 and analysis module 1902, more detailed descriptions of their respective technical features, and descriptions of beneficial effects, etc., reference may be made to the corresponding method embodiment parts above, and the description is omitted here.

[0390] FIG. 20 is a schematic structural diagram of an image encoding apparatus 2000 provided by the present invention, and any of the above-described encoding method embodiments can be executed by the encoding apparatus 2000. The image encoding apparatus 2000 includes an acquisition module 2001, a processing module 2002, and an encoding module 2003. The acquisition module 2001 is used to obtain the maximum code length of the block to be encoded, and the maximum code length is the maximum bit stream length that is currently allowed to be cached in the storage space for the encoding side to cache the encoded bit stream. The processing module 2002 is used to pre-encode the block to be encoded based on the first mode to obtain the first code length of the block to be encoded. The first mode is one of a plurality of prediction modes on the encoding side, and the first code length is the length of the bit stream obtained after encoding the block to be encoded. The encoding module 2003 is used to encode the block to be encoded in the second mode based on the skip residual mode when the first code length is greater than or equal to the maximum code length. Based on the skip residual mode, the code length obtained by encoding the block to be encoded in the second mode is less than or equal to the maximum code length, and the second mode is one of a plurality of prediction modes on the encoding side.

[0391] For more detailed descriptions of the above acquisition module 2001, processing module 2002, and encoding module 2003, more detailed descriptions of their respective technical features, and descriptions of beneficial effects, etc., any of them may refer to the corresponding method embodiment part above, and the description is omitted here.

[0392] FIG. 21 is a schematic structural diagram of an image decoding apparatus 2100 provided by the present invention, and any of the above-described decoding method embodiments can be executed by the decoding apparatus 2100. The image decoding apparatus 2100 includes an analysis module 2101, a prediction module 2102, and a reconstruction module 2103. The analysis module 2101 is used to analyze the bit stream of the block to be decoded and obtain an identifier of a second mode for predicting the block to be decoded. The prediction module 2102 is used to determine a predicted value of the block to be decoded based on the second mode. The reconstruction module 2103 is used to determine the predicted value of the block to be decoded as a reconstructed value of the block to be decoded.

[0393] For more detailed descriptions of the above analysis module 2101, prediction module 2102, and reconstruction module 2103, more detailed descriptions of their respective technical features, and descriptions of beneficial effects, etc., reference may be made to the corresponding method embodiment parts above, and the description is omitted here.

[0394] The present invention further provides an electronic device 2200 for executing any of the above-described image encoding / decoding method embodiments. As shown in FIG. 22, FIG. 22 is a schematic structural diagram of the electronic device 2200 provided by the present invention, and the electronic device 2200 includes a processor 2201 and a communication interface 2202. The processor 2201 and the communication interface 2202 are coupled to each other. As can be understood, the communication interface 2202 may be a transceiver or an input / output interface.

[0395] In one example, the electronic device 2200 may further include a memory 2203 for storing instructions executed by the processor 2201, or for storing input data required for the processor 2201 to execute instructions, or for storing data generated after the processor 2201 executes instructions.

[0396] In the embodiments of the present invention, there is no limitation on the specific connection medium among the above communication interface 2202, processor 2201, and memory 2203. In the embodiments of the present invention, in FIG. 22, the communication interface 2202, processor 2201, and memory 2203 are connected via a bus 2204, and the bus is shown as a thick line in FIG. 22. The connection manners among other components are merely exemplary and are not limited thereto. The bus may include an address bus, a data bus, a control bus, and the like. For the sake of simplicity of expression, only one thick line is shown in FIG. 22, but it does not mean that there is only one bus or only one type of bus.

[0397] The memory 2203 may be used to store software programs and modules such as program instructions / modules corresponding to the image decoding method or the image encoding method provided by the embodiments of the present invention. The processor 2201 may execute the software programs and modules stored in the memory 2203 to execute various functional applications and data processing, thereby implementing any of the image decoding methods or image encoding methods provided above. The communication interface 2202 may be used for signaling and data communication with other devices. In the present invention, the electronic device 2200 may have a plurality of communication interfaces 2202.

[0398] For better understanding, the processor in the embodiments of the present invention may be a central processing unit (CPU), a neural processing unit (NPU), or a graphic processing unit (GPU), and may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0399] The steps of the method in the embodiments of the present invention may be implemented by hardware or by a processor executing software instructions. The software instructions may be composed of corresponding software modules, and the software modules may be stored in a random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), register, hard disk, removable hard disk, CD-ROM, or any other form of storage medium well-known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from and write information to the storage medium. Of course, the storage medium may be a component of the processor. The processor and the storage medium may be disposed within an ASIC. Further, the ASIC may be disposed within a network device or a terminal device. Of course, the processor and the storage medium may exist within a network device or a terminal device as independent components.

[0400] Embodiments of the present invention further provide an encoding and decoding system including an encoding side and a decoding side. The encoding side may be used to execute any of the image encoding methods provided above, and the decoding side may be used to execute a corresponding image decoding method.

[0401] In the above embodiments, all or part of them may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of it may be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are executed. The computer may be a general-purpose computer, a dedicated computer, a computer network, a network device, a user device, or other programmable devices. The computer program or instructions may be stored in a computer-readable storage medium, or may be transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire or wirelessly. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device such as a server or a data center that integrates one or more available media. The available medium may be a magnetic medium such as a floppy disk, a hard disk, or a magnetic tape, an optical medium such as a digital video disc (DVD), or a semiconductor medium such as a solid state drive (SSD).

[0402] In each embodiment of the present invention, if there is no special explanation or logical contradiction, the terms and / or descriptions between different embodiments are consistent and can be cited from each other, and the technical features in different embodiments can be combined according to their internal logical relationships to form new embodiments.

[0403] As can be understood, the various numerical numbers related to the embodiments of the present invention are merely for convenience of explanation and are not intended to limit the scope of the embodiments of the present invention. The magnitude of the numbers of each of the above processes does not mean the execution order, and the execution order of each process should be determined by its function and internal logic.

Claims

Claim 1 An image decoding method applied to the decoding side, comprising: determining whether a fallback mode is used for an image block corresponding to the block to be decoded based on a bit stream of the block to be decoded, wherein a code length of the block to be decoded is less than or equal to a maximum code length of the block to be decoded, and the maximum code length is a maximum bit stream length that is allowed to be cached in a bit stream buffer; when the fallback mode is used for the image block corresponding to the block to be decoded, obtaining a first fixed code length, where the first fixed code length is a code length of one sub-block to be decoded, and the block to be decoded includes one or more of the sub-blocks to be decoded; analyzing the bit stream based on the first fixed code length to decode the block to be decoded. An image decoding method, characterized by the above. Claim 2 The block to be decoded includes one or more components, and the decoding side determines whether the fallback mode is used for each of the one or more components, respectively. The method according to claim 1, characterized by the above. Claim 3 The block to be decoded includes a first chrominance component, a second chrominance component, and a luminance component. Based on the bit stream of the block to be decoded, the step of determining whether the fallback mode is used for the image block corresponding to the block to be decoded during encoding includes: analyzing a bit stream of any one of the first chrominance component, the second chrominance component, or the luminance component to determine whether the fallback mode is used for the first chrominance component, the second chrominance component, and the luminance component. The method according to claim 1, characterized by the above. Claim 4 A sum of a code length of the first chrominance component, a code length of the second chrominance component, and a code length of the luminance component is a code length of the block to be decoded, and the code length of the luminance component is greater than the code length of the first chrominance component or the code length of the second chrominance component. The method according to claim 3, characterized by the above. Claim 5 The first fixed code length is a preset code length value or the target bits per pixel of the block to be decoded, and the target bits per pixel is used to indicate the code length required when decoding each pixel of the block to be decoded at the target compression ratio. The method according to any one of claims 1 to 4, characterized in that.

6. The step of analyzing the bitstream based on the first fixed code length to decode the block to be decoded includes: Based on the code length of the block to be decoded and the first fixed code length, dividing the block to be decoded into a plurality of sub-blocks to be decoded; Based on the first fixed code length, analyzing the bitstream of each sub-block to be decoded of the block to be decoded to decode the block to be decoded. The method according to claim 5, characterized in that.

7. The step of analyzing the bitstream based on the first fixed code length to decode the block to be decoded includes: Based on the first fixed code length, analyzing the bitstream of each sub-block to be decoded of the block to be decoded to obtain a residual value of each sub-block to be decoded; Based on the predicted value and the residual value of each sub-block to be decoded, reconstructing each sub-block to be decoded to obtain a reconstructed block of each sub-block to be decoded. The method according to any one of claims 1 to 6, characterized in that.

8. When the codeword for indicating the prediction mode in the bitstream of the block to be decoded is the first codeword, and the fallback mode is not used for the image block corresponding to the block to be decoded, further including the step of reconstructing the image block of the block to be decoded based on the original value mode, and the first codeword is used to indicate the fallback mode or the original value mode. The method according to any one of claims 1 to 6, characterized in that.

9. An image encoding method applied to the encoding side, including: Obtaining the maximum code length of the block to be encoded, where the maximum code length is the maximum bitstream length allowed to be cached in the bitstream buffer. Based on the first mode, pre-encoding the block to be encoded to obtain a first code length of the block to be encoded, where the first mode is one of a plurality of prediction modes on the encoding side, and the first code length is the length of the bitstream obtained after encoding the block to be encoded, the step of; When the first code length is greater than or equal to the maximum code length, encoding the block to be encoded in the fallback mode, where the code length obtained by encoding the block to be encoded based on the fallback mode is less than or equal to the maximum code length, the step of, including; An image encoding method characterized by this.

10. The block to be encoded includes a first chrominance component, a second chrominance component, and a luminance component. When the first code length is greater than or equal to the maximum code length, the step of encoding the block to be encoded in the fallback mode is; When the first code length is greater than or equal to the maximum code length, determining the code length of each component according to a preset ratio based on the maximum code length, where the sum of the first chrominance maximum code length, the second chrominance maximum code length, and the luminance maximum code length is the maximum code length or the code length flowing out of the bitstream buffer per unit time, the step of; Based on one or more of the first chrominance maximum code length, the second chrominance maximum code length, and the luminance maximum code length, determining whether to encode the first chrominance component, the second chrominance component, and the luminance component in the fallback mode, the step of, including; The method according to claim 9, characterized by this.

11. An image decoding method applied to the decoding side, Based on the bitstream of the block to be decoded, determining whether the fallback mode is used during encoding for the image block corresponding to the block to be decoded, where the code length obtained when the image block is encoded based on the fallback mode is greater than or equal to the minimum code length of the block to be decoded, and the minimum code length is the minimum bitstream length that is allowed to be cached in the bitstream buffer, the step of; When a fallback mode is used during encoding for an image block corresponding to the block to be decoded, a step of obtaining a first fixed code length based on the bit stream, where the first fixed code length is the code length of one sub-block to be decoded, and the block to be decoded includes one or more of the sub-blocks to be decoded, a step of analyzing the bit stream based on the first fixed code length to decode the block to be decoded, and including, An image decoding method characterized by this.

12. An image encoding method applied to the encoding side, a step of obtaining the minimum code length of the block to be encoded, where the minimum code length is the minimum bit stream length that is allowed to be cached in the bit stream buffer, a step of pre-encoding the block to be encoded based on a first mode to obtain a first code length of the block to be encoded, where the first mode is one of a plurality of prediction modes on the encoding side, and the first code length is the length of the bit stream obtained after encoding the block to be encoded, when the first code length is less than or equal to the minimum code length, a step of encoding the block to be encoded in a fallback mode, where the code length obtained by encoding the block to be encoded based on the fallback mode is greater than or equal to the minimum code length, and including, An image encoding method characterized by this.

13. Based on the bit stream of the block to be decoded, determine whether a fallback mode is used during encoding for the image block corresponding to the block to be decoded, When a fallback mode is used during encoding for an image block corresponding to a block to be decoded, a determination module for obtaining a first fixed code length based on the bitstream, wherein the code length obtained when the image block is encoded based on the fallback mode is less than or equal to the maximum code length of the block to be decoded, and the maximum code length is the maximum bitstream length that is allowed to be cached in a bitstream buffer, and the first fixed code length is the code length of one sub-block to be decoded, and the block to be decoded includes one or more of the sub-blocks to be decoded, the determination module; An analysis module for analyzing the bitstream based on the first fixed code length to decode the block to be decoded, and An image decoding apparatus characterized by the above.

14. An acquisition module for obtaining the maximum code length of a block to be encoded, wherein the maximum code length is the maximum bitstream length that is allowed to be cached in a bitstream buffer, the acquisition module; A processing module for pre-encoding the block to be encoded based on a first mode to obtain a first code length of the block to be encoded, wherein the first mode is one of a plurality of prediction modes on the encoding side, and the first code length is the length of the bitstream obtained after encoding the block to be encoded, the processing module; An encoding module for encoding the block to be encoded in a fallback mode when the first code length is greater than or equal to the maximum code length, wherein the code length obtained by encoding the block to be encoded based on the fallback mode is less than or equal to the maximum code length, the encoding module, and An image encoding apparatus characterized by the above.

15. Based on the bitstream of the block to be decoded, determine whether a fallback mode is used during encoding for the image block corresponding to the block to be decoded, When a fallback mode is used during encoding for an image block corresponding to the block to be decoded, a determination module for obtaining a first fixed code length based on the bitstream, wherein the code length obtained when the image block is encoded based on the fallback mode is not less than the minimum code length of the block to be decoded, and the minimum code length is the minimum bitstream length that is allowed to be cached in the bitstream buffer, and the first fixed code length is the code length of one sub-block to be decoded, and the block to be decoded includes one or more of the sub-blocks to be decoded, the determination module; An analysis module for analyzing the bitstream based on the first fixed code length to decode the block to be decoded. An image decoding apparatus characterized by the above.

16. An acquisition module for obtaining the minimum code length of a block to be encoded, wherein the minimum code length is the minimum bitstream length that is allowed to be cached in the bitstream buffer, the acquisition module; A processing module for pre-encoding the block to be encoded based on a first mode to obtain a first code length of the block to be encoded, wherein the first mode is one of a plurality of prediction modes on the encoding side, and the first code length is the length of the bitstream obtained after encoding the block to be encoded, the processing module; An encoding module for encoding the block to be encoded in a fallback mode when the first code length is not more than the minimum code length, wherein the code length obtained by encoding the block to be encoded based on the fallback mode is not less than the minimum code length, the encoding module. An image encoding apparatus characterized by the above.

17. An electronic device including a processor and a memory, wherein the memory is used to store computer instructions, and the processor calls and executes the computer instructions from the memory and is used to implement the method according to any one of Claims 1 to 10. An electronic device characterized by the above.

18. A computer-readable storage medium storing a computer program or instructions, wherein when the computer program or instructions are executed by an electronic device, the method according to any one of claims 1 to 10 is implemented. A computer-readable storage medium characterized by the above.

19. An electronic device including a processor and a memory, wherein the memory is used to store computer instructions, and the processor calls and executes the computer instructions from the memory and is used to implement the method according to any one of claims 11 and 12. An electronic device characterized by the above.

20. A computer-readable storage medium storing a computer program or instructions, wherein when the computer program or instructions are executed by an electronic device, the method according to any one of claims 11 and 12 is implemented. A computer-readable storage medium characterized by the above.

Images