Encoding method, decoding method and related apparatus

Abstract

The application provides an encoding method, a decoding method and related devices. The encoding method comprises: decomposing an image into wavelet coefficients of a low-frequency subband and wavelet coefficients of a high-frequency subband based on a wavelet transform architecture, and then independently encoding and decoding the wavelet coefficients of the low-frequency subband and the high-frequency subband. By using VLC encoding on the quantized coefficients of the high-frequency subband and using arithmetic encoding on control information, the encoding efficiency can be improved while reducing the encoding complexity.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202510121229.9, filed on January 24, 2025, entitled "Encoding Method, Decoding Method and Related Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of image processing, and in particular to an encoding method, a decoding method, and related apparatus. Background Technology

[0004] Digital video capabilities can be applied to a wide variety of digital video devices, including digital television, digital live broadcasting systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio phones (i.e., "smartphones"), video conferencing devices, video streaming devices, and the like. Digital video devices implement video compression technologies, such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 Advanced Video Coding (AVC), the H.265 / HEVC video coding standard, and extensions to such standards. By implementing such video compression technologies, digital video devices can transmit, receive, encode, decode, and / or store digital video information more efficiently.

[0005] Current encoding and decoding technologies suffer from high complexity. Summary of the Invention

[0006] This application provides an encoding method, a decoding method, and related apparatus to reduce encoding and decoding complexity.

[0007] Firstly, this application provides a decoding method, comprising: acquiring an image bitstream of an original image, the image bitstream including arithmetic-coded data and variable-length coded (VLC) data, wherein the arithmetic-coded data is obtained using arithmetic coding and the VLC-coded data is obtained using VLC entropy coding; acquiring control information based on the arithmetic-coded data; acquiring quantization coefficients of high-frequency subbands based on the control information and VLC-coded data; acquiring reconstruction coefficients of high-frequency subbands based on the quantization coefficients of high-frequency subbands; and performing an inverse wavelet transform on the reconstruction coefficients of high-frequency subbands to obtain a reconstructed image of the original image. Thus, by using VLC coding for some syntax elements of the high-frequency subbands and arithmetic coding for other syntax elements, this application allows the decoding end to decode the VLC-coded data and arithmetic-coded data separately using VLC decoding and CABAC decoding methods to obtain the quantization coefficients and control information of the high-frequency subbands, thereby reducing encoding / decoding complexity and increasing encoding / decoding speed. Furthermore, this application uses a wavelet transform architecture for image encoding / decoding, which can further improve encoding / decoding efficiency based on the lightweight compression characteristics of the wavelet transform architecture. Furthermore, this application combines the fast encoding and decoding speed of VLC encoding with the high compression ratio of arithmetic encoding on the encoding side, which can improve the encoding and decoding speed while ensuring the compression ratio of the encoded data.

[0008] In one possible implementation, the image bitstream also includes low-frequency subband coded data. The decoding method further includes obtaining the quantization coefficients of the low-frequency subband based on the low-frequency subband coded data, and obtaining the reconstruction coefficients of the low-frequency subband based on the quantization coefficients of the low-frequency subband. Then, performing an inverse wavelet transform on the reconstruction coefficients of the high-frequency subband to obtain the reconstructed image of the original image includes: performing an inverse wavelet transform on the reconstruction coefficients of the high-frequency subband and the reconstruction coefficients of the low-frequency subband to obtain the reconstructed image of the original image.

[0009] For example, the reconstruction coefficients of the low-frequency subband can be the inverse quantization coefficients of the low-frequency subband in this application, and the reconstruction coefficients of the high-frequency subband can be the inverse quantization coefficients of the high-frequency subband in this application.

[0010] For example, the reconstruction coefficients of the low-frequency subband can also be the reconstructed low-frequency subband in this application, and the reconstruction coefficients of the high-frequency subband can also be the reconstructed high-frequency subband in this application.

[0011] For example, the frequency of the high-frequency subband is greater than the frequency of the low-frequency subband. This can be understood as the frequency of various coefficients (such as wavelet coefficients, reconstruction coefficients, inverse quantization coefficients, etc.) corresponding to the high-frequency subband in this application being greater than the frequency of the coefficients of the low-frequency subband.

[0012] In one possible implementation, the arithmetic-coded data and VLC-coded data in the image bitstream can be decoded independently. This allows the decoder to separately decode the independently decodeable arithmetic-coded data and VLC-coded data at the decoding end, thereby improving decoding efficiency.

[0013] In one possible implementation, the VLC-encoded data includes first VLC-encoded data generated based on the quantization coefficients of a first high-frequency sub-band, second VLC-encoded data generated based on the quantization coefficients of a second high-frequency sub-band, and third VLC-encoded data generated based on the quantization coefficients of a third high-frequency sub-band; wherein the first, second, and third VLC-encoded data in the image bitstream can be decoded independently. Thus, this application decomposes the image into low-frequency sub-bands and multiple high-frequency sub-bands using wavelet transform, enabling encoding and decoding of each low-frequency and high-frequency sub-band, thereby reducing encoding and decoding complexity and improving encoding and decoding efficiency. Furthermore, for the high-frequency sub-bands, the encoding end performs VLC encoding on the quantization coefficients of each high-frequency sub-band, using a different encoding method than the control information. This allows the decoding end to independently decode the quantization coefficients and control information of the high-frequency sub-bands using different decoding methods, effectively improving the decoding efficiency of the high-frequency sub-bands and thus improving the decoded image of the original image.

[0014] In one possible implementation, the image bitstream also includes offset information, which indicates the position of independently decodeable encoded data within the image bitstream. This allows the decoding side to perform parallel decoding of the independently decodeable encoded data based on the offset information, thereby improving decoding efficiency.

[0015] In one possible implementation, control information is used to indicate whether the quantization coefficients of high-frequency sub-band blocks are all zeros. The quantization coefficients of high-frequency sub-band blocks with non-all zeros are encoded into the image bitstream using VLC entropy coding. Thus, on the encoding side, the quantization coefficients of high-frequency sub-band blocks with non-all zeros can be encoded into the image bitstream, while the quantization coefficients of high-frequency sub-band blocks with all zeros do not require encoding or decoding. This effectively improves encoding / decoding efficiency, reduces the data volume of the image bitstream, and increases the throughput between the encoder and decoder. Furthermore, the decoding end can obtain the complete quantization coefficients based on the control information, including the quantization coefficients of high-frequency sub-band blocks with all zeros, thereby improving the accuracy of the reconstructed coefficients.

[0016] In one possible implementation, obtaining the quantization coefficients of a high-frequency subband based on control information and VLC encoded data includes: when the control information indicates that the quantization coefficients of the high-frequency subband block are not all zeros, performing entropy decoding on the VLC encoded data using VLC entropy decoding based on the control information to obtain the quantization coefficients of the high-frequency subband block; and when the control information indicates that the quantization coefficients of the high-frequency subband block are all zeros, determining the quantization coefficients of the high-frequency subband block based on the control information. Thus, the decoder of this application can obtain the complete quantization coefficients of the high-frequency subband during the entropy decoding process, i.e., the quantization coefficients of the high-frequency subband block with all zero quantization coefficients.

[0017] In one possible implementation, obtaining the quantization coefficients of the high-frequency subband based on control information and VLC encoded data includes: performing entropy decoding on the VLC encoded data using VLC entropy decoding to obtain the VLC entropy decoding result; and obtaining the quantization coefficients of the high-frequency subband based on the control information and the VLC entropy decoding result. Thus, the decoder of this application can, after entropy decoding, process the quantization coefficients after entropy decoding based on the control information to obtain the complete quantization coefficients of the high-frequency subband, i.e., the quantization coefficients of the high-frequency subband block including all 0 quantization coefficients.

[0018] In one possible implementation, control information is obtained based on arithmetic-coded data, including: entropy decoding of the arithmetic-coded data using an arithmetic decoding method to obtain the control information. In this way, the decoding end can independently decode the quantization coefficients and control information of the high-frequency sub-band using different decoding methods, thereby effectively improving the decoding efficiency of the high-frequency sub-band and thus improving the decoded image of the original image.

[0019] Secondly, this application provides an encoding method, comprising: performing wavelet transform on the original image to obtain wavelet coefficients of a high-frequency subband; quantizing the wavelet coefficients of the high-frequency subband to obtain quantization coefficients of the high-frequency subband; obtaining control information based on at least one of the quantization coefficients and wavelet coefficients of the high-frequency subband; performing variable-length coding (VLC) on the quantization coefficients of the high-frequency subband to obtain VLC-coded data; and performing arithmetic coding on the control information to obtain arithmetic-coded data; wherein the VLC-coded data and the arithmetic-coded data are used to generate an image bitstream of the original image. Thus, by using VLC coding for some syntax elements of the high-frequency subband and arithmetic coding for other syntax elements, this application allows the decoding end to decode the VLC-coded data and arithmetic-coded data respectively based on VLC decoding and arithmetic decoding to obtain the quantization coefficients and control information of the high-frequency subband, thereby improving the encoding and decoding efficiency. Furthermore, this application uses a wavelet transform architecture for image encoding and decoding, which can further improve encoding and decoding efficiency based on the lightweight compression characteristics of the wavelet transform architecture.

[0020] In one possible implementation, the method performs wavelet transform on the original image to obtain wavelet coefficients for a low-frequency sub-band. The method further includes quantizing the wavelet coefficients of the low-frequency sub-band to obtain quantized coefficients for the low-frequency sub-band, and entropy encoding the quantized coefficients of the low-frequency sub-band to obtain low-frequency sub-band encoded data, which is also used to generate the image bitstream of the original data.

[0021] In one possible implementation, the arithmetic-coded data and VLC-coded data in the image bitstream can be decoded independently; wherein, the arithmetic-coded data is obtained by entropy coding of the control information using arithmetic coding, and the VLC-coded data is obtained by entropy coding of the quantization coefficients of the high-frequency subband using VLC entropy coding.

[0022] In one possible implementation, the wavelet coefficients of the high-frequency subband include wavelet coefficients of a first high-frequency subband, wavelet coefficients of a second high-frequency subband, and wavelet coefficients of a third high-frequency subband; the VLC encoded data includes first VLC encoded data generated based on the quantization coefficients of the first high-frequency subband, second VLC encoded data generated based on the quantization coefficients of the second high-frequency subband, and third VLC encoded data generated based on the quantization coefficients of the third high-frequency subband; wherein the first VLC encoded data, second VLC encoded data, and third VLC encoded data in the image bitstream can be decoded independently.

[0023] In one possible implementation, the image bitstream also includes offset information, which is used to indicate the position of independently decodeable encoded data in the image bitstream.

[0024] In one possible implementation, control information is used to indicate whether the quantization coefficients of the high-frequency subband blocks in the high-frequency subband are all zeros; wherein, the quantization coefficients of the high-frequency subband blocks with non-all zero quantization coefficients are encoded into the image bitstream using VLC entropy coding.

[0025] Thirdly, this application provides a decoder, characterized in that it includes: an interface unit for acquiring an image bitstream of the original image, the image bitstream including arithmetic encoded data and variable-length encoded (VLC) data, wherein the arithmetic encoded data is obtained using arithmetic encoding and the VLC encoded data is obtained using VLC entropy encoding; a high-frequency subband entropy decoding unit for acquiring control information based on the arithmetic encoded data; the high-frequency subband entropy decoding unit is also used for acquiring quantization coefficients of the high-frequency subband based on the control information and the VLC encoded data; a high-frequency inverse quantization unit for acquiring reconstruction coefficients of the high-frequency subband based on the quantization coefficients of the high-frequency subband; and a wavelet transform unit for acquiring a reconstructed image of the original image from the reconstruction coefficients of the high-frequency subband.

[0026] In one possible implementation, the image bitstream further includes low-frequency subband coded data, and the decoder further includes a low-frequency subband entropy decoding unit for obtaining the quantization coefficients of the low-frequency subband based on the low-frequency subband coded data. The decoder also includes a low-frequency inverse quantization unit for obtaining the reconstruction coefficients of the low-frequency subband based on the quantization coefficients of the low-frequency subband. Specifically, the wavelet transform unit performs an inverse wavelet transform on the reconstruction coefficients of the high-frequency subband and the low-frequency subband to obtain the reconstructed image of the original image.

[0027] For example, the wavelet transform unit can also be called the inverse wavelet transform unit at the decoder end, which is used to perform inverse wavelet transform processing, and it can also be called the forward wavelet transform unit at the encoder end, which is used to perform forward wavelet transform processing.

[0028] In one possible implementation, arithmetic encoded data and VLC encoded data in the image bitstream can be decoded independently.

[0029] In one possible implementation, the VLC encoded data includes first VLC encoded data generated based on the quantization coefficients of a first high-frequency subband, second VLC encoded data generated based on the quantization coefficients of a second high-frequency subband, and third VLC encoded data generated based on the quantization coefficients of a third high-frequency subband; wherein the first VLC encoded data, second VLC encoded data, and third VLC encoded data in the image bitstream can be decoded independently.

[0030] In one possible implementation, the image bitstream also includes offset information, which indicates the position of independently decodeable encoded data within the image bitstream.

[0031] In one possible implementation, control information is used to indicate whether the quantization coefficients of the high-frequency subband blocks are all zeros, wherein the quantization coefficients of the high-frequency subband blocks with non-all zeros are encoded into the image bitstream using VLC entropy coding.

[0032] In one possible implementation, the high-frequency subband entropy decoding unit is specifically configured to: when the control information indicates that the quantization coefficients of the high-frequency subband block are not all zeros, perform entropy decoding on the VLC encoded data using the VLC entropy decoding method based on the control information to obtain the quantization coefficients of the high-frequency subband block; and when the control information indicates that the quantization coefficients of the high-frequency subband block are all zeros, determine the quantization coefficients of the high-frequency subband block based on the control information.

[0033] In one possible implementation, the high-frequency subband entropy decoding unit is specifically used to: perform entropy decoding on VLC encoded data using VLC entropy decoding method to obtain VLC entropy decoding results; and obtain the quantization coefficients of the high-frequency subband based on control information and VLC entropy decoding results.

[0034] In one possible implementation, the high-frequency subband entropy decoding unit is specifically used to: perform entropy decoding on arithmetic-coded data using an arithmetic decoding method to obtain control information.

[0035] Fourthly, this application provides an encoder, characterized in that it comprises: a wavelet transform unit for performing wavelet transform on the original image to obtain wavelet coefficients of a high-frequency subband; a high-frequency subband quantization unit for quantizing the wavelet coefficients of the high-frequency subband to obtain quantization coefficients of the high-frequency subband; a control unit for acquiring control information based on at least one of the quantization coefficients of the high-frequency subband and the wavelet coefficients of the high-frequency subband; and a high-frequency subband entropy coding unit for performing variable-length coding (VLC) coding on the quantization coefficients of the high-frequency subband to obtain VLC-coded data, and performing arithmetic coding on the control information to obtain arithmetic-coded data, wherein the VLC-coded data and the arithmetic-coded data are used to generate an image bitstream of the original image.

[0036] In one possible implementation, the wavelet transform unit performs a wavelet transform on the original image to obtain wavelet coefficients for the low-frequency subband. The encoder also includes a low-frequency subband quantization unit for quantizing the wavelet coefficients of the low-frequency subband to obtain quantized coefficients for the low-frequency subband. The encoder further includes a low-frequency subband entropy coding unit for entropy coding the quantized coefficients of the low-frequency subband to obtain low-frequency subband coded data, which is also used to generate the image bitstream of the original image.

[0037] In one possible implementation, the arithmetic-coded data and VLC-coded data in the image bitstream can be decoded independently; wherein, the arithmetic-coded data is obtained by entropy coding of the control information using arithmetic coding, and the VLC-coded data is obtained by entropy coding of the quantization coefficients of the high-frequency subband using VLC entropy coding.

[0038] In one possible implementation, the wavelet coefficients of the high-frequency subband include wavelet coefficients of a first high-frequency subband, wavelet coefficients of a second high-frequency subband, and wavelet coefficients of a third high-frequency subband; the VLC encoded data includes first VLC encoded data generated based on the quantization coefficients of the first high-frequency subband, second VLC encoded data generated based on the quantization coefficients of the second high-frequency subband, and third VLC encoded data generated based on the quantization coefficients of the third high-frequency subband; wherein the first VLC encoded data, second VLC encoded data, and third VLC encoded data in the image bitstream can be decoded independently.

[0039] In one possible implementation, the image bitstream also includes offset information, which is used to indicate the position of independently decodeable encoded data in the image bitstream.

[0040] In one possible implementation, control information is used to indicate whether the quantization coefficients of the high-frequency subband blocks in the high-frequency subband are all zeros; wherein, the quantization coefficients of the high-frequency subband blocks with non-all zero quantization coefficients are encoded into the image bitstream using VLC entropy coding.

[0041] Fifthly, this application provides an encoder, including: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, cause the encoder to perform the method in the first aspect or any possible implementation thereof.

[0042] In a sixth aspect, this application provides a decoder, comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, cause the decoder to perform the method in the second aspect or any possible implementation thereof.

[0043] In a seventh aspect, this application provides a chip including one or more interface circuits and one or more processors; the one or more processors receive or transmit data through the one or more interface circuits, and when the one or more processors execute computer instructions, cause the electronic device to perform the method in the first aspect or any possible implementation of the first aspect.

[0044] Eighthly, this application provides a chip including one or more interface circuits and one or more processors; the one or more processors receive or transmit data through the one or more interface circuits, and when the one or more processors execute computer instructions, cause the electronic device to perform the method in the second aspect or any possible implementation of the second aspect.

[0045] Ninthly, this application provides a computer-readable storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to perform the method of the first aspect or any possible implementation thereof.

[0046] In a tenth aspect, this application provides a computer-readable storage medium storing a computer program that, when run on a computer or processor, causes the computer or processor to perform the method of the second aspect or any possible implementation thereof.

[0047] In one aspect, this application provides a computer program product, which includes computer instructions that, when executed by a computer or processor, cause the computer or processor to perform the method in the first aspect or any possible implementation thereof.

[0048] In a twelfth aspect, this application provides a computer program product including computer instructions that, when executed by a computer or processor, cause the computer or processor to perform the method in the second aspect or any possible implementation thereof.

[0049] In a thirteenth aspect, this application provides a computer-readable storage medium that stores a bitstream from the third aspect or any possible implementation thereof.

[0050] In a fourteenth aspect, this application provides an encoder including processing circuitry that can be used to perform the methods in the second aspect or any possible implementation thereof.

[0051] In a fifteenth aspect, this application provides a decoder including processing circuitry that can be used to perform the methods in the first aspect or any possible implementation thereof.

[0052] In a sixteenth aspect, this application provides a bitstream generated according to the second aspect and any implementation thereof.

[0053] In this embodiment, the electronic device, computer-readable storage medium, computer program product, chip, or codec are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above. Attached Figure Description

[0054] Figure 1A This is a schematic block diagram illustrating an exemplary video encoding and decoding system;

[0055] Figure 1B This is a schematic block diagram illustrating an exemplary video decoding system;

[0056] Figure 2 A schematic block diagram of an encoder as an example is shown;

[0057] Figure 3 A schematic block diagram of a decoder as an example;

[0058] Figure 4 This is a schematic diagram of the structure of a video decoding device as an example.

[0059] Figure 5 This is a schematic diagram of the structure of an exemplary device;

[0060] Figure 6 A schematic block diagram of a wavelet transform-based encoder is shown as an example.

[0061] Figure 7 A schematic block diagram of a wavelet transform-based decoder is shown as an example.

[0062] Figures 8A-8B A schematic block diagram of an encoder as an example is shown;

[0063] Figure 9A This is an example of a subgraph partitioning diagram;

[0064] Figure 9B This is an example of a subgraph partitioning diagram;

[0065] Figure 10 This is an example of a wavelet transform schematic diagram;

[0066] Figures 11A-11B A schematic block diagram of a decoder as an example;

[0067] Figure 12 A schematic block diagram of an encoder as an example is shown;

[0068] Figure 13 This is a schematic diagram illustrating the structure of an image bitstream as an example.

[0069] Figure 14 This is a schematic diagram illustrating the structure of an image bitstream as an example.

[0070] Figure 15 This is a schematic diagram illustrating the structure of an image bitstream as an example.

[0071] Figures 16A-16C This is a schematic diagram illustrating the structure of an image bitstream as an example.

[0072] Figure 17 A schematic block diagram of a decoder as an example;

[0073] Figure 18 A schematic block diagram of a decoder as an example;

[0074] Figure 19 This is a schematic diagram of the structure of an exemplary device. Detailed Implementation

[0075] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0076] The embodiments of this application are described below with reference to the accompanying drawings. In the following description, reference is made to the accompanying drawings, which form part of this application and illustrate specific aspects of the embodiments of this application or to which specific aspects of the embodiments of this application may be used. It should be understood that the embodiments of this application may be used in other aspects and may include structural or logical variations not depicted in the drawings. Therefore, the following detailed description should not be construed in a limiting sense, and the scope of this application is defined by the appended claims. For example, it should be understood that the disclosure of the described methods is equally applicable to corresponding devices or systems for performing the methods, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units, such as functional units, to perform the described one or more method steps (e.g., one unit performs one or more steps, or multiple units, each performing one or more of multiple steps), even if such one or more units are not explicitly described or illustrated in the drawings. On the other hand, for example, if a specific apparatus is described based on one or more units such as functional units, the corresponding method may include a step to perform the functionality of one or more units (e.g., a step to perform the functionality of one or more units, or multiple steps, each of which performs the functionality of one or more units among a plurality of units), even if such one or more steps are not explicitly described or illustrated in the accompanying drawings. Furthermore, it should be understood that, unless otherwise expressly stated, features of the various exemplary embodiments and / or aspects described herein can be combined with each other.

[0077] In the embodiments of this application, the modules / components shown in the framework diagram (or structural diagram or system diagram) are merely examples of this application. The actual framework (or structure or system) may include more or fewer modules / components than those shown in the diagram, or may have different component configurations. Furthermore, the various components / modules shown in the diagrams may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0078] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.

[0079] The following is a brief introduction to some concepts that may be involved in the embodiments of this application. These concepts are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0080] The residual is the difference between the reconstructed value and the predicted value of a sample or data element.

[0081] A residual block is an M×N residual matrix composed of the residuals corresponding to the coded blocks.

[0082] Dequantization is the process of scaling the quantized residual to obtain the reconstructed residual value.

[0083] A partition divides a set into subsets. Each element in the set belongs to one and only one subset.

[0084] Partition type: The way the subsets obtained from the partition are organized.

[0085] A decoded picture is an image reconstructed by the decoder based on the bitstream.

[0086] Prediction is the specific implementation of the prediction process.

[0087] The prediction process uses previously decoded samples to obtain the predicted value for the current sample.

[0088] Syntax element: The result of parsing data units in a bitstream.

[0089] A bitstream is a binary data stream that encodes all or part of an image sample.

[0090] Video coding generally refers to the processing of a sequence of images that form a video or video sequence. In the field of video coding, the terms "picture," "frame," or "image" can be used synonymously. Video coding is performed on the source side and typically involves processing (e.g., by compression) the raw video images to reduce the amount of data required to represent them, thus enabling more efficient storage and / or transmission. Video decoding is performed on the destination side and typically involves inverse processing relative to the encoder to reconstruct the video images. The combination of encoding and decoding is also known as encoding and decoding.

[0091] A video sequence consists of a series of pictures, which are further divided into slices, and slices into blocks. Video coding is performed on a block-by-block basis. In some newer video coding standards, the concept of a block has been further expanded. For example, the H.264 standard uses macroblocks (MBs), which can be further divided into multiple prediction blocks (partitions) for predictive coding. The High Efficiency Video Coding (HEVC) standard uses basic concepts such as coding units (CUs), prediction units (PUs), and transform units (TUs) to functionally divide various block units, and employs a novel tree-based structure for description. For instance, a CU can be divided into smaller CUs using a quadtree, and these smaller CUs can be further divided, forming a quadtree structure. The CU is the basic unit for partitioning and encoding the image. Similar tree structures exist for PUs and TUs. A PU corresponds to a prediction block and is the basic unit for predictive coding. CUs are further divided into multiple PUs according to partitioning patterns. TU can correspond to a transform block, which is the basic unit for transforming the prediction residual. However, whether it is CU, PU or TU, they all essentially belong to the concept of a block (or image block).

[0092] For example, in HEVC, the CTU is split into multiple CUs using a quadtree structure represented as a coding tree. At the CU level, a decision is made on whether to use inter-picture (temporal) or intra-picture (spatial) prediction to encode picture regions. Each CU can be further split into one, two, or four PUs based on the PU splitting type. The same prediction process is applied within a PU, and relevant information is transmitted to the decoder based on the PU. After obtaining residual blocks by applying the prediction process based on the PU splitting type, the CU can be segmented into transform units (TUs) according to other quadtree structures similar to the coding tree used for CUs. In the latest developments in video compression technology, quadtree and binary tree (QTBT) frame segmentation is used to divide coding blocks. In the QTBT block structure, CUs can be square or rectangular in shape.

[0093] In this paper, for ease of description and understanding, the image block to be processed in the current image is referred to as the current block. For example, in encoding, it refers to the block currently being encoded; in decoding, it refers to the block currently being decoded. The decoded image block in the reference image used to predict the current block is called the reference block. That is, the reference block is the block that provides a reference signal for the current block, where the reference signal represents the pixel value within the image block. The block in the reference image that provides a prediction signal for the current block is called the prediction block, where the prediction signal represents the pixel value, sampled value, or sampled signal within the prediction block. For example, after traversing multiple reference blocks, an optimal reference block is found. This optimal reference block will provide prediction for the current block; this block is called the prediction block.

[0094] In lossless video coding, the original video image can be reconstructed, meaning the reconstructed video image has the same quality as the original (assuming no transmission loss or other data loss during storage or transmission). In lossy video coding, further compression is performed, for example, through quantization, to reduce the amount of data required to represent the video image. However, the decoder cannot fully reconstruct the video image, meaning the quality of the reconstructed video image is lower or worse than the original video image.

[0095] The system architecture used in the embodiments of this application is described below. See also Figure 1A , Figure 1A A schematic block diagram of the video encoding and decoding system 10 used in the embodiments of this application is provided as an example. Figure 1A As shown, the video encoding and decoding system 10 may include a source device 12 and a destination device 14. The source device 12 generates encoded video data; therefore, the source device 12 may be referred to as a video encoding device. The destination device 14 can decode the encoded video data generated by the source device 12; therefore, the destination device 14 may be referred to as a video decoding device. Various embodiments of the source device 12, the destination device 14, or both may include one or more processors and memory coupled to said one or more processors. The memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other media that can be used to store desired program code in the form of computer-accessible instructions or data structures. The source device 12 and the destination device 14 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handsets, televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, wireless communication devices, or the like.

[0096] Source device 12 and destination device 14 can communicate via link 13, through which destination device 14 can receive encoded video data from source device 12. Link 13 may include one or more media or devices capable of transmitting encoded video data from source device 12 to destination device 14. In one example, link 13 may include one or more communication media enabling source device 12 to transmit encoded video data to destination device 14 in real time. In this example, source device 12 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and transmit the modulated video data to destination device 14. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, wide area network, or global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices facilitating communication from source device 12 to destination device 14.

[0097] The source device 12 includes an encoder 20. Optionally, the source device 12 may also include an image source 16, an image preprocessor 18, and a communication interface 22. In specific implementations, the encoder 20, image source 16, image preprocessor 18, and communication interface 22 may be hardware components or software programs within the source device 12. These are described below:

[0098] Image source 16 may include or be any type of image capture device for, for example, capturing real-world images, and / or any type of image or commentary (for screen content encoding, some text on the screen is also considered as an image to be encoded or part of an image) generation device, such as a computer graphics processor for generating computer-animated images, or any type of device for acquiring and / or providing real-world images, computer-animated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). Image source 16 may be a camera for capturing images or a memory for storing images. Image source 16 may also include any type of (internal or external) interface for storing previously captured or generated images and / or acquiring or receiving images. When image source 16 is a camera, image source 16 may be, for example, a local or integrated camera integrated into a source device; when image source 16 is a memory, image source 16 may be a local or integrated memory integrated into a source device. When the image source 16 includes an interface, the interface may be, for example, an external interface for receiving images from an external video source, such as an external image capture device, like a camera, external storage, or an external image generation device, such as an external computer graphics processor, computer, or server. The interface can be any type of interface according to any proprietary or standardized interface protocol, such as a wired or wireless interface, or an optical interface.

[0099] An image can be viewed as a two-dimensional array or matrix of pixels. Pixels in the array are also called sampling points. The number of sampling points in the array or image along the horizontal and vertical directions (or axes) defines the image's size and / or resolution. To represent color, three color components are typically used; that is, an image can be represented as or contain three sampling arrays. For example, in RBG format or color space, an image includes corresponding red, green, and blue sampling arrays. However, in video coding, each pixel is typically represented in a luma / chroma format or color space. For example, for a YUV format image, this includes a luma component indicated by Y (sometimes also indicated by L) and two chroma components indicated by U and V. The luma component Y represents the brightness or grayscale level intensity (e.g., both are the same in a grayscale image), while the two chroma components U and V represent chroma or color information components. Accordingly, a YUV format image includes a luma sampling array of luma sample values ​​(Y) and two chroma sampling arrays of chroma values ​​(U and V). An RGB format image can be converted or transformed to YUV format, and vice versa; this process is also called color transformation or conversion. If the image is black and white, it may only include a luminance sampling array. In this embodiment, the image transmitted from image source 16 to image processor can also be referred to as raw image data 17.

[0100] Image preprocessor 18 is configured to receive raw image data 17 and perform preprocessing on the raw image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. For example, the preprocessing performed by image preprocessor 18 may include retouching, color format conversion (e.g., from RGB format to YUV format), color correction, or noise reduction.

[0101] Encoder 20 (or video encoder 20) is used to receive preprocessed image data 19 and process the preprocessed image data 19 using a relevant prediction mode (such as the prediction mode in the various embodiments herein) to provide encoded image data 21. In some embodiments, encoder 20 may be used to perform the various embodiments described below to implement the encoding method described herein on the encoding side.

[0102] Communication interface 22 can be used to receive encoded image data 21 and transmit the encoded image data 21 via link 13 to destination device 14 or any other device (such as a memory) for storage or direct reconstruction. The other device can be any device used for decoding or storage. Communication interface 22 can, for example, be used to encapsulate the encoded image data 21 into a suitable format, such as data packets, for transmission over link 13.

[0103] Destination device 14 includes decoder 30. Optionally, destination device 14 may also include communication interface 28, image post-processor 32, and display device 34. These are described below:

[0104] Communication interface 28 can be used to receive encoded image data 21 from source device 12 or any other source, such as a storage device, for example, an encoded image data storage device. Communication interface 28 can be used to transmit or receive encoded image data 21 via link 13 between source device 12 and destination device 14 or via any type of network, such as a wired or wireless connection, any type of network, such as a wired or wireless network or any combination thereof, or any type of private and public network, or any combination thereof. Communication interface 28 can be used, for example, to decapsulate data packets transmitted by communication interface 22 to obtain encoded image data 21.

[0105] Both communication interface 28 and communication interface 22 can be configured as unidirectional or bidirectional communication interfaces, and can be used, for example, to send and receive messages to establish connections, acknowledge and exchange any other information related to the communication link and / or data transmission, such as encoded image data transmission.

[0106] Decoder 30 (or video decoder 30) is used to receive encoded image data 21 and provide decoded image data 31 or decoded image 31 (hereinafter further based on...) Figure 3 or Figure 4 or Figure 5 (Description of structural details of decoder 30). In some embodiments, decoder 30 can be used to perform the various embodiments described below to implement the decoding method described in this application on the decoding side.

[0107] Image post-processor 32 is used to perform post-processing on decoded image data 31 (also referred to as reconstructed image data) to obtain post-processed image data 33. The post-processing performed by image post-processor 32 may include: color format conversion (e.g., from YUV format to RGB format), color correction, retouching or resampling, or any other processing, and may also be used to transmit the post-processed image data 33 to display device 34.

[0108] Display device 34 is used to receive post-processed image data 33 to display an image to, for example, a user or viewer. Display device 34 can be or may include any class of displays for presenting reconstructed images, such as integrated or external displays or monitors. For example, displays may include liquid crystal displays (LCDs), organic light emitting diode (OLED) displays, plasma displays, projectors, micro-LED displays, liquid crystal on silicon (LCoS), digital light processors (DLP), or any other class of displays.

[0109] Although, Figure 1A Source device 12 and destination device 14 are illustrated as separate devices, but device embodiments may also include the functionality of both source device 12 and destination device 14, or both; that is, the functionality of source device 12 or its corresponding features and the functionality of destination device 14 or its corresponding features. In such embodiments, the same hardware and / or software, or separate hardware and / or software, or any combination thereof, may be used to implement the functionality of source device 12 or its corresponding features and the functionality of destination device 14 or its corresponding features.

[0110] Both encoder 20 and decoder 30 can be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the technology is implemented in part in software, the device can store the software instructions in a suitable non-transitory computer-readable storage medium, and one or more processors can be used to execute the instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) can be considered as one or more processors.

[0111] In some cases, Figure 1A The video encoding and decoding system 10 shown is merely an example, and the technology of this application can be applied to video encoding setups (e.g., video encoding or video decoding) that do not necessarily involve any data communication between the encoding and decoding devices. In other instances, data may be retrieved from local storage, streamed over a network, etc. A video encoding device may encode data and store it in storage, and / or a video decoding device may retrieve data from storage and decode it. In some instances, encoding and decoding are performed by devices that do not communicate with each other but only encode data to storage and / or retrieve data from storage and decode the data.

[0112] See Figure 1B , Figure 1B It is a inclusion according to an exemplary embodiment Figure 2 encoder 20 and / or Figure 3 The illustrated diagram shows an example of a video decoding system 40 with decoder 30. The video decoding system 40 can implement various combinations of technologies according to embodiments of this application. In the illustrated implementation, the video decoding system 40 may include an imaging device 41, an encoder 20, a decoder 30 (and / or a video encoder / decoder implemented by logic circuitry of a processing unit 46), an antenna 42, one or more processors 43, one or more memories 44, and a display device 45.

[0113] like Figure 1B As shown, the imaging device 41, antenna 42, processing unit 46, logic circuit, encoder 20, decoder 30, processor 43, memory 44, and display device 45 are capable of communicating with each other. As discussed, although encoder 20 and decoder 30 are used as examples to describe the video decoding system 40, in different instances, the video decoding system 40 may contain only encoder 20 or only decoder 30.

[0114] In some instances, antenna 42 can be used to transmit or receive encoded video data streams. Additionally, in some instances, display device 45 can be used to present video data. In some instances, logic circuitry can be implemented using processing unit 46. Processing unit 46 can include an ASIC, graphics processor, general-purpose processor, etc. Video decoding system 40 can also include an optional processor 43, which can similarly include an ASIC, graphics processor, general-purpose processor, etc. In some instances, logic circuitry can be implemented in hardware, such as dedicated video encoding hardware, while processor 43 can be implemented in general-purpose software, operating system, etc. Furthermore, memory 44 can be any type of memory, such as volatile memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (e.g., flash memory, etc.). In a non-limiting instance, memory 44 can be implemented using cache memory. In some instances, logic circuitry can access memory 44 (e.g., for implementing an image buffer). In other instances, the logic circuitry and / or processing unit 46 may include memory (e.g., cache, etc.) for implementing image buffers, etc.

[0115] In some instances, the encoder 20 implemented via logic circuitry may include (e.g., implemented via processing unit 46 or memory 44) an image buffer and (e.g., implemented via processing unit 46) a graphics processing unit. The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the encoder 20 implemented via logic circuitry to implement a reference... Figure 2 And / or any other encoder system or subsystem described herein, and the various modules discussed herein. Logic circuits may be used to perform the various operations discussed herein.

[0116] In some instances, decoder 30 can be implemented via logic circuitry in a similar manner to implement the reference. Figure 3 The decoder 30 and / or any other decoder system or subsystem described herein are various modules discussed. In some instances, the logic circuit-implemented decoder 30 may include an image buffer (implemented via processing unit 46 or memory 44) and a graphics processing unit (e.g., implemented via processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the logic circuit-implemented decoder 30 to implement reference Figure 3 And / or the various modules discussed in any other decoder system or subsystem described herein.

[0117] In some instances, antenna 42 can be used to receive an encoded stream of video data. As discussed herein, the encoded stream may contain data related to encoded video frames, indicators, index values, mode selection data, etc., such as data related to code segmentation (e.g., transform coefficients or quantized transform coefficients, optional indicators, and / or data defining code segmentation). Video decoding system 40 may also include a decoder 30 coupled to antenna 42 for decoding the encoded stream. Display device 45 is used to display the video frames.

[0118] It should be understood that, referring to the examples described for encoder 20 in the embodiments of this application, decoder 30 can be used to perform the reverse process. Regarding signaling syntax elements, decoder 30 can be used to receive and parse such syntax elements, and accordingly decode the associated video data. In some examples, encoder 20 can entropy-encode syntax elements into an encoded video stream. In such instances, decoder 30 can parse such syntax elements and accordingly decode the associated video data.

[0119] It should be noted that the encoding and decoding method described in the embodiments of this application is mainly used for the encoding and decoding process of video or images. This process exists in both the encoder 20 and the decoder 30. The encoder 20 and decoder 30 in the embodiments of this application can be, for example, the encoder / decoder corresponding to video standard protocols such as H.263, H.264, HEVV, MPEG-2, MPEG-4, VP8, VP9, ​​or next-generation video standard protocols (such as H.266).

[0120] See Figure 2 , Figure 2 A schematic / conceptual block diagram illustrating an example of encoder 20. Figure 2 In this example, encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy coding unit 270. Prediction processing unit 260 may include inter-frame prediction unit 244, intra-frame prediction unit 254, and mode selection unit 262. Inter-frame prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). Figure 2 The encoder 20 shown can also be called a hybrid video encoder or a video encoder based on a hybrid video codec.

[0121] See Figure 3 , Figure 3This is a schematic / conceptual block diagram illustrating an example instance of decoder 30. Decoder 30 is used to receive encoded image data (e.g., encoded bitstream) 21, for example, encoded by encoder 20, to obtain decoded image 331. During the decoding process, decoder 30 receives video data from encoder 20, such as encoded video bitstreams representing image blocks of encoded video stripes and associated syntax elements.

[0122] exist Figure 3 In one example, decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (e.g., a summer 314), a buffer 316, a loop filter 320, a decoded image buffer 330, and a prediction processing unit 360. Prediction processing unit 360 may include an inter-frame prediction unit 344, an intra-frame prediction unit 354, and a mode selection unit 362. In some instances, decoder 30 may perform substantially the same functions as... Figure 2 The video encoder 20 describes the encoding process as the inverse of the decoding process.

[0123] See Figure 4 , Figure 4 This is a schematic diagram of the structure of a video decoding device 400 (e.g., a video encoding device 400 or a video decoding device 400) provided in an embodiment of this application. The video decoding device 400 is suitable for implementing the embodiments described herein. In one embodiment, the video decoding device 400 may be a video decoder (e.g., ... Figure 1A decoder 30) or video encoder (e.g. Figure 1A The encoder 20). In another embodiment, the video decoding device 400 may be the one described above. Figure 1A decoder 30 or Figure 1A One or more components in encoder 20.

[0124] The video decoding device 400 includes: an input port 410 and a receiving unit (Rx) 420 for receiving data; a processor, logic unit, or central processing unit (CPU) 430 for processing data; a transmitter unit (Tx) 440 and an output port 450 for transmitting data; and a memory 460 for storing data. The video decoding device 400 may also include photoelectric conversion components and electro-optical (EO) components coupled to the input port 410, receiver unit 420, transmitter unit 440, and output port 450 for the input or output of optical or electrical signals.

[0125] Processor 430 is implemented in both hardware and software. Processor 430 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. Processor 430 communicates with ingress port 410, receiver unit 420, transmitter unit 440, egress port 450, and memory 460. Processor 430 includes either an encoding module 470 or a decoding module 470. The encoding / decoding module 470 implements the embodiments disclosed herein to implement the encoding or decoding methods provided in the embodiments of this application. For example, the encoding / decoding module 470 implements, processes, or provides various encoding operations. Therefore, the encoding / decoding module 470 provides a substantial improvement to the functionality of the video decoding device 400 and affects the transitions of the video decoding device 400 to different states. Alternatively, the encoding / decoding module 470 can be implemented with instructions stored in memory 460 and executed by processor 430.

[0126] Memory 460 includes one or more disks, tape drives, and solid-state drives, which can be used as overflow data storage devices to store programs while they are selectively executed, and to store instructions and data read during program execution. Memory 460 can be volatile and / or non-volatile, and can be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random access memory (SRAM).

[0127] See Figure 5 , Figure 5 It is available as an exemplary embodiment. Figure 1A A simplified block diagram of device 500 comprising one or both of the source device 12 and the destination device 14. Device 500 can implement the technology of this application. In other words, Figure 5 This is a schematic block diagram illustrating one implementation of an encoding or decoding device (referred to as decoding device 500) according to an embodiment of this application. The decoding device 500 may include a processor 510, a memory 530, and a bus system 550. The processor and memory are connected via the bus system. The memory stores instructions, and the processor executes the instructions stored in the memory. The memory of the decoding device stores program code, and the processor can call the program code stored in the memory to execute various video encoding or decoding methods described in this application. To avoid repetition, further details are omitted here.

[0128] In this embodiment, the processor 510 may be a central processing unit (CPU), or it may be other general-purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0129] The memory 530 may include a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may also be used as memory 530. Memory 530 may include code and data 531 accessed by processor 510 using bus 550. Memory 530 may further include an operating system 533 and an application program 535, which includes at least one program that allows processor 510 to execute the encoding or decoding methods described in this application. For example, application program 535 may include applications 1 to N, which further include video encoding or decoding applications that execute the encoding or decoding methods described in this application.

[0130] In addition to the data bus, the bus system 550 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 550 in the diagram.

[0131] Optionally, the decoding device 500 may also include one or more output devices, such as a display 570. In one example, the display 570 may be a haptic display that combines a display with a haptic unit capable of operatively sensing touch input. The display 570 may be connected to the processor 510 via a bus 550.

[0132] For example, commonly used transform methods in image coding include discrete cosine transform and wavelet transform. Wavelet transform is a local transform method that can perform localized, multi-scale analysis of images, focusing on the details of signal changes, making it very suitable for image coding tasks.

[0133] See Figure 6 , Figure 6 This is a schematic / conceptual block diagram illustrating an encoder based on wavelet transform. Figure 6In the example, encoder 60 includes, but is not limited to, a wavelet forward transform unit 610, a quantization unit 620, and an entropy coding unit 630. Specifically, the encoder acquires the original image 601 through an interface unit (also called an input interface, not shown in the figure). The original image 601 is input to the wavelet forward transform unit 610, and after wavelet transform (also called wavelet forward transform), wavelet transform coefficients 602 are obtained, which can also be simply referred to as wavelet coefficients in this embodiment. The quantization unit 620 quantizes the wavelet transform coefficients 602 to obtain quantization coefficients 603. The entropy coding unit 630 entropy codes the quantization coefficients 603 to obtain the image bitstream 604 of the original image, which can also be called a compressed bitstream or bitstream.

[0134] It should be noted that, Figure 6 The wavelet transform-based encoder architecture shown is for illustrative purposes only, and other examples may include more units or modules. For example, it may include, but is not limited to, prediction units, transform units, etc., and this application does not impose any limitations.

[0135] See Figure 7 , Figure 7 This is a schematic / conceptual block diagram illustrating an exemplary wavelet transform-based decoder. Figure 7 In the example, decoder 70 includes, but is not limited to, entropy decoding unit 710, inverse quantization unit 720, and wavelet inverse transform unit 730. Specifically, at the decoding end, the decoder interfaces with image bitstream 701 through an input interface (also called an interface unit, not shown in the figure). Entropy decoding unit 710 performs entropy decoding on image bitstream 701 to obtain quantization coefficients 702. Here, image bitstream 701 can be based on... Figure 6 The encoder 60 generates a bitstream 604. The dequantization unit 720 dequantizes the quantization coefficients 702 to obtain dequantized coefficients 703 (also known as reconstructed coefficients, wavelet coefficients, etc.). The wavelet inverse transform unit 730 performs an inverse wavelet transform on the dequantized coefficients 703 to obtain the reconstructed image 704.

[0136] It should be noted that, Figure 7 The wavelet transform-based decoder architecture described herein is for illustrative purposes only, and other instances may include more units or modules. For example, it may include, but is not limited to, prediction units, inverse transform units, etc., and this application does not impose any limitations.

[0137] For example, the wavelet coefficients obtained after wavelet transform include: wavelet coefficients of the high-frequency subband and wavelet coefficients of the low-frequency subband. Figure 6 and Figure 7In the illustrated architecture, the codec performs the same processing on the wavelet coefficients of the high-frequency subband and the low-frequency subband, resulting in high processing complexity. In this embodiment, the wavelet coefficients of the high-frequency subband can be simply referred to as the high-frequency subband, and the wavelet coefficients of the low-frequency subband can be simply referred to as the low-frequency subband. It can be understood that the high-frequency subband obtained after wavelet transform can optionally be a set of wavelet coefficients from the high-frequency subband, and the low-frequency subband obtained after wavelet transform can optionally be a set of wavelet coefficients from the low-frequency subband.

[0138] For example, the frequency of the high-frequency sub-band is greater than the frequency of the low-frequency sub-band. This can be understood as the frequency of various coefficients corresponding to the high-frequency sub-band in this application (such as wavelet coefficients, reconstruction coefficients, inverse quantization coefficients, etc. involved in this application) being greater than the frequency of the coefficients of the low-frequency sub-band.

[0139] This application provides a wavelet transform-based codec that can independently encode and decode low-frequency and high-frequency sub-bands, effectively reducing encoding and decoding complexity and improving efficiency. For example, an image undergoes wavelet transform to obtain low-frequency and high-frequency sub-bands, which are then encoded to generate low-frequency and high-frequency sub-band bitstreams, respectively. The low-frequency sub-band can be understood as a sub-image representing the low-frequency signal (or low-frequency information) of the original image, and the high-frequency sub-band can be understood as a sub-image representing the high-frequency signal (or high-frequency information) of the original image.

[0140] See Figure 8A , Figure 8A A schematic / conceptual block diagram of an encoder is shown as an example. Figure 8A In the encoder 80, the following are included but not limited to: a sub-graph partitioning unit 810, a wavelet forward transform unit 820, a low-frequency sub-band processing path 830, and a high-frequency sub-band processing path 840. Optionally, the wavelet forward transform unit and the wavelet inverse transform unit in the embodiments of this application can also be collectively referred to as wavelet transform units, which can be understood as wavelet transform units being able to perform wavelet forward transform processing and wavelet inverse transform processing.

[0141] The low-frequency subband processing path 830 includes, but is not limited to: a transform / quantization unit 831 (also known as a low-frequency subband transform / quantization unit) and a low-frequency subband entropy coding unit 832.

[0142] The high-frequency subband processing path 840 includes, but is not limited to: a transformation / quantization unit 841 (also known as a high-frequency subband transformation / quantization unit), a high-frequency subband entropy coding unit 842, and a control unit 843.

[0143] Specifically, encoder 80 receives image 801. Image 801 can be an image in an image sequence that forms a video or video sequence. The image can be called the original image, the current image, or the image to be encoded. Image 801 can be the original image or an image obtained by processing the original image.

[0144] The sub-image partitioning unit 810 is used to acquire the current image and partition it to obtain at least one sub-image. Specifically, the sub-image partitioning unit 810 partitions the current image into N sub-images according to a sub-image partitioning method, where N is an integer greater than 0 (or an integer greater than 1). The sub-image partitioning method can include, but is not limited to, at least one of the following:

[0145] The width and / or height of the subgraph are multiples of 128;

[0146] The maximum width of the subimage is 1024 pixels;

[0147] The minimum height and / or width of the subimage is 256 pixels;

[0148] The original image resolution is less than or equal to 1080p, and N is an integer greater than 1 and less than or equal to 8; or,

[0149] The original image has a length greater than or equal to 4320 pixels, a width greater than or equal to 2160 pixels, and N is an integer greater than 1 and less than or equal to 16; or,

[0150] The original image has a length greater than or equal to 7680 pixels, a width greater than or equal to 4320 pixels, and N is an integer greater than 1 and less than or equal to 32.

[0151] For example, common video resolutions are shown in Table 1:

[0152] Table 1

[0153] 1280x720 1280x1080 1440x1080 1920x1080 (1080p) 2048x1080 2048x1556 3840x2160 (4k) 4096x2160 5120x2700 6144x3240 7680x4320 (8k) 8192x4320

[0154] The values ​​mentioned above are merely illustrative examples and can be set according to actual needs.

[0155] For example, the subgraph partitioning unit 810 can set specific subgraph specifications (including width and height) based on the above subgraph partitioning method. The specific values ​​can be set according to actual needs within the range specified by the subgraph partitioning method.

[0156] Reference Figure 9A , Figure 9A This is a schematic diagram illustrating subgraph partitioning as an example. Figure 9AIn the example, the subgraph partitioning unit 810 can partition the image 801 into m*n subgraphs according to the subgraph partitioning method. Optionally, in this example, the width and height of each subgraph are multiples of 128.

[0157] Reference Figure 9B , Figure 9B This is a schematic diagram illustrating subgraph partitioning as an example. Figure 9B In the example, the subgraph partitioning unit 810 can partition the image into m*n subgraphs according to the subgraph partitioning method. When the subgraph partitioning unit 810 partitions the boundary portion of the current image, the width and / or height of the boundary subgraphs may optionally not be a multiple of 128. Figure 9B This example only illustrates the situation where the width of some boundaries is not a multiple of 128. (Refer to...) Figure 9B In this embodiment of the application, for sub-graphs whose width and height are not multiples of 128, the sub-graph partitioning unit 810 can pad these sub-graphs, for example... Figure 9B The gray area in the diagram. Taking subgraph 1_n as an example, the height of subgraph 1_n is a multiple of 128, but its width is not. Subgraph partitioning unit 810 fills subgraph 1_n.

[0158] In one possible implementation, the width and / or height of the filled subgraph can optionally be less than or equal to the subgraph size set by the subgraph partitioning method. For example, the size (i.e., width and height) of the filled subgraph 1_n can be the same as that of subgraph 1_1. As another example, the size of the filled subgraph 1_n can be less than the size of subgraph 1_1, where the width of subgraph 1_1 is a, the height is b, the width of subgraph 1_n before filling is c, the height is b, c is less than b, and c is not a multiple of 128. The subgraph partitioning unit 810 fills the subgraph 1_n, and the width of the filled subgraph 1_n is d, and the height is b. Here, d can optionally be a multiple of 16, and d is greater than c and less than or equal to a.

[0159] In this way, by padding the sub-images, each macroblock can meet the 8x8 (pixel) partitioning requirement during the block division process of the encoding. Furthermore, by finding the least common multiple of the width and / or height of the sub-image before padding and 16, sub-images that do not meet the multiple of 16 are padded. This ensures that the size of the sub-image is a multiple of 16 while minimizing the size of the padded sub-image, thereby reducing the encoding complexity of the padded sub-image.

[0160] Optionally, the sub-image partitioning method can be the same for each image in the same video sequence.

[0161] Optionally, the sub-image division unit 810 can divide the image into sub-images along the horizontal and vertical directions, starting from the upper left corner, according to a preset sub-image size. The division order can be set according to actual needs. Typically, the sub-images that need to be filled are boundary sub-images, for example... Figure 9B As shown.

[0162] In this embodiment, each sub-image is encoded and decoded independently. Specifically, the image is divided into N sub-images, and the encoder 80 encodes each of the N sub-images separately. During the encoding process, a sub-image can also be referred to as the current sub-image or the sub-image to be encoded.

[0163] Still refer to Figure 8A The wavelet forward transform unit 820 is used to perform wavelet transform (also called wavelet forward transform) on the sub-graph to obtain low-frequency sub-bands and high-frequency sub-bands. The low-frequency sub-band (also called the first sub-band, first frequency component, first sub-graph, etc.) includes low-frequency signals in the sub-graph that satisfy the low-frequency filtering coefficient, and the high-frequency sub-band (also called the second sub-band, second frequency component, second sub-graph, etc.) includes high-frequency signals in the sub-graph that satisfy the high-frequency filtering coefficient.

[0164] Reference Figure 10 , Figure 10 This is a schematic diagram of wavelet transform as an example. Figure 10 In the example, the wavelet forward transform unit 820 acquires the current sub-image, for example, sub-image 1_1. The wavelet forward transform unit 820 performs a wavelet transform on the current sub-image to obtain wavelet coefficients for the low-low (LL) sub-band (abbreviated as LL sub-band), low-high (LH) sub-band (abbreviated as LH sub-band), high-high (HH) sub-band (abbreviated as HH sub-band), and high-low (HL) sub-band (abbreviated as HL sub-band). Specifically, the above wavelet transform can be a horizontal wavelet transform and a vertical wavelet transform. Optionally, the HH sub-band, LH sub-band, and HL sub-band in the high-frequency sub-band can also be referred to as the first high-frequency sub-band, the second high-frequency sub-band, and the third high-frequency sub-band, respectively, in this application. The corresponding coefficients, such as quantization coefficients, can also be referred to as the quantization coefficients of the first high-frequency sub-band, the quantization coefficients of the second high-frequency sub-band, and the quantization coefficients of the third high-frequency sub-band.

[0165] In this embodiment, the low-frequency subband includes an LL subband, and the high-frequency subband includes an LH subband, an HH subband, and an HL subband. Optionally, the LL subband, LH subband, HH subband, and HL subband have the same dimensions (including width and height).

[0166] Still refer to Figure 8AThe low-frequency subband processing path 830 is used to encode the low-frequency subband to obtain low-frequency subband encoded data 806, which can also be called coded low-frequency subband. The high-frequency subband processing path 840 is used to encode the high-frequency subband to obtain high-frequency subband encoded data 809, which can also be called coded high-frequency subband.

[0167] Specifically, the wavelet forward transform unit 820 outputs wavelet coefficients 803 of the low-frequency subband to the transform / quantization unit 831, and outputs wavelet coefficients 804 of the high-frequency subband to the transform / quantization unit 841.

[0168] For example, the transform / quantization unit 831 is used to obtain the wavelet coefficients 803 of the low-frequency sub-band, perform quantization processing on the wavelet coefficients 803 of the low-frequency sub-band, or perform transform and quantization processing, and output the quantized coefficients 805 of the low-frequency sub-band, or the quantized wavelet coefficients of the low-frequency sub-band.

[0169] The transform / quantization unit 831 outputs the quantization coefficients 805 of the low-frequency subband to the low-frequency subband entropy encoding unit 832. Optionally, the transform / quantization unit 831 may include, but is not limited to, a transform processing unit and a quantization processing unit (not shown in the figure).

[0170] The transform processing unit is used to transform the low-frequency subband and output transform coefficients, also known as low-frequency subband transform coefficients. The quantization processing unit is used to quantize the subband or the transformed transform coefficients and output quantized coefficients (also known as quantization results).

[0171] The low-frequency subband entropy coding unit 832 is used to acquire the data to be encoded and perform entropy coding on the data to be encoded to obtain low-frequency subband encoded data. The data to be encoded may include, but is not limited to, the quantization coefficients 805 of the low-frequency subband. In some instances, the data to be encoded may also include parameters or information (or syntax elements) input from other modules not shown in the figure.

[0172] Optionally, the low-frequency subband entropy coding unit 832 is used to encode the data to be encoded using an entropy coding algorithm or scheme (e.g., at least one of the following: variable length coding (VLC), context adaptive VLC (CAVLC), arithmetic coding, and context adaptive binary arithmetic coding (CABAC)).

[0173] Table 2 shows a schematic diagram of low-frequency subband coded data.

[0174] Table 2

[0175]

[0176]

[0177] Where ae(v) indicates that the corresponding data uses arithmetic coding, se(v) indicates that the corresponding data uses exponential Golomb coding, ll_mb_qp_delta indicates the difference between the current red-box quantization parameter and the predicted quantization parameter, mb_mode indicates the prediction mode of the currently encoded unit (e.g., macroblock), and inter_no_residual_flag indicates whether the quantization residual system of the inter-frame coding mode is all zero. Exponential Golomb coding is a type of VLC coding scheme.

[0178] The control information includes at least the luma transform block size (tb_size_luma), the first flag of the luma intra-prediction mode (intra_pred_mode_luma_first_flag), the second flag of the luma intra-prediction mode (intra_pred_mode_luma_second_flag), the first flag of the chroma intra-prediction mode (intra_pred_mode_chroma_first_flag), and the second flag of the chroma intra-prediction mode (intra_pred_mode_chroma_second_flag). Specifically, intra_pred_mode_luma_first_flag and intra_pred_mode_luma_second_flag together represent the luma component intra-prediction mode, and intra_pred_mode_chroma_first_flag and intra_pred_mode_chroma_second_flag together represent the chroma component intra-prediction mode.

[0179] The low-frequency subband coded data is decoded to obtain the low-frequency subband macroblock quantization coefficients (decode_coefficients). Table 3 shows a schematic diagram of the low-frequency subband macroblock quantization coefficients.

[0180] Table 3. Format of Quantization Coefficients for Low-Frequency Subband Macroblocks

[0181]

[0182]

[0183] Among them, `coded_block_flag` indicates whether the transform block includes non-zero coefficients; a value of "1" indicates that the transform block contains non-zero coefficients, and a value of "0" indicates that all coefficients in the transform block are 0. `last_coeff_nz_flag` indicates whether it is the last position in the scanning order during encoding; a value of "0" indicates that the last position in the scanning order during encoding is a non-zero coefficient, and a value of "1" indicates the position of the last non-zero coefficient to be decoded. `last_nz_pos` indicates the position of the last non-zero coefficient, and the position of the last non-zero coefficient is related to the transform block size. `significant_coeff_flag` indicates whether the absolute value of the current quantization coefficient is greater than 0; a value of "0" indicates that the absolute value is equal to 0, and a value of "1" indicates that the absolute value is greater than 0. `coeff_abs_level_greater1_flag` indicates whether the absolute value of the current quantization coefficient is greater than 1; a value of "0" indicates that the absolute value is greater than 1, and a value of "1" indicates that the absolute value is less than or equal to 1. `coeff_abs_level_remaining` indicates the remaining value of the current quantization coefficient, and `coeff_sign_flag` indicates whether the current quantization coefficient is an integer or a negative number. (Refer to...) Figure 8A The transform / quantization unit 841 is used to quantize the wavelet coefficients 804 of the high-frequency subband, or to perform both transform and quantization processing, to obtain the quantized coefficients 807 of the high-frequency subband, which can also be referred to as the quantized wavelet coefficients of the high-frequency subband. The transform / quantization unit 841 outputs the quantized coefficients 807 of the high-frequency subband to the high-frequency subband entropy coding unit 842. For details not described, please refer to the low-frequency subband transform / quantization unit; further details are omitted here.

[0184] The control unit 843 is used to acquire control information based on at least one of the wavelet coefficients 804 and the quantization coefficients 807 of the high-frequency sub-band. In this embodiment, the control information is used to indicate the encoding mode used by the high-frequency sub-band during the encoding process. Correspondingly, during the decoding process, the decoding end can determine the encoding mode of the high-frequency sub-band based on the control information, thereby selecting the corresponding decoding mode.

[0185] In the embodiments of this application, the control information may include, but is not limited to, one or more syntax elements. For example, it may include wavelet coefficients or quantization coefficients of high-frequency subband blocks (i.e., macroblocks, or blocks to be coded) in the high-frequency subband, indicating whether they are all zero (i.e., all coefficients in the block are zero). Furthermore, the control information may also include syntax elements indicating entropy coding methods, etc., which are not limited in this application.

[0186] The following example illustrates the use of a syntax element (e.g., denoted as sigflag) in the control information to indicate that the coefficients of a high-frequency subband block are all zeros. In this embodiment, encoding and decoding are based on macroblocks as the basic unit. This can be understood as the wavelet coefficients of a high-frequency subband including wavelet coefficients from multiple high-frequency subband blocks. Each high-frequency subband block's wavelet coefficients can be understood as a set of wavelet coefficients, which may include one or more coefficients, or it may be an empty set. Each high-frequency subband block corresponds to a control information element used to indicate the encoding mode of that high-frequency subband block. For example, during encoding, the control information can determine the value of sigflag based on each high-frequency subband block. For instance, sigflag being 1 indicates that the quantization coefficients of the corresponding high-frequency subband block are all zeros, meaning that all quantization coefficients (e.g., up to 16 quantization coefficients) corresponding to that high-frequency subband block are 0 (of course, wavelet coefficients can also be understood as all zeros). A Sigflag value of 0 indicates that the quantization coefficients of the corresponding high-frequency subband block are not all zeros, meaning that all quantization coefficients corresponding to this high-frequency subband block include non-zero values.

[0187] The high-frequency subband entropy coding unit 842 is used to acquire the data to be encoded and perform entropy coding on the data to obtain high-frequency subband encoded data 809, also known as coded high-frequency subband. The high-frequency subband encoded data 808 includes, but is not limited to, VLC encoded data 809-1 and arithmetic encoded data 809-2. Optionally, the data to be encoded may include, but is not limited to, the quantization coefficients 807 and control information of the high-frequency subband. In some instances, the data to be encoded may also include parameters or information (or syntax elements) input from other modules not shown in the figure.

[0188] Table 4 shows a schematic diagram of one format of high-frequency subband encoded data.

[0189] Table 4

[0190]

[0191]

[0192] Among them, mb_has_coef_flag is the macroblock all-zero flag, with a value of "0" indicating that all coefficients of the macroblock are zero, and a value of "1" indicating that the macroblock has at least one non-zero coefficient; mb_all_one_flag is the macroblock all-one flag, with a value of "1" indicating that all 4x4 blocks in the macroblock contain non-zero coefficients, and a value of "0" indicating that at least one 4x4 block in the macroblock is an all-zero block; transform_skip_flag is the macroblock transform skip flag, with a value of "1" indicating that the macroblock skips the 2x2 Hadamard transform, and a value of "0" indicating that the macroblock performs the 2x2 Hadamard transform. If the bitstream does not have transform_skip_flag, the value of transform_skip_flag for the macroblock is 0, and the value of transform_skip_flag for the chroma macroblock is 0; significance_flag is the block importance flag (i.e., sigflag mentioned above), with a value of "0" indicating that all coefficients in the 4x4 block are zero, and a value of "1" indicating that the 4x4 block contains non-zero coefficients.

[0193] The high-frequency subband encoded data is decoded to obtain the high-frequency subband macroblock quantization coefficients decode_hf_coef. Table 5 shows a schematic diagram of the high-frequency subband macroblock quantization coefficients.

[0194] Table 5

[0195]

[0196] Among them, `block_mode_flag` is the block sparsity flag; a value of "0" indicates that the corresponding 4x4 block coefficients are sparse, and a special mode is used to parse the coefficient values; a value of "1" indicates that the block coefficients are dense, and the 16 coefficient values ​​are parsed using direct mode. `table_idx_flag` is the block code table flag; a value of "0" indicates that the offset of the code table index number is 0, and a value of "1" indicates that the offset of the code table index number is 1. `coeff_level` represents the current quantization coefficient value. `sub_significance_flag` is the sub-block importance flag; a value of "0" indicates that the current sub-block has 4... All coefficients are zero; a value of "1" indicates that the four coefficients include a non-zero coefficient. `pattern_0001_flag` is the special pattern flag for the sub-block coefficients; a value of "0" indicates that the four coefficients are not in a special pattern, and a value of "1" indicates that the four coefficients are in a special pattern. `pattern_0001_code` is the special pattern value for the sub-block coefficients, representing the pattern index value of the four coefficient values ​​in the current sub-block. `max_grt1_flag` is the absolute value flag for the sub-block coefficients; a value of "0" indicates that the absolute values ​​of the four coefficients in the current sub-block are all less than or equal to 1, and a value of "1" indicates that the absolute value of at least one of the four coefficients is greater than 1.

[0197] For example, the high-frequency subband entropy coding unit includes a VLC entropy coding unit 842-1 and an arithmetic entropy coding unit 842-2. The VLC entropy coding unit 842-1 is used to encode the quantization coefficients 807 of the high-frequency subband using the VLC entropy coding method (or entropy coding scheme / entropy coding algorithm) to generate VLC coded data. The VLC coded data includes VLC coded data of the HH high-frequency subband (also referred to as the first VLC coded data), VLC coded data of the HL high-frequency subband (also referred to as the second VLC coded data), and VLC coded data of the LH high-frequency subband (also referred to as the third VLC coded data).

[0198] In this embodiment, the encoding mode adopted by the VLC entropy encoding unit 842-1 may include, but is not limited to: encoding the quantization coefficients of high-frequency sub-band blocks with quantization coefficients that are not all zeros to obtain encoded data corresponding to the high-frequency sub-band block, and writing the encoded data of the high-frequency sub-band block into the image bitstream. For high-frequency sub-band blocks with quantization coefficients that are all zeros, the VLC entropy encoding unit does not perform entropy encoding. That is, the VLC encoded data 809-1 in this embodiment includes, but is not limited to, encoded data generated based on high-frequency sub-band blocks with quantization coefficients that are not all zeros. In this way, by reducing the amount of encoding, the encoding complexity can be effectively reduced, while the data volume of the bitstream can be reduced, further reducing the bandwidth occupied by the bitstream transmission, and shortening the transmission time between the encoding and decoding ends.

[0199] For example, the encoding mode adopted by the arithmetic entropy encoding unit 842-2 may include, but is not limited to: entropy encoding the control information of each high-frequency subband block to obtain the corresponding encoded data.

[0200] In the embodiments of this application, the VLC encoding method can also be the CAVLC encoding method, and this application does not limit it.

[0201] Optionally, the encoder 80 may also include, but is not limited to, a combining unit (not shown in the figure), which may also be called a multiplexer (MUX). The combining unit is used to generate an image bitstream based on the low-frequency subband coded data 806 and the high-frequency subband coded data 809.

[0202] Specifically, the combining unit writes low-frequency subband coded data 806 into the image bitstream and writes high-frequency subband coded data 809 into the image bitstream. In this embodiment, by encoding the low-frequency subband and high-frequency subband separately, the low-frequency subband coded data and high-frequency subband coded data can be decoded independently. That is, at the decoding end, it can independently decode the low-frequency subband coded data and high-frequency subband coded data in the image bitstream, thereby improving decoding efficiency.

[0203] As mentioned above, a single image can be divided into multiple sub-images, for example, N sub-images. Each sub-image can be decomposed into wavelet coefficients of a high-frequency sub-band and wavelet coefficients of a low-frequency sub-band after wavelet forward transform. Encoder 80 independently encodes the low-frequency and high-frequency sub-bands of each sub-image to obtain the low-frequency sub-band encoded data and high-frequency sub-band encoded data of a single sub-image. In this way, the combining unit can obtain the high-frequency sub-band encoded data and low-frequency sub-band encoded data of each of the N sub-images, that is, obtain N high-frequency sub-band encoded data (optionally, the high-frequency sub-band encoded data of each sub-image includes the HL sub-band encoded data corresponding to the HL sub-band, the HH sub-band encoded data corresponding to the HH sub-band, and the LH sub-band encoded data corresponding to the LH sub-band) and N low-frequency sub-band encoded data. The combining unit can optionally write the low-frequency sub-band encoded data and high-frequency sub-band encoded data of the N sub-images into the image bitstream. The format (or structure) of the image bitstream will be described in detail below.

[0204] exist Figure 8A In the coding scenario shown, the basic coding unit on the coding side is the macroblock of the high-frequency subband of each subgraph. This can be understood as follows: the high-frequency subband coding data includes the coding data of each subgraph, and the coding data of each subgraph includes the coding data of each high-frequency subband. The high-frequency subband coding data includes, but is not limited to: HL subband coding data, LH subband coding data, and HH subband coding data. The coding data of each high-frequency subband includes the coding data of each high-frequency subband block (i.e., macroblock). The coding data of each macroblock further includes: arithmetic coding data generated based on the control information corresponding to the macroblock, and VLC coding data generated based on the quantization coefficients of the high-frequency subband block.

[0205] In the embodiments of this application, the encoded data of each sub-graph can be decoded independently, the encoded data of each high-frequency sub-band of each sub-graph can be decoded independently, and the arithmetic encoded data and VLC encoded data of each high-frequency sub-band block of each high-frequency sub-band can be decoded independently.

[0206] Reference Figure 8B , Figure 8B This is a schematic / conceptual block diagram illustrating another encoder as an example. This architecture is contrasted with... Figure 8A The architecture lacks sub-image partitioning units. Specifically, the wavelet forward transform unit performs a wavelet transform on image 801 to generate high-frequency and low-frequency sub-bands of image 801. Other unit descriptions are as follows... Figure 8AThe same applies here, and will not be repeated. In this example, the high-frequency subband coded data 809 includes coded data of the image's high-frequency subbands, including but not limited to: HL subband coded data, LH subband coded data, and HH subband coded data. Each high-frequency subband coded data includes coded data of each high-frequency subband block (i.e., macroblock). The coded data of each macroblock further includes: arithmetic coded data generated based on the macroblock's control information, and VLC coded data generated based on the quantization coefficients of the high-frequency subband block.

[0207] In the embodiments of this application, the encoded data of each high-frequency subband can be decoded independently, and the arithmetic encoded data and VLC encoded data of each high-frequency subband block of each high-frequency subband can be decoded independently.

[0208] For example, during the process of writing each encoded data into the image bitstream by the combining unit, a first position pointer and a second position pointer can be set. The first position pointer is used to point to the starting position of the region where the VLC encoded data is located (e.g., the first image data region in the embodiment below), and the second position pointer is used to point to the starting position of the region where the arithmetic encoded data is located (e.g., the second image data region in the embodiment below). For example, during the process of writing the encoded data of the macroblock, the first position pointer and / or the second position pointer move according to the position of the starting position of the pointed image region in the image bitstream. Accordingly, after the encoded data is written, the combining unit can generate offset information corresponding to the positions of the arithmetic encoded data and the VLC encoded data in the image bitstream based on the current positions of the first position pointer and the second position pointer. A detailed description of the offset information can be found in the embodiment below, and will not be repeated here.

[0209] In this way, by combining the fast encoding and decoding speed of VLC encoding with the high compression ratio of arithmetic encoding on the encoding side, this application can improve encoding and decoding efficiency while ensuring the compression ratio of encoded data.

[0210] See Figure 11A , Figure 11A This is a schematic / conceptual block diagram of a decoder as an example. Decoder 110 is used to receive, for example, a bitstream encoded by an encoder to obtain a decoded image, also referred to as a reconstructed image 1108. During the decoding process, the decoder receives a bitstream (also referred to as the bitstream of the original image, or the image data of the original image) from the encoder.

[0211] exist Figure 11A In the example shown, the decoder 110 includes, but is not limited to: a high-frequency subband processing path 1120, a low-frequency subband processing path 1110, a wavelet inverse transform unit 1130, and a subgraph stitching unit 1140.

[0212] The low-frequency subband decoding path 1110 includes, but is not limited to: a low-frequency subband entropy decoding unit 1111 and an inverse quantization / inverse transform unit 1112 (also referred to as a low-frequency subband inverse quantization / inverse transform unit). The high-frequency subband decoding path 1120 includes, but is not limited to: a high-frequency subband entropy decoding unit 1121 and an inverse quantization / inverse transform unit 1122 (also referred to as a high-frequency subband inverse quantization / inverse transform unit).

[0213] For example, decoder 110 receives an image bitstream and obtains a reconstructed image 1108 (also known as a decoded image) based on high-frequency subband coded data 1102 and low-frequency subband coded data 1101 in the image bitstream.

[0214] For example, the low-frequency subband entropy decoding unit 1111 is used to acquire low-frequency subband encoded data 1101 in the image bitstream to obtain entropy decoding data, which includes, but is not limited to, the quantization coefficients 1103 of the low-frequency subband. Specifically, the low-frequency subband entropy decoding unit 1111 performs entropy decoding processing on the low-frequency subband encoded data 1101 to obtain entropy decoding data. The low-frequency subband entropy decoding unit 1111 outputs the quantization coefficients 1103 of the low-frequency subband to the inverse quantization / inverse transform unit 1112.

[0215] The inverse quantization / inverse transform unit 1112 is used to obtain the quantization coefficients 1103 of the low-frequency subband to obtain the reconstructed coefficients 1105 of the low-frequency subband, which can also be simply referred to as the reconstructed low-frequency subband. In some instances, it can also be called the reconstructed value of the low-frequency subband or the reconstructed wavelet coefficients of the low-frequency subband. Specifically, the low-frequency inverse quantization / inverse transform unit performs inverse quantization and / or inverse transform processing on the quantization coefficients to obtain the reconstructed low-frequency subband, such as the reconstructed LL subband. Here, inverse quantization can also be called inverse quantization, and inverse transform can also be called inverse transform.

[0216] Optionally, the inverse quantization / inverse transform unit 1112 may include, but is not limited to, an inverse quantization processing unit and an inverse transform processing unit (not shown in the figure). The inverse quantization processing unit can be used to perform inverse quantization processing on the input parameters to obtain inverse quantized coefficients, also known as inverse-quantized coefficients. The inverse transform unit can be used to perform inverse transform processing on the input parameters to obtain inverse transform coefficients, also known as inverse-transformed coefficients. For example, the inverse transform processing unit can perform inverse transform processing on the inverse-quantized coefficients output by the inverse quantization processing unit to obtain inverse-transformed inverse quantization coefficients.

[0217] Optionally, the reconstruction coefficients of the low-frequency subband can be the inverse quantization coefficients after inverse quantization processing, or the inverse transform coefficients after inverse quantization and inverse transform processing. The reconstruction coefficients of the high-frequency subband are described in the same way and will not be repeated below.

[0218] The high-frequency subband entropy decoding unit 1121 is used to acquire the high-frequency subband encoded data 1102 in the image bitstream to obtain entropy decoding data (also known as entropy encoding result). The entropy decoding data includes, but is not limited to, the quantization coefficients 1104 and control information of the high-frequency subband.

[0219] For example, the high-frequency subband entropy decoding unit 1121 includes, but is not limited to, a VLC entropy decoding unit 1121-1 and an arithmetic entropy decoding unit 1121-2. Specifically, the VLC entropy decoding unit 1121-1 uses VLC entropy decoding to entropy decode the VLC encoded data of each high-frequency subband block of each high-frequency subband in each sub-graph of the high-frequency subband encoded data 1102, to obtain the quantization coefficient isoentropy decoded data of the high-frequency subband block. The arithmetic entropy decoding unit 1121-2 uses arithmetic decoding to entropy decode the arithmetic encoded data of each high-frequency subband block of each high-frequency subband in each sub-graph of the high-frequency subband encoded data 1102, to obtain the control information isoentropy decoded data corresponding to each high-frequency subband block.

[0220] In one possible implementation, as described above, the control information is used to indicate the encoding mode of the quantization coefficients of the high-frequency subband block. The high-frequency subband entropy encoding unit 1121 can process the quantization coefficients (also called entropy decoding results) of the high-frequency subband block obtained by the VLC entropy decoding unit 1121-1 based on the control information to obtain the complete quantization coefficients of the high-frequency subband block. For example, as described above, the control information may include, but is not limited to, syntax elements indicating whether the quantization coefficients of the high-frequency subband block are all zeros, such as denoted as sigflag. Optionally, on the encoding side, the encoding mode uses the quantization coefficients of high-frequency subband blocks with non-all zero quantization coefficients encoded into the image bitstream, while the quantization coefficients of high-frequency subband blocks with all zero quantization coefficients are not encoded into the image bitstream. During the decoding process, the complete quantization coefficients of the high-frequency subband block can be obtained by first performing entropy decoding on the VLC encoded data to obtain the current quantization coefficients of the high-frequency subband block. The high-frequency subband entropy decoding unit 1121 can determine, based on control information, which high-frequency subband blocks have all 0 quantization coefficients and which have non-all 0 quantization coefficients. Correspondingly, the high-frequency subband entropy decoding unit 1121 can update the current quantization coefficients of the high-frequency subband blocks based on the control information to recover the complete quantization coefficients of the high-frequency subband blocks. The quantization coefficients 1104 of the high-frequency subband include the quantization coefficients of the high-frequency subband blocks with all 0 quantization coefficients and the quantization coefficients of the high-frequency subband blocks with non-all 0 quantization coefficients.

[0221] In another possible implementation, the VLC entropy decoding unit 1121-1 can perform entropy decoding on the VLC encoded data based on control information to obtain the quantization coefficients 1104 of the high-frequency subband. That is, during the entropy decoding process, the VLC entropy decoding unit 1121-1 can obtain the complete quantization coefficients of the high-frequency subband based on control information, including the quantization coefficients of the high-frequency subband blocks with all 0 quantization coefficients and the quantization coefficients of the high-frequency subband blocks with non-all 0 quantization coefficients.

[0222] Of course, in some instances, if all coefficients of the high-frequency subband block are not all zero, the high-frequency subband quantization coefficients decoded by the VLC entropy decoding unit 1121-1 are denoted as the high-frequency subband quantization coefficient 1104 in the figure.

[0223] For example, the coded data of each high-frequency sub-band can be decoded independently, and the arithmetic coded data and VLC coded data of each high-frequency sub-band block can be decoded independently. In this way, the decoding end can achieve parallel decoding of the quantization coefficients and control information of the high-frequency sub-band block based on different decoding methods, which can effectively improve decoding efficiency.

[0224] The high-frequency subband entropy decoding unit 1121 outputs the quantization coefficients 1104 of the high-frequency subband to the inverse quantization / inverse transform unit 1122.

[0225] The inverse quantization / inverse transform unit 1122 is used to obtain the quantization coefficients 1104 of the high-frequency subband, resulting in the reconstructed coefficients 1106 of the high-frequency subband, which can be simply referred to as the reconstructed high-frequency subband. In some instances, it can also be called the reconstructed value of the high-frequency subband, the reconstructed wavelet coefficients of the high-frequency subband, etc. The inverse quantization / inverse transform unit 1122 outputs the reconstructed high-frequency subband to the wavelet inverse transform unit 1130. Specifically, the inverse quantization / inverse transform unit 1122 performs inverse quantization processing on the quantization coefficients 1104 of the high-frequency subband, or performs inverse quantization and inverse transform processing, to obtain the reconstructed high-frequency subband. For example, this includes reconstructing the HH subband (e.g., the reconstructed coefficients of the HH subband), reconstructing the HL subband (e.g., the reconstructed coefficients of the HL subband), and reconstructing the LH subband (e.g., the reconstructed coefficients of the LH subband).

[0226] For example, the wavelet inverse transform unit 1130 is used to obtain the reconstruction coefficients 1105 of the low-frequency sub-band and the reconstruction coefficients 1106 of the high-frequency sub-band to obtain the reconstructed sub-map 1107. Specifically, the wavelet inverse transform unit 1130 can obtain the reconstructed low-frequency sub-band and the reconstructed high-frequency sub-band of each sub-map. The wavelet inverse transform unit performs wavelet inverse transform processing on the obtained reconstructed low-frequency sub-band and reconstructed high-frequency sub-band corresponding to the same sub-map to obtain the reconstructed sub-map. Its inverse transform process is the inverse of the wavelet forward transform process, which will not be described in detail here.

[0227] The sub-image stitching unit 1140, also known as an image reconstruction unit or sub-image combination unit, is used to obtain reconstructed sub-images of the original image to obtain a reconstructed image of the original image. Specifically, the sub-image stitching unit 1140 obtains N reconstructed sub-images corresponding to the current decoded image. Based on the N reconstructed sub-images, the sub-image stitching unit 1140 can obtain the reconstructed image (also known as the decoded image or the decoded image) of the current image.

[0228] exist Figure 8A and Figure 11A Based on the codec shown, there can be many more variations of the codec, such as including more processing units or modules.

[0229] See Figure 11B , Figure 11B This is a schematic / conceptual block diagram of a decoder, as exemplarily shown. The decoder corresponds to... Figure 8B The encoder architecture.

[0230] In this example, the wavelet inverse transform unit obtains the reconstructed image of the original image based on the reconstruction coefficients 1105 of the low-frequency subband and 1106 of the high-frequency subband, without requiring a sub-image stitching process. Descriptions of other units can be found in [reference needed]. Figure 11A This will not be elaborated upon here.

[0231] exist Figure 8B and Figure 11B Based on the codec shown, there can be many more variations of the codec, such as including more processing units or modules.

[0232] Reference Figure 12 , Figure 12 A schematic / conceptual block diagram of an encoder is shown as an example. Figure 12 In the example, the encoder includes, but is not limited to: subgraph partitioning unit 1210, wavelet forward transform unit 1220, low-frequency subband processing path 1230, and high-frequency subband processing path 1240.

[0233] The descriptions of subgraph partitioning unit 1210 and wavelet forward transform unit 1220 can be found in [reference]. Figure 8A The relevant content will not be repeated here.

[0234] The low-frequency subband processing path 1230 is used to obtain the wavelet coefficients 1203 of the low-frequency subband to obtain the low-frequency subband encoded data 1213. The low-frequency subband processing path 1230 includes, but is not limited to: a block partitioning unit 1231 (also called the low-frequency subband block partitioning unit 1231 or the first block partitioning unit 1231), a residual calculation unit 1232, a prediction unit 1237, a control unit 1238, a transform / quantization unit 1233 (also called the low-frequency subband transform / quantization unit), an inverse quantization / inverse transform unit 1234 (also called the low-frequency subband inverse quantization / inverse transform unit), a low-frequency subband splicing unit 1236, and a low-frequency subband entropy coding unit 1239, etc.

[0235] The high-frequency subband processing path 1240 is used to obtain the wavelet coefficients 1214 of the high-frequency subband in order to obtain the high-frequency subband coded data 1218. The high-frequency subband processing path 1240 includes, but is not limited to: block partitioning unit 1244 (also referred to as high-frequency subband block partitioning unit 1244 or second block partitioning unit 1244), transform / quantization unit 1241 (also referred to as high-frequency subband transform / quantization unit 1241 or second transform / quantization unit 1241), high-frequency subband entropy coding unit 1242, etc.

[0236] Alternatively, in some instances, the encoder may also include a... Figure 12 The structure shown may contain more or fewer units or modules.

[0237] The following is combined Figure 12 The encoder shown provides a detailed description of the image encoding method provided in this application:

[0238] The codec receives image 1201. A description of image 1201 can be found above and will not be repeated here.

[0239] Sub-image partitioning unit 1210 partitions image 1201 into sub-images and outputs N sub-images. N is an integer greater than 0. In this embodiment, each sub-image is encoded and decoded independently. During the encoding process, sub-image 1202 can be referred to as the current sub-image or the sub-image to be encoded.

[0240] Wavelet forward transform unit 1220 performs wavelet forward transform on the current sub-image to obtain wavelet coefficients 1203 (hereinafter referred to as low-frequency sub-band) and wavelet coefficients 1214 (hereinafter referred to as high-frequency sub-band) of the current sub-image. The wavelet coefficients 1203 of the low-frequency sub-band include the wavelet coefficients of the LL sub-band, and the high-frequency sub-band includes the wavelet coefficients of the LH, HL, and HH sub-bands. In this embodiment, each sub-image of the image can be independently encoded and decoded, and the high-frequency sub-band and low-frequency sub-band of each sub-image are independently encoded and decoded. The LH, HL, and HH sub-bands in the high-frequency sub-band can also be independently encoded and decoded.

[0241] The block partitioning unit 1231 (which may be called the low-frequency sub-band block partitioning unit) is used to obtain the wavelet coefficients 1203 of the low-frequency sub-band of the current sub-graph 1202, and to obtain at least one macroblock 1204 of the low-frequency sub-band of the sub-graph 1202 based on the wavelet coefficients 1203 of the low-frequency sub-band. The macroblock can also be understood as a set of partial coefficients in the wavelet coefficients of the low-frequency sub-band.

[0242] Specifically, the block partitioning unit 1231 partitions the wavelet coefficients 1203 of the low-frequency sub-band of the current subgraph 1202 into blocks based on the block partitioning method, obtaining at least one macroblock 1204 of the low-frequency sub-band of the current subgraph, for example, M macroblocks, where M is an integer greater than 0 (or greater than 1). The low-frequency block partitioning unit 1231 outputs the macroblocks 1204 of the wavelet coefficients 1203 of the current low-frequency sub-band one by one to the residual calculation unit 1232 and the control unit 1238.

[0243] In the embodiments of this application, macroblock 1204 is a basic encoding / decoding unit. During the encoding process, macroblock 1204 may also be referred to as the current block, the current image block, or the macroblock to be encoded, the block to be encoded, or the image block to be encoded, etc.

[0244] Alternatively, the block partitioning method includes, but is not limited to:

[0245] The wavelet coefficients 1203 of both the high-frequency subband and the low-frequency subband are divided into basic coding units of 8x8 macroblocks (in pixels).

[0246] For example, as described above, each subband uses a macroblock as the basic coding unit. The macroblock currently to be encoded is referred to as the current macroblock. Specifically, the low-frequency subband processing path 1230 encodes each macroblock 1204 of the wavelet coefficients 1203 of the low-frequency subband block by block. For example, encoding and prediction are performed on each macroblock 1204. The following description only uses the encoding process of the current macroblock 1204; the processing flow for other macroblocks is the same, and this application will not provide specific examples for each. For example, in encoding, it refers to the macroblock currently being encoded; in decoding, it refers to the macroblock currently being decoded. The decoded macroblock in the reference image used for predicting the current macroblock 1204 is called the reference block (i.e., the low-frequency subband reconstruction block 1209 in the figure). The reference block is the block that provides the reference signal for the current block, where the reference signal represents the pixel value within the reference block. The block in the reference image that provides the prediction signal for the current block can be called prediction block 1205, where the prediction signal represents the pixel value, sample value, or sample signal within prediction block 1205. For example, after traversing multiple reference blocks, an optimal reference block is found, which will provide the prediction for the current block; this block is called prediction block 1205.

[0247] Specifically, still refer to Figure 12The residual calculation unit 1232 is used to obtain the current macroblock 1204 and the prediction block 1205 (further details of the prediction block 1205 are provided below) to obtain the residual block 1206. Specifically, the residual calculation unit 1232 performs residual calculation on the current macroblock 1204 and the prediction block 1205 to obtain the residual block 1206. The residual calculation unit 1232 outputs the residual block 1206 to the transform / quantization unit 1233.

[0248] The transform / quantization unit 1233 is used to acquire the residual block 1206 to obtain the residual coefficients 1207. Specifically, the transform / quantization unit 1233 performs transform and / or quantization processing on the residual block 1206 to obtain the residual coefficients 1207, which can also be called the quantization coefficients of the residual block, or the quantized residual block. The transform / quantization unit 1233 outputs the residual coefficients 1207 to the inverse quantization single / inverse transform unit 1234 and the low-frequency subband entropy coding unit 1239.

[0249] The inverse quantization / inverse transform unit 1234, also known as the inverse quantization / inverse transform unit, is used to obtain the residual coefficients 1207 to obtain the residual reconstruction block 1208. Specifically, the inverse quantization / inverse transform unit 1234 performs inverse quantization and / or inverse transform processing on the residual coefficients 1207 to obtain the residual reconstruction block 1208, which can also be referred to as the inverse quantization coefficients of the residual block, the inverse quantized residual block, etc. The inverse quantization / inverse transform unit 1234 outputs the residual reconstruction block 1208 to the low-frequency subband reconstruction unit 1235.

[0250] The dequantization / inverse transform unit 1234 may include a dequantization unit and an inverse transform unit (not shown in the figure). The dequantization unit is used to dequantize the input coefficients, and the inverse transform unit is used to inverse transform the input coefficients.

[0251] The low-frequency subband reconstruction unit 1235 is used to obtain a low-frequency subband reconstruction block 1209 based on the prediction block 1205 and the residual reconstruction block 1208. Specifically, the low-frequency subband reconstruction unit 1235 adds the residual reconstruction block 1208 to the prediction block 1205 to obtain the low-frequency subband reconstruction block 1209, which can also be referred to as the reconstructed low-frequency subband macroblock. Optionally, the low-frequency subband reconstruction unit 1235 outputs the low-frequency subband reconstruction block 1209 to the prediction unit 1237 and the low-frequency subband splicing unit 1236. Optionally, the low-frequency subband reconstruction unit 1235 outputs the low-frequency subband reconstruction block 1209 to the control unit 1238.

[0252] The low-frequency subband stitching unit 1236 is used to obtain the reconstructed low-frequency subband 1211, which can also be referred to as the reconstructed value or reconstructed data of the low-frequency subband, based on the low-frequency subband reconstruction block 1209. Optionally, the low-frequency subband stitching unit 1236 outputs the reconstructed low-frequency subband 1211 to the prediction unit 1237. Optionally, the low-frequency subband stitching unit 1236 outputs the reconstructed low-frequency subband 1211 to the control unit 1238.

[0253] Specifically, as described above, the low-frequency subband uses macroblocks as the basic coding unit, and the low-frequency subband splicing unit 1236 can obtain M low-frequency subband reconstruction blocks of a low-frequency subband. Based on the M low-frequency subband reconstruction blocks, the low-frequency subband splicing unit 1236 can reconstruct the corresponding low-frequency subband, that is, obtain the reconstructed low-frequency subband.

[0254] The control unit 1238, also known as the mode selection unit, is used to determine the syntax element 1213 based on the macroblock 1204 (i.e. the current block); or to determine the syntax element 1213 based on the current macroblock 1204, the low-frequency subband reconstruction block 1209, and the reconstructed low-frequency subband 1211.

[0255] Syntax element 1213 includes at least one syntax element, such as pattern information. Pattern information, also known as prediction mode information, is used to indicate the prediction mode (or prediction method) of prediction unit 1237, such as inter-frame or intra-frame prediction mode. Control unit 1238 can output syntax element 1213 to prediction unit 1237 and low-frequency subband entropy coding unit 1239.

[0256] Prediction unit 1237, also known as prediction processing unit, is used to acquire syntax element 1213 and perform prediction processing based on syntax element 1213. Specifically, prediction unit 1237 may select a prediction mode based on syntax element 1213 (e.g., mode information in the syntax element). In one example, prediction unit 1237 may acquire low-frequency subband reconstruction block 1209 based on syntax element 1213 to acquire prediction block 1205. Specifically, prediction unit 1237 may perform intra-frame prediction based on low-frequency subband reconstruction block 1209 to acquire prediction block 1205. In another example, prediction unit 1237 may acquire reconstructed low-frequency subband 1211 based on syntax element 1213 to acquire prediction block 1205.

[0257] The prediction unit 1237 outputs prediction block 1205 to the residual calculation unit 1232 and the low-frequency sub-band splicing unit 1236.

[0258] The low-frequency subband entropy coding unit 1239 is used to obtain low-frequency subband encoded data 1213, also known as encoded low-frequency subband, based on residual coefficients 1207 and syntax elements 1213. Specifically, the low-frequency subband entropy coding unit 1239 uses an entropy coding algorithm or scheme (e.g., variable length coding (VLC) scheme, context adaptive VLC (CAVLC) scheme, arithmetic coding scheme, context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding methods or techniques) to entropy encode the residual coefficients 1207 and syntax elements 1213 to obtain low-frequency subband encoded data 1213 output in the form of, for example, an encoded bitstream.

[0259] Still refer to Figure 12 The block partitioning unit 1244, also known as the high-frequency subband block partitioning unit, is used to obtain the wavelet coefficients 1214 of the high-frequency subband of the current subgraph 1202, and to obtain the macroblocks 1215 of the high-frequency subband of the subgraph 1202 based on the wavelet coefficients 1214. These macroblocks, also known as high-frequency subband blocks, include quantization coefficients, and each macroblock has one or more quantization coefficients. For a detailed description, please refer to the low-frequency subband section, which will not be repeated here.

[0260] Specifically, the block partitioning unit 1244 partitions the high-frequency subband of the current subgraph 1202 (hereinafter referred to as the current high-frequency subband) into blocks based on the block partitioning method, to obtain at least one macroblock 1215 of the current subgraph 1202, for example, M macroblocks, where M is an integer greater than 0 (or an integer greater than 1). Other undescribed parts can be referred to the relevant description of the block partitioning unit 1231, and will not be repeated here.

[0261] Block partitioning unit 1244 outputs the macroblocks of the current high-frequency subband one by one to the transformation / quantization unit 1241.

[0262] The transform / quantization unit 1241 is used to transform and / or quantize the current macroblock 1215 to obtain the quantization coefficients 1216 of the macroblock (also referred to as the quantization coefficients of the high-frequency subband block). The transform / quantization unit 1241 outputs the quantization coefficients 1216 of the current macroblock to the high-frequency subband entropy coding unit 1242.

[0263] The control unit 1243 generates control information 1217 corresponding to the current macroblock based on macroblock 1214 and / or the quantization coefficients 1216 of the macroblock, and outputs the control information 1217 to the high-frequency subband entropy coding unit 1242.

[0264] The high-frequency subband entropy coding unit 1242 is used to perform entropy coding on the data to be encoded to obtain high-frequency subband coded data 1218. The data to be encoded may include, but is not limited to, quantization coefficients 1216 and control information 1217 of the high-frequency subband. The high-frequency subband coded data 1218 includes, but is not limited to, VLC coded data and arithmetic coded data for each high-frequency subband block of each subgraph. Figure 12 The processing flow of the high-frequency subband processing path 1240 can be referred to Figure 8A The relevant descriptions will not be repeated here.

[0265] Optionally, in Figure 12 Based on this, variations of the decoder in this application may also include a decoder architecture that does not include the sub-image partitioning unit 1210, wherein the macroblock partitioning units for the low-frequency subband and high-frequency subband can partition the wavelet coefficients of the image. Further descriptions can be found in [reference needed]. Figure 8B and Figure 12 This will not be elaborated upon here.

[0266] The lightweight compression encoding / decoding method provided in this application supports both full I-frame configuration and I / P frame alternating encoding configuration. Figure 12 The encoder architecture shown is in Figure 8A By adding relevant modules required for the prediction process to the wavelet transform architecture shown, the compression efficiency of I / P frame alternating coding for scenarios with fixed camera positions and slow camera movement can be improved.

[0267] Optionally, the encoder may also include a combining unit (not shown in the figure) for generating an image bitstream based on low-frequency subband coded data and high-frequency subband coded data, the specific process of which can be referred to above. Figure 8A Related descriptions.

[0268] The following provides a detailed description of the bitstream output by the encoder in the embodiments of this application. The bitstream structure described below can be applied to... Figure 8A , Figure 8B and Figure 12 The encoder shown can, of course, also be applied to other encoders based on... Figure 8A , Figure 8B or Figure 12 Encoder variant.

[0269] For example, as described above, the low-frequency subband entropy coding unit 1239 and the high-frequency subband entropy coding unit 1242 output low-frequency subband coded data 1213 and high-frequency subband coded data 1218, respectively. This can be understood as the encoder 120 independently encoding the low-frequency and high-frequency subbands of each subgraph, outputting low-frequency and high-frequency subband coded data corresponding to the current subgraph.

[0270] Reference Figure 13 , Figure 13 This is a schematic diagram illustrating the structure of an image bitstream as an example. Figure 13 In the example, the image bitstream includes, but is not limited to: image header information and image data (also referred to as image data region).

[0271] For example, the image data includes, but is not limited to, at least one image data region (also referred to as an image data sub-region), such as, but not limited to, a first image data region and a second image data region. During the encoding process, the encoder (e.g., via a combination unit) writes high-frequency subband encoded data and low-frequency subband encoded data into the image bitstream. For example, the encoder writes high-frequency subband encoded data into the first image data region and low-frequency subband encoded data into the second image data region.

[0272] For example, image header information includes, but is not limited to, offset information and image size information.

[0273] For example, image size information is used to indicate the size of the original image. As mentioned above, during the encoding process, some sub-images may be padded during sub-image partitioning to ensure that the length and width of each sub-image are multiples of 16. Thus, during decoding, the size of the reconstructed image obtained by the decoder may be larger than the original image size. The decoder can process the reconstructed image based on the image size information to remove the padded portions.

[0274] For example, offset information is used to indicate the position of a data region in the image bitstream, and can also be understood as indicating the position of independently decodeable coded data in the image bitstream. When decoding coded data (i.e., the image bitstream) according to this application, the offset information in the image header information can be used to obtain independently decodeable coded data, and decoding operations can be performed on the coded data. The independently decodeable coded data (e.g., low-frequency subband coded data and high-frequency subband coded data) can be decoded synchronously during decoding to improve decoding efficiency.

[0275] In one example, offset information can be the offset (or difference, typically measured in x bytes) between the start position of the image data region containing independently decodeable encoded data in the image bitstream and the end position of the image header information. For example, in Figure 13 In the example shown, the offset information may include, but is not limited to: the offset of the starting position of the fourth image data region relative to the ending position of the image header information, and the offset of the starting position of the second image data region relative to the ending position of the image header information. The starting position of the first image data region is the starting position of the image data; the offset information may or may not include this starting position. By decoding the image header information, these offsets are obtained, and the starting position of each independently decodeable segment of encoded data can be determined.

[0276] In another example, the offset information could be the difference between the starting positions of adjacent, independently decodeable encoded data regions within the image bitstream. For example, in Figure 13 In the example shown, the offset information may include, but is not limited to: Scheme 1: the length L0 of the first image data region, or the length L0 of the first image data region and the length L1 of the second image data region; and the length L3 of the third image data region, or the length L4 of the fourth image data region. Scheme 2: the length L3 of the third image data region and the length L4 of the fourth image data region. L0 = L3 + L4, where the starting position of the first image data region is the starting position of the image data, and the offset information may or may not include this starting position. The starting position of the third image data region is the same as the starting position of the first image data region. In this way, the decoder can obtain the high-frequency subband encoded data in the first image data region based on the length of the first image data region. And based on the length of the first image data region (L0 or L3 + L4) and the starting position of the first image data region, it can obtain the starting position of the second image data region and read the low-frequency subband encoded data therein. The reading method of the third data region is similar to that of the fourth data region. For example, based on the length (L3) of the third image data region and the starting position of the third image data region, the starting position of the fourth image data region can be obtained.

[0277] Optionally, Figure 13 (And the schematic diagrams of each bitstream in the embodiments below) The positions of the data regions corresponding to the encoded data shown are merely illustrative examples. For instance, in some instances, the encoded data of the low-frequency subband can be written into the first image data region, and the encoded data of the high-frequency subband can be written into the second image data region. As another example, arithmetic encoded data can be written into the third image data region, and VLC encoded data can be written into the fourth image data region; these will not be repeated below.

[0278] For example, as described above, the encoder sets a corresponding pointer at the starting position of each data region during the encoding process. Taking the pointer at the starting position of the fourth image data region as an example, each time the encoder writes encoded data of a high-frequency sub-band block into the third image data region, the length of the third image data region increases. The pointer at the starting position of the fourth image data region then moves backward according to the length of the field occupied by the encoded data of the high-frequency sub-band block (which can be greater than or equal to the length of the encoded data). After all encoded data has been written into the image data stream, the encoder can use the position pointed to by the current pointer of the fourth image data region to indicate the offset information of the fourth image data region's position.

[0279] It should be noted that the embodiments in this application only use the image bitstream of a single image as an example for illustration, that is, the bitstream includes only one image data. In the process of encoding video images, the encoder can generate an encoded image bitstream for each image, that is, the image bitstream includes multiple image data, and each image data carries the encoded data of the corresponding image.

[0280] Figure 13 Specifically, the high-frequency subband encoded data of the sub-image can be written into one image data region, and the low-frequency subband encoded data of all sub-images can be written into another image data region. Furthermore, the VLC encoded data of all sub-images is continuous, and the arithmetic encoded data of all sub-images is continuous.

[0281] In another embodiment, refer to Figure 14 , Figure 14 This is a schematic diagram illustrating the structure of an image bitstream as an example. Figure 14 In the example, the high-frequency subband encoded data for each subgraph includes: VLC encoded data and arithmetic encoded data.

[0282] The VLC encoded data further includes: VLC encoded data of the quantization coefficients of the HH subband (written into the first image data sub-region), VLC encoded data of the quantization coefficients of the HL subband (written into the second image data sub-region), and VLC encoded data of the quantization coefficients of the LH subband (written into the third image data sub-region).

[0283] The arithmetic-coded data further includes: arithmetic-coded data of control information for the HH subband (written into the fourth image data sub-region), arithmetic-coded data of control information for the HL subband (written into the fifth image data sub-region), and arithmetic-coded data of control information for the LH subband (written into the sixth image data sub-region).

[0284] exist Figure 14In the example shown, the offset information may include, but is not limited to: the position information of the first image data sub-region, the position information of the second image data sub-region, the position information of the third image data sub-region, the position information of the fourth image data region, and the position information of each image data sub-region, so that during decoding, the encoded data of the HH subband, HL subband, and LH subband in the VLC encoded data can be decoded independently to obtain the quantization coefficients of each subband. Also, the encoded data of the HH subband, HL subband, and LH subband in the arithmetic encoded data can be decoded independently to obtain the control information of each subband.

[0285] Reference Figure 15 , Figure 15 This is a schematic diagram illustrating the structure of an image bitstream as an example. Figure 15 In the example, the high-frequency subband encoded data is written to the corresponding image data region according to the type of subband. Specifically, taking the encoded data of the quantized data of the HL subband as an example, the encoded data of the quantized data of the HL subband of each sub-image (e.g., sub-image 1 to sub-image N) of the image is written to the first image data sub-region. The writing order of each encoded data in the figure is only an illustrative example.

[0286] In this context, for example, sub-image 1-HL in the figure represents the encoded data of the quantization coefficients of the HL subband of sub-image 1 in the image. The encoded data of the quantization coefficients of the HL subband of each sub-image further includes the encoded data of the quantization coefficients of the macroblocks (i.e., high-frequency subband blocks) of the HL subband of that sub-image. For example, sub-image 1-HL includes, but is not limited to: sub-image 1-HL-MB0 to sub-image 1-HL-MBm. Sub-image 1-HL-MBm represents the encoded data of the quantization coefficients of the macroblock MBm in the HL subband of sub-image 1. The HH subband encoded data is similar to the LH subband encoded data and the HL subband encoded data, and will not be described further here.

[0287] In this embodiment of the application, the low-frequency subband coding data includes, but is not limited to: LL subband coding data of each sub-image of the image (not shown in the figure).

[0288] It should be noted that, Figure 15 (And the bitstream diagrams in the examples below) only show the bitstream of the quantization coefficient encoded data. The bitstream of the quantization coefficient encoded data can be referred to the quantization coefficient encoded data (not shown in the figure). That is, the quantization coefficient encoded data of each high-frequency subband includes the quantization coefficient encoded data of each macroblock of each sub-map, which will not be repeated below.

[0289] For example, during encoding, the quantization coefficients and control information of each sub-band of each sub-graph are encoded independently; correspondingly, during decoding, the quantization coefficients and control information of each sub-band of each sub-graph can be decoded independently. Figure 15 In the example shown, it can be understood that the four coded subbands of each subgraph can be decoded independently, and the quantization coefficients and control information of each high-frequency subband can be decoded independently. For example, during decoding, the decoder can obtain the quantization coefficient encoded data of the HL subband, the LH subband encoded data, the HH subband encoded data, and the LL subband encoded data of subgraph 1 based on the offset information, and perform independent decoding to obtain the decoded subgraph 1.

[0290] In this example, the coded subbands and their coded macroblocks of each sub-image in the figure can also be written into the corresponding image data sub-regions, not shown in the figure and will not be repeated below. Correspondingly, offset information can be used to indicate the location of the image region to which the independently coded subbands of each sub-image belong. For example, in Figure 15 In the example, the offset information may optionally include, but is not limited to, the offset of the image region to which the coded sub-band of each sub-image belongs, that is, the difference between the starting position of the image data sub-region to which the coded sub-band of the sub-image belongs and the starting position of the adjacent preceding image sub-region. Of course, the offset information can also be the length of the sub-region to which the coded sub-band of each sub-image is located. Figure 15 The instructions provided are merely illustrative examples. During decoding, the decoder, based on offset information, can obtain the positions of the four encoded subbands (LL, HH, HL, and LH subband encoded data) of the VLC and arithmetic encoded data of a single sub-image within the image bitstream, and perform decoding to obtain the decoded subbands of the corresponding sub-image (including LL, HH, HL, and LH subband decoded data). The decoded data includes quantization coefficients and control information. For example, during decoding, the decoder obtains sub-image 1-HL (including each encoded macroblock contained in its sub-regions, the same applies below), sub-image 1-HH, sub-image 1-LH, and sub-image 1-LL. Thus, the decoder can obtain the decoded sub-image 1 based on the aforementioned encoded subbands, without needing to decode other sub-images sequentially according to the bitstream order. This improves decoding efficiency while reducing the usage of the decoder's buffer (e.g., DPB), lowering the hardware processing pressure and storage burden on the decoder.

[0291] In the embodiments of this application, the subbands in the high-frequency subband coded data of the image data can be interleaved and sorted. The interleaving and sorting can be at the sub-image granularity, the subband type granularity, or the macroblock (MB) granularity. Several interleaving and sorting methods are provided below. It should be noted that the interleaving methods shown in the embodiments of this application are only illustrative examples. In other embodiments, other interleaving methods can also be set according to the encoding and decoding requirements.

[0292] Reference Figure 16A , Figure 16A This is a schematic diagram illustrating the structure of an image bitstream as an example. Figure 16A In the example, the interleaving of VLC-coded data and arithmetic-coded data is done at the sub-band type of the subgraph.

[0293] Specifically, such as Figure 16A As shown, in VLC encoded data, the encoded data of the quantization coefficients of the HL subband, LH subband, and HH subband of each sub-image of the image are continuously written into the image bitstream.

[0294] In the arithmetic coding data, the coded data of the control information of the HL subband, LH subband, and HH subband of each sub-image of the image are continuously written into the image bitstream. The sub-image order and macroblock order shown in the figure are merely illustrative examples.

[0295] For example, such as Figure 16A As shown, sub-images 1-HL, 1-HH, and 1-LH are consecutively written into the image bitstream in VLC encoded data. Sub-image 1-HL represents the encoded data of the quantization coefficients of the HL subband of sub-image 1, which includes, but is not limited to, the encoded data of the quantization coefficients of all macroblocks (MB) in the HL subband of sub-image 1, such as MB0-HL to MBm-HL. Only the bitstream format of sub-image 1 is shown in the figure; other sub-images are similar and will not be described in detail here. The encoded data of the LL subband can be referred to above and will not be repeated here.

[0296] Reference Figure 16B , Figure 16B This is a schematic diagram illustrating the structure of an image bitstream as an example. Figure 16B In the example, the interleaving method is interleaving at the macroblock level of each subgraph.

[0297] Specifically, in Figure 16B In the example shown, in the VLC encoded data, the encoded data of the quantization coefficients of the high-frequency subbands of each sub-image of the image (including the encoded data of the quantization coefficients of the LH subband, the HH subband, and the HL subband) are continuously written into the third image data region.

[0298] In the arithmetic coding data, the coded data of the control information of the high-frequency subband of each sub-image of the image (including the coded data of the control information of the LH subband, the coded data of the control information of the HH subband and the coded data of the control information of the HL subband) are continuously written into the fourth image data area (not shown in the figure).

[0299] For example, such as Figure 16BAs shown, sub-images 1-HL-MB0, 1-HH-MB0, and 1-LH-MB0 are consecutively written into the first image data region. Specifically, sub-image 1-HL-MBm represents the encoded data of the quantization coefficients of macroblock MBm in the HL subband of sub-image 1; sub-image 1-HH-MBm represents the encoded data of the quantization coefficients of macroblock MBm in the HH subband of sub-image 1; and sub-image 1-LH-MBm represents the encoded data of the quantization coefficients of macroblock MBm in the LH subband of sub-image 1. The figure only shows the encoded data structure of sub-image 1 in the bitstream; the other sub-images are similar and will not be illustrated individually here.

[0300] In this example, during decoding, the decoding end can decode the high-frequency sub-band encoded data according to the sub-image order, that is, each sub-image in the third image data region and the fourth image data region is decoded independently. Figure 15 , Figure 16A as well as Figure 16C The difference lies in the decoding process. During decoding, the bitstream structure shown in the attached diagram is applied. The decoding end decodes the encoded data of different types of subbands in each sub-image of the VLC encoded data and arithmetic encoded data separately to obtain the quantization coefficients and control information of the four subbands of an image. It further obtains the inverse quantization coefficients of the sub-images and stores the additional data. For Figure 16B The bitstream structure shown requires the decoder to decode only the high-frequency and low-frequency subbands of each subgraph in the VLC-coded and arithmetic-coded data. For example, as... Figure 16B As shown, when decoding the first image data region, the decoding end can decode each coded macroblock one by one according to the coded macroblock order of each sub-image in the region. That is, the coded data of the three high-frequency sub-bands of sub-image 1 are written to the first image data region continuously. Therefore, during decoding, the coded data of the quantization coefficients of the three high-frequency sub-bands of sub-image 1 can be decoded one by one to obtain the quantization coefficients of the high-frequency sub-bands. No additional data storage is required, which can reduce the pressure on storage.

[0301] exist Figure 16B In the example shown, offset information is used to indicate the positions of the first image data region, the second image data region, the third image data region, and the fourth image data region. That is, Figure 16B In the example shown, both the VLC-encoded data and the arithmetic-encoded data include one independently decodeable sub-bitstream, which is different from... Figure 15 , Figure 16A as well as Figure 16C The framework, or decoding method, has lower hardware performance requirements, requiring only the simultaneous decoding of three sub-streams of the sub-graph (VLC encoded data and arithmetic encoded data corresponding to low-frequency sub-band encoded data and high-frequency sub-band encoded data).

[0302] Reference Figure 16C , Figure 16C This is a schematic diagram illustrating the structure of an image bitstream as an example. Figure 16C In the example, the interleaving method is interleaving at the sub-band of each subgraph.

[0303] Specifically, such as Figure 16C As shown, each sub-image in the VLC-coded data is continuously written to the third image data region according to its high-frequency sub-band type. For example, taking sub-image 1 as an example, the encoded data of the quantization coefficients of each macroblock in the HL sub-band of sub-image 1 is continuously written to the third image data region, that is, sub-image 1-HL-MB0 to sub-image 1-HL-MBm (representing the encoded data of the quantization coefficients of macroblocks) of sub-image 1 are continuously written to the image bitstream. The encoded data of the quantization coefficients of each macroblock in the HH sub-band encoded data of sub-image 1 is continuously written to the first image data region, that is, sub-image 1-HH-MB0 to sub-image 1-HH-MBm (representing the encoded data of the quantization coefficients of macroblocks) of sub-image 1 are continuously written to the image bitstream. The encoded data of the quantization coefficients of each macroblock in the LH sub-band encoded data of sub-image 1 is continuously written to the first image data region, that is, sub-image 1-LH-MB0 to sub-image 1-LH-MBm (representing the encoded data of the quantization coefficients of macroblocks) of sub-image 1 are continuously written to the image bitstream. Other subgraphs are similar and will not be described in detail here. The order of subband types is for illustrative purposes only.

[0304] Each sub-image in the arithmetic-coded data (not shown in the figure) is continuously written to the third image data region according to its high-frequency sub-band type. For example, taking sub-image 1 as an example, the encoded data of the control information of each macroblock in the HL sub-band of sub-image 1 is continuously written to the third image data region; the encoded data of the control information of each macroblock in the HH sub-band encoded data of sub-image 1 is continuously written to the first image data region; and the encoded data of the control information of each macroblock in the LH sub-band encoded data of sub-image 1 is continuously written to the first image data region. Other sub-images are similar and will not be described in detail here.

[0305] In this example, with Figure 15 Similar to the description in [the original text], each sub-band of each sub-image in the VLC-coded data and arithmetic-coded data can be decoded independently. Correspondingly, offset information can be used to indicate the location of the image region to which the independently coded sub-band of each sub-image belongs. This allows for the acquisition of independently decodeable coded data for each sub-image in the VLC-coded data and arithmetic-coded data based on the offset information during decoding, thereby improving decoding efficiency.

[0306] In the embodiments of this application, multiple independently decoded encoded data can be decoded simultaneously, or one or more high-frequency subbands can be decoded simultaneously, and the number of simultaneous decodes depends on the decoder hardware performance.

[0307] Reference Figure 17 , Figure 17A schematic / conceptual block diagram of a decoder is shown as an example. Figure 17 In the example, the decoder receives, for example, an image bitstream encoded by an encoder to obtain a decoded image of the original image, also known as a decoded image, reconstructed image, etc. During the decoding process, the decoder receives the image bitstream from the encoder, which includes, but is not limited to, image header information and image data. The image bitstream can be... Figure 13 A~ Figure 16C Any bitstream format.

[0308] exist Figure 17 In the example shown, the decoder 170 includes, but is not limited to: low-frequency subband processing path 1710, high-frequency subband processing path 1720, wavelet inverse transform unit 1730, image combination unit 1740 (also known as image stitching unit), etc.

[0309] For example, the low-frequency subband processing path 1710 is used to acquire low-frequency subband encoded data 1701 to acquire reconstructed low-frequency subband 1706 (also referred to as decoded low-frequency subband). The low-frequency subband processing path 1710 includes, but is not limited to: low-frequency subband entropy decoding unit 1711, inverse quantization / inverse transform unit 1712 (also referred to as low-frequency subband inverse quantization / inverse transform unit), low-frequency subband reconstruction unit 1713, low-frequency subband splicing unit 1715, prediction unit 1714, etc.

[0310] The high-frequency subband processing path 1720 is used to acquire high-frequency subband encoded data 1734 to acquire reconstructed high-frequency subband 1731, which can also be called the reconstructed value of high-frequency subband or the reconstructed data of high-frequency subband, including but not limited to: high-frequency subband entropy decoding unit 1721, inverse quantization / inverse transform unit 1721 (also called high-frequency subband inverse quantization / inverse transform unit), high-frequency subband reconstruction unit 1731, etc.

[0311] In some instances, Figure 17 The decoder shown can perform largely the same functions as the reference. Figure 12 The encoder describes the encoding process as the inverse of the decoding process.

[0312] The following is combined Figure 17 The decoder 170 shown herein provides a detailed description of the decoding method in the embodiments of this application.

[0313] For example, the decoder 170 can obtain high-frequency subband encoded data and low-frequency subband encoded data in the image bitstream based on the image header information in the image bitstream. Furthermore, as described above, during the encoding process, the encoder uses macroblocks as the basic encoding unit, and correspondingly, during the decoding process, the decoder also uses macroblocks (e.g., encoded macroblocks) as the basic decoding unit for decoding.

[0314] For example, the low-frequency subband entropy decoding unit 1711 performs entropy decoding on the low-frequency subband encoded data 1701 in the image bitstream, using macroblocks as the basic decoding unit, to obtain the quantization coefficients 1702 (i.e., the quantization coefficients of the current macroblock) and syntax elements 1707 of the low-frequency subband block. The description of the quantization coefficients 1702 of the low-frequency subband block can be found on the encoder side and will not be repeated here. Specifically, the low-frequency subband entropy decoding unit 1711 obtains the encoded macroblocks (i.e., the encoded data of the macroblocks) of the low-frequency subbands (e.g., LL subbands) of each sub-image in the image bitstream, and performs entropy decoding on each encoded macroblock to obtain the quantization coefficients 1702 (which can be simply referred to as the quantization coefficients of the low-frequency subband macroblock) and syntax elements 1707 of the corresponding low-frequency subband block for each encoded macroblock. During the decoding process, the currently decoded encoded macroblock can be called the current block.

[0315] The low-frequency subband decoding unit is used to output the quantization coefficients 1702 of the low-frequency subband block to the inverse quantization / inverse transform unit 1712, and to output the syntax elements 1707 to the prediction unit 1714.

[0316] The inverse quantization / inverse transform unit 1712 is used to obtain the quantization coefficients 1702 of the low-frequency subband block to obtain the inverse quantization coefficients 1703 of the low-frequency subband block. Specifically, the inverse quantization / inverse transform unit 1712 performs inverse quantization on the quantization coefficients of the current block of the low-frequency subband, or performs inverse quantization and inverse transform to obtain the inverse quantization coefficients of the current block of the low-frequency subband, or it can be the inverse transform coefficients of the current block of the low-frequency subband (depending on whether inverse transform processing has been performed). The inverse quantization / inverse transform unit 1712 outputs the inverse quantization coefficients 1703 of the low-frequency subband to the low-frequency subband reconstruction unit 1713, for example, the inverse quantization coefficients of the current block of the low-frequency subband.

[0317] The low-frequency subband reconstruction unit 1713 is used to obtain the low-frequency subband reconstruction block 1704, which can also be called the reconstruction coefficient of the low-frequency subband block, based on the inverse quantization coefficients 1703 and the prediction block 1705 of the low-frequency subband block. Specifically, the low-frequency subband reconstruction unit 1713 adds a prediction block to the inverse quantization coefficients of the current block of the low-frequency subband to obtain the low-frequency subband reconstruction block 1704 corresponding to the current macroblock.

[0318] The prediction unit 1714 is used to obtain syntax element 1707 and perform corresponding prediction processing according to syntax element 1707. For example, intra-frame prediction can be performed based on low-frequency subband reconstruction block 1704, or inter-frame prediction can be performed based on reconstructed low-frequency subband 1706. The execution method can be referred to the coding side, and will not be repeated here. The prediction unit 1714 outputs prediction block 1705 to the low-frequency subband reconstruction block 1704 unit.

[0319] For example, the high-frequency subband entropy decoding unit 1721 acquires the high-frequency subband encoded data 1734 in the image bitstream to obtain the quantization coefficients 1708 of the high-frequency subband block. The processing method is described with reference to FIG11, and will not be repeated here.

[0320] The inverse quantization / inverse transform unit 1722, also known as the high-frequency subband inverse quantization / inverse transform unit, is used to obtain the quantization coefficients 1708 of the high-frequency subband block to obtain the reconstruction coefficients 1709 of the high-frequency subband block. The reconstruction coefficients can be either inverse quantization coefficients after inverse quantization processing, or inverse transform coefficients after inverse quantization and inverse transform processing.

[0321] The high-frequency subband reconstruction unit 1723 is used to obtain the reconstruction coefficients 1709 of the high-frequency subband block to obtain the reconstructed high-frequency subband 1731, which can also be referred to as the reconstructed value or reconstructed data of the high-frequency subband. The reconstructed high-frequency subband may optionally include the reconstruction of the HL subband, the reconstruction of the HH subband, or the reconstruction of the LH subband.

[0322] The inverse wavelet transform unit 1730 is used to acquire the reconstructed high-frequency subband 1731 and the reconstructed low-frequency subband 1706 to obtain the reconstructed sub-image 1732. Specifically, the inverse wavelet transform unit 1730 acquires the reconstructed low-frequency subband 1706 output by the low-frequency subband stitching unit 1715 and the reconstructed high-frequency subband 1731 output by the high-frequency subband reconstruction unit 1723, and performs an inverse wavelet transform on the reconstructed low-frequency subband 1706 and the reconstructed high-frequency subband 1731 to obtain the reconstructed sub-image 1732. The inverse wavelet transform unit 1730 outputs the reconstructed sub-image 1732 to the image combining unit (which can also be called the image stitching unit, etc.).

[0323] Image combining unit 1740 is used to acquire reconstructed sub-images 1732 to obtain a reconstructed image 1733 of the original image, which can also be called a decoded image or a decoded image, etc. Specifically, image combining unit 1740 can acquire N reconstructed sub-images (N is an integer greater than 0) of the image (referring to the original image), and stitch (or combine) the N reconstructed sub-images according to the division method (including size and position) of each reconstructed sub-image during encoding to obtain the reconstructed image 1733.

[0324] Optionally, after obtaining the reconstructed image, the image combining unit 1740 can determine whether the reconstructed image contains a padding portion based on the image size information in the image header information and the size information of the current reconstructed image. In one example, if the size of the current reconstructed image is the same as the size indicated by the image size information (i.e., the same as the original image size), the image combining unit 1740 can send the reconstructed image to the display device. In this case, the sizes of the displayed image, the original image, and the reconstructed image are all the same. In another example, if the size of the current reconstructed image is not the same as the size indicated by the image size information (e.g., larger than the original image size), the image combining unit 1740 can remove the padding portion of the current reconstructed image based on the size indicated by the image size information to obtain the displayed image. The size of the displayed image is the same as the size of the original image. Specifically, in the embodiments of this application, when the encoding side performs sub-image division, the division order is preset, usually from left to right in the horizontal direction and from top to bottom in the vertical direction. Correspondingly, at least one sub-image that is filled is usually located at the edge of the image, such as... Figure 9B As shown. For example, the image combining unit 1740 can crop the vertical and / or horizontal edges of the image according to size information to remove the fill portion of the sub-image at the edge.

[0325] Optionally, in some instances, the image reconstruction unit may also perform the above-mentioned operation of removing the padding portion during the process of acquiring the reconstructed image, so that the size of the reconstructed image is the same as the size of the original image.

[0326] Optionally, the decoder is used, for example, to output a reconstructed image via the decoder's output port (or output interface) for presentation to or viewing by the user.

[0327] Other variations of the decoder can be used to decode compressed image bitstreams.

[0328] For example, as described above, each sub-image in an image bitstream may include multiple independently decodable encoded data. This can also be understood as a further inclusion of multiple independently decodable encoded data within independently decodable VLC encoded data and arithmetic encoded data. For a detailed description, please refer to... Figure 15 , Figure 16A , Figure 16B and Figure 16C Related descriptions.

[0329] Optionally, in the above scenario, the high-frequency subband processing path may include at least one high-frequency processing sub-path (not shown in the figure). For example, the high-frequency subband processing path may include multiple high-frequency processing sub-paths to simultaneously process the VLC-coded data and arithmetic-coded data of a subgraph. Of course, in some instances, the more high-frequency subband processing paths there are, the higher the decoding efficiency. The fewer the paths, the lower the requirements for hardware design complexity.

[0330] The following is based on Figure 16A The decoding method is illustrated with an example of the bitstream structure. Figure 16A In the example shown, the low-frequency subband entropy decoding unit 1701 acquires the LL subband encoded data and decodes it using macroblock-based decoding units. Taking sub-figure 1 as an example, the low-frequency subband entropy decoding unit 1701 acquires the LL subband encoded data of sub-figure 1 and decodes each encoded macroblock. The low-frequency subband decoding path 1710 processes the current block (inverse quantization / inverse transform, prediction, reconstruction, etc.) to output the reconstructed low-frequency subband of sub-figure 1 to the wavelet inverse transform unit 1730.

[0331] The high-frequency subband entropy decoding unit 1721 acquires the encoded data of the quantization coefficients of the LH subband, HL subband, and HH subband in the VLC encoded data, as well as the encoded data of the control information of the LH subband, HL subband, and HH subband in the arithmetic encoded data. Taking MB0 (i.e., the high-frequency subband block) of sub-image 1 as an example, specifically, the high-frequency entropy decoding unit 1721 acquires the encoded data of the control information and the encoded data of the quantization coefficients of the LH subband, HH subband, and MB0 of the LH subband in the image bitstream based on the offset information, and performs entropy decoding on them to obtain the control information of the LH subband, HH subband, and MB0 of the LH subband in sub-image 1. Based on the control information, it obtains the complete quantization coefficients of the LH subband, HH subband, and MB0 of the LH subband in sub-image 1.

[0332] The inverse quantization / inverse transform unit 1722 can process the quantization coefficients of MB0 in the LH subband, HH subband and LH subband of subgraph 1 to obtain the inverse quantization coefficients of MB0 in the LH subband, HH subband and LH subband.

[0333] The high-frequency subband reconstruction unit 1723 can obtain the reconstructed block of MB0 based on the inverse quantization coefficients of MB0 in the LH subband, HH subband, and LH subband. The processing of macroblocks in other sub-maps is similar and will not be described in detail here.

[0334] Optionally, if the encoder does not perform sub-image partitioning on the original image, then... Figure 17 The output of the wavelet inverse transform unit 1730 in the decoder can also be a reconstructed image of the original image, i.e. Figure 17 The image combination unit 1740 may not be included.

[0335] Reference Figure 18 , Figure 18 A schematic / conceptual block diagram of a decoder is shown as an example. Figure 18 In the example, the wavelet inverse transform unit 1730 can obtain the reconstruction block 1934 of the current block, which can also be called the reconstruction coefficients of the current macroblock, based on the reconstruction coefficients 1709 of the high-frequency sub-band block and the reconstruction coefficients of the low-frequency sub-band block (i.e., the low-frequency sub-band reconstruction block 1704). The sub-graph combination unit 1750 can obtain the reconstructed sub-graph 1732, which can also be called the reconstructed data of the current sub-graph, based on multiple reconstruction blocks 1934 of the sub-graph. Specifically, the sub-graph combination unit 1750 stitches together multiple reconstruction blocks of the sub-graph to obtain the corresponding reconstructed sub-graph. Descriptions of other modules and parameters can be found in [reference needed]. Figure 17 This will not be elaborated upon here.

[0336] In one possible implementation, other syntax elements in each high-frequency subband block besides the control information and quantization coefficients mentioned above can also be encoded and decoded using arithmetic coding or VLC coding. The control information can be understood as the syntax element used to obtain the quantization coefficients, enabling the decoder to obtain the complete quantization coefficients based on the control information. The other syntax elements besides the control information can be understood as being used in the decoding step after obtaining the quantization coefficients in the high-frequency subband processing path. For example, other syntax elements may include quantization parameters, which indicate the parameters used by the encoder to quantize the macroblock. Correspondingly, the decoder can obtain the inverse quantization coefficients based on the quantization parameters when performing inverse quantization.

[0337] Figure 19 A schematic block diagram of an apparatus 1900 according to an embodiment of this application is shown. The apparatus 1900 may include a processor 1901 and a transceiver 1902, and optionally, a memory 1903.

[0338] The various components of device 1900 are coupled together via bus 1904, which includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 1904 in the figure.

[0339] Optionally, the memory 1903 can be used to store instructions capable of implementing the foregoing method embodiments. The processor 1901 can be used to execute the instructions in the memory 1903, control the transceiver 1902 to receive signals, and control the transceiver 1902 to transmit signals.

[0340] The device 1900 may be an encoder and / or decoder as described in the above method embodiments. Exemplarily, the device may be a terminal device or a server.

[0341] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0342] This application also provides a chip, including one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the steps of the above-described related method steps to implement the methods in the above embodiments are executed. This embodiment also provides a computer-readable storage medium storing computer instructions, which, when executed on an electronic device, cause the electronic device to execute the above-described related method steps to implement the methods in the above embodiments. Exemplarily, the computer-readable storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0343] This embodiment also provides a computer program product containing computer instructions that, when executed by a computer or processor, cause the computer to perform the aforementioned related steps to implement the methods in the above embodiments. Exemplarily, the computer program product can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art.

[0344] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the methods in the above-described method embodiments.

[0345] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0346] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0347] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0348] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0349] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0350] Any content in the various embodiments of this application, as well as any content in the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0351] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A decoding method, characterized in that, include: Obtain the image bitstream of the original image, wherein the image bitstream includes arithmetic encoded data and variable-length encoded (VLC) data, wherein the arithmetic encoded data is obtained using arithmetic encoding and the VLC encoded data is obtained using VLC entropy encoding. Based on the arithmetic encoding data, control information is obtained; Based on the control information and the VLC encoded data, the quantization coefficients of the high-frequency subband are obtained; Based on the quantization coefficients of the high-frequency sub-band, the reconstruction coefficients of the high-frequency sub-band are obtained; Perform inverse wavelet transform on the reconstruction coefficients of the high-frequency subband to obtain the reconstructed image of the original image.

2. The method according to claim 1, characterized in that, The arithmetic encoded data and the VLC encoded data in the image bitstream can be decoded independently.

3. The method according to claim 2, characterized in that, The VLC encoded data includes first VLC encoded data generated based on the quantization coefficients of the first high-frequency sub-band, second VLC encoded data generated based on the quantization coefficients of the second high-frequency sub-band, and third VLC encoded data generated based on the quantization coefficients of the third high-frequency sub-band. The first VLC encoded data, the second VLC encoded data, and the third VLC encoded data in the image bitstream can be decoded independently.

4. The method according to claim 2 or 3, characterized in that, The image bitstream also includes offset information, which is used to indicate the position of independently decodeable encoded data in the image bitstream.

5. The method according to any one of claims 1-4, characterized in that, The control information is used to indicate whether the quantization coefficients of the high-frequency subband blocks are all zeros, wherein the quantization coefficients of the high-frequency subband blocks with non-all zeros are encoded into the image bitstream using the VLC entropy coding method.

6. The method according to claim 5, characterized in that, The step of obtaining the quantization coefficients of the high-frequency subband based on the control information and the VLC encoded data includes: When the control information indicates that the quantization coefficients of the high-frequency sub-band block are not all zero, the VLC encoded data is entropy decoded using the VLC entropy decoding method based on the control information to obtain the quantization coefficients of the high-frequency sub-band block. When the control information indicates that the quantization coefficient of the high-frequency sub-band block is all 0, the quantization coefficient of the high-frequency sub-band block is determined based on the control information.

7. The method according to claim 5, characterized in that, The step of obtaining the quantization coefficients of the high-frequency subband based on the control information and the VLC encoded data includes: The VLC encoded data is entropy decoded using the VLC entropy decoding method to obtain the VLC entropy decoding result. Based on the control information and the VLC entropy decoding result, the quantization coefficients of the high-frequency subband are obtained.

8. The method according to any one of claims 1-7, characterized in that, The process of obtaining control information based on the arithmetic encoding data includes: The arithmetic-coded data is entropy-decoded using an arithmetic decoding method to obtain the control information.

9. An encoding method, characterized in that, include: Wavelet transform is performed on the original image to obtain the wavelet coefficients of the high-frequency subband; The wavelet coefficients of the high-frequency subband are quantized to obtain the quantization coefficients of the high-frequency subband; Control information is obtained based on at least one of the quantization coefficients and wavelet coefficients of the high-frequency sub-band; The quantization coefficients of the high-frequency subband are subjected to variable-length coding (VLC) to obtain VLC-coded data; The control information is arithmetic-encoded to obtain arithmetic-coded data; The VLC encoded data and the arithmetic encoded data are used to generate the image bitstream of the original image.

10. The method according to claim 9, characterized in that, The arithmetic encoded data and VLC encoded data in the image bitstream can be decoded independently.

11. The method according to claim 10, characterized in that, The wavelet coefficients of the high-frequency sub-band include wavelet coefficients of the first high-frequency sub-band, wavelet coefficients of the second high-frequency sub-band, and wavelet coefficients of the third high-frequency sub-band; the VLC encoded data includes first VLC encoded data generated based on the quantization coefficients of the first high-frequency sub-band, second VLC encoded data generated based on the quantization coefficients of the second high-frequency sub-band, and third VLC encoded data generated based on the quantization coefficients of the third high-frequency sub-band. The first VLC encoded data, the second VLC encoded data, and the third VLC encoded data in the image bitstream can be decoded independently.

12. The method according to claim 10 or 11, characterized in that, The image bitstream also includes offset information, which is used to indicate the position of independently decodeable encoded data in the image bitstream.

13. The method according to claim 9, characterized in that, The control information is used to indicate whether the quantization coefficients of the high-frequency subband blocks in the high-frequency subband are all 0; wherein, the quantization coefficients of the high-frequency subband blocks with non-all 0 quantization coefficients are encoded into the image bitstream using the VLC entropy coding method.

14. A decoder, characterized in that, include: An interface unit is used to acquire the image bitstream of the original image. The image bitstream includes arithmetic encoded data and variable-length encoded (VLC) data. The arithmetic encoded data is obtained using arithmetic encoding, and the VLC encoded data is obtained using VLC entropy encoding. A high-frequency subband entropy decoding unit is used to obtain control information based on the arithmetic coding data; The high-frequency subband entropy decoding unit is also used to obtain the quantization coefficients of the high-frequency subband based on the control information and the VLC encoded data; A high-frequency inverse quantization unit is used to obtain the reconstruction coefficients of the high-frequency subband based on the quantization coefficients of the high-frequency subband. The wavelet transform unit is used to perform inverse wavelet transform on the reconstruction coefficients of the high-frequency sub-band and the reconstruction coefficients of the low-frequency sub-band to obtain the reconstructed image of the original image.

15. The decoder according to claim 14, characterized in that, The arithmetic encoded data and the VLC encoded data in the image bitstream can be decoded independently.

16. The decoder according to claim 15, characterized in that, The VLC encoded data includes first VLC encoded data generated based on the quantization coefficients of the first high-frequency sub-band, second VLC encoded data generated based on the quantization coefficients of the second high-frequency sub-band, and third VLC encoded data generated based on the quantization coefficients of the third high-frequency sub-band. The first VLC encoded data, the second VLC encoded data, and the third VLC encoded data in the image bitstream can be decoded independently.

17. The decoder according to claim 15 or 16, characterized in that, The image bitstream also includes offset information, which is used to indicate the position of independently decodeable encoded data in the image bitstream.

18. The decoder according to any one of claims 14-17, characterized in that, The control information is used to indicate whether the quantization coefficients of the high-frequency subband blocks are all zeros, wherein the quantization coefficients of the high-frequency subband blocks with non-all zeros are encoded into the image bitstream using the VLC entropy coding method.

19. The decoder according to claim 18, characterized in that, The high-frequency sub-band entropy decoding unit is specifically used for: When the control information indicates that the quantization coefficients of the high-frequency sub-band block are not all zero, the VLC encoded data is entropy decoded using the VLC entropy decoding method based on the control information to obtain the quantization coefficients of the high-frequency sub-band block. When the control information indicates that the quantization coefficient of the high-frequency sub-band block is all 0, the quantization coefficient of the high-frequency sub-band block is determined based on the control information.

20. The decoder according to claim 18, characterized in that, The high-frequency sub-band entropy decoding unit is specifically used for: The VLC encoded data is entropy decoded using the VLC entropy decoding method to obtain the VLC entropy decoding result. Based on the control information and the VLC entropy decoding result, the quantization coefficients of the high-frequency subband are obtained.

21. The decoder according to any one of claims 14-20, characterized in that, The high-frequency sub-band entropy decoding unit is specifically used for: The arithmetic-coded data is entropy-decoded using an arithmetic decoding method to obtain the control information.

22. An encoder, characterized in that, include: The wavelet transform unit is used to perform wavelet transform on the original image to obtain the wavelet coefficients of the high-frequency subband; A high-frequency subband quantization unit is used to quantize the wavelet coefficients of the high-frequency subband to obtain the quantization coefficients of the high-frequency subband. The control unit is configured to acquire control information based on at least one of the quantization coefficients and wavelet coefficients of the high-frequency sub-band. The high-frequency subband entropy coding unit is used to perform variable-length coding (VLC) on the quantization coefficients of the high-frequency subband to obtain VLC-coded data, and to perform arithmetic coding on the control information to obtain arithmetic-coded data, wherein the VLC-coded data and the arithmetic-coded data are used to generate the image bitstream of the original image.

23. The encoder according to claim 22, characterized in that, The arithmetic encoded data and VLC encoded data in the image bitstream can be decoded independently.

24. The encoder according to claim 23, characterized in that, The wavelet coefficients of the high-frequency sub-band include wavelet coefficients of the first high-frequency sub-band, wavelet coefficients of the second high-frequency sub-band, and wavelet coefficients of the third high-frequency sub-band; the VLC encoded data includes first VLC encoded data generated based on the quantization coefficients of the first high-frequency sub-band, second VLC encoded data generated based on the quantization coefficients of the second high-frequency sub-band, and third VLC encoded data generated based on the quantization coefficients of the third high-frequency sub-band. The first VLC encoded data, the second VLC encoded data, and the third VLC encoded data in the image bitstream can be decoded independently.

25. The encoder according to claim 23 or 24, characterized in that, The image bitstream also includes offset information, which is used to indicate the position of independently decodeable encoded data in the image bitstream.

26. The encoder according to claim 22, characterized in that, The control information is used to indicate whether the quantization coefficients of the high-frequency subband blocks in the high-frequency subband are all 0; wherein, the quantization coefficients of the high-frequency subband blocks with non-all 0 quantization coefficients are encoded into the image bitstream using the VLC entropy coding method.

27. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a computer or processor, causes the computer or processor to perform the method as described in any one of claims 1 to 13.

28. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a computer or processor, cause the steps of the method as described in any one of claims 1 to 13 to be performed.

29. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a bitstream that is generated according to the encoding method described in any one of claims 9 to 13.

Images