Coding methods and related apparatuses

Abstract

The disclosure provides a encoding method, including: obtaining N first syntax elements and quantized coefficients of each block group of N block groups based on a bitstream, where each first syntax element in the N first syntax elements indicates whether transform coefficients of all blocks in a respective block group of the N block groups are obtained by a first transform, each block group of the N block groups comprises M blocks, and N and M are positive integers; obtaining dequantized coefficients of each block group by performing an inverse quantization on the quantized coefficients of each block group; obtaining M inverse transform blocks for each block group by reforming an inverse transform on the dequantized coefficients of each block group, where the inverse transform is determined based on a respective first syntax element in the N first syntax elements; and reconstructing an image based on the M inverse transform blocks for each block group.

Description

Coding Methods and Related ApparatusesTECHNICAL FIELD

[0001] The present disclosure relates generally to the field of video coding technologies, and more particularly to coding methods and apparatuses.BACKGROUND

[0002] Video coding (encoding and decoding) is widely used in digital video applications, for example, a broadcast digital television, video transmission over the Internet and / or mobile networks, real-time conversational applications such as video chat and / or video conferencing, high-density digital versatile discs (DVDs) that are also referred to as BLU-RAY discs, video content capturing and editing systems, or security applications of camcorders.

[0003] With the development of the block-based hybrid video coding approach in the H.261 standard in 1990, some video coding technologies and tools are developed and form a basis for some video coding standards. Some video coding standards may include moving picture experts group 1 (MPEG-1) video, MPEG-2 video, international telecommunication union telecommunication standardization sector (ITU-T) H.262 / MPEG-2, ITU-T H.263, ITU-T H.264 / MPEG-4 part 10: advanced video coding (AVC), ITU-T H.265 / high efficiency video coding (HEVC), ITU-T H.266 / versatile video coding (VVC), or extensions, for example, scalability and / or 3D (three-dimensional) extensions, of these standards.

[0004] As videos are created and used more widely, video traffic is one of burdens on networks and data storage. Therefore, one of goals of most of video coding standards is to reduce a bit rate without sacrificing picture quality.SUMMARY

[0005] Implementations and / or aspects of the present disclosure provide encoding methods, decoding methods, and related apparatuses thereof, to reduce a bit rate without sacrificing picture quality and thus improve encoding efficiency or decoding efficiency.

[0006] According to a first aspect, a method is described. The method may be applied at a decoder side, for example, a decoder or a module in a decoder, a circuit or a chip that is responsible for a decoding function in a decoder, or a logical module or software that can implement all or some functions of a decoder. For example, the method is applied to a decoder. The method includes: obtaining N first syntax elements and quantized coefficients of each block group of N block groups based on a bitstream, where each first syntax element in the N first syntax elements indicates whether transform coefficients of all blocks in a respective block group of the N block groups are obtained by a first transform, each block group of the N block groups includes M blocks, and N and M are positive integers; obtaining dequantized coefficients of each block group by performing an inverse quantization on the quantized coefficients of each block group; obtaining M inverse transform blocks for each block group by performing an inverse transform on the dequantized coefficients of each block group, where the inverse transform is determined based on a respective first syntax element in the N first syntax elements; and reconstructing an image based on the M inverse transform blocks for each block group.

[0007] The terms “picture”, “frame”, or “image” may be used as synonyms.

[0008] Compared with some solutions in which per-block signaling is used to indicate transform type of each block, in these implementations in the first aspect, a group-based signaling is used to indicate transform type of a group of blocks, thereby reducing bit overhead for signaling the transform type and not sacrificing image quality.

[0009] In a possible implementation, one first syntax element in the N first syntax elements indicates that transform coefficients of all blocks in a first block group of the N block groups are obtained by the first transform, and the performing an inverse transform on the dequantized coefficients of each block group includes: obtaining an inverse transform block of each block in the first block group, by performing an inverse first transform on dequantized coefficients of a respective block in the first block group.

[0010] In a possible implementation, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further includes: obtaining M per-subgroup syntax elements based on the bitstream, where each per-subgroup syntax element of the M per-subgroup syntax elements indicates whether transform coefficients of all blocks in a respective subgroup in the second block group are obtained by the first transform. In this case, the performing an inverse transform on the dequantized coefficients of each block group includes: obtaining an inverse transform block of each block in the second block group based on the M per-subgroup syntax elements. For example, when a per-subgroup syntax element in the M per-subgroup syntax elements indicates that transform coefficients of all blocks in a respective subgroup in the second block group are obtained by the first transform, an inverse first transform may be performed on the transform coefficients of all the blocks in the respective subgroup. When another per-subgroup syntax element in the M per-subgroup syntax elements indicates that transform coefficients of not all blocks in a respective subgroup in the second block group are obtained by the first transform, reference may be made as the following description related to the second block group, where the block group may be replaced with the block subgroup.

[0011] In this case where blocks in the same block group are not of the same transform type, fine granularity syntax elements are farther decoded for smaller group of blocks, and each smaller group of blocks are subset of the original group of blocks.

[0012] In a possible implementation, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method farther includes: obtaining M per-block syntax elements based on the bitstream, where each per-block syntax element of the M per-block syntax elements indicates whether transform coefficients of a respective block in the second block group are obtained by the first transform; where the performing an inverse transform on the dequantized coefficients of each block group includes: obtaining an inverse transform block of each block in the second block group based on the M per-block syntax elements.

[0013] In this case, block-level syntax elements with finer granularity are used to indicate the transform type per block, and the decoder may know more details about the transform type, thereby improving the accuracy of decoding.

[0014] In a possible implementation, L per-block syntax elements in the M per-block syntax elements each indicates transform coefficients of a respective block in the second block group are obtained by the first transform, L is a non-negative integer less than M, and the obtaining an inverse transform block of each block in the second block group based on the M per-block syntax elements includes: obtaining an inverse transform block of each block in L blocks of the second block group by performing an inverse first transform on dequantized coefficients of a respective block in the L blocks.

[0015] In a possible implementation, the obtaining an inverse transform block of each block in the second block group based on the M per-block syntax elements further includes: obtaining an inverse transform block of eachblock in (M-L) blocks of the second block group by performing an inverse identity transform on dequantized coefficients of a respective block in the (M-L) blocks.

[0016] In a possible implementation, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further includes: obtaining M per-block syntax elements based on the bitstream, where each per-block syntax element of the M per-block syntax elements indicates a transform of a respective block in the second block group; where the performing an inverse transform on the dequantized coefficients of each block group includes: obtaining an inverse transform block of each block of the M blocks in the second block group by performing an inverse transform corresponding to a transform indicated by a respective per-block syntax element on dequantized coefficients of a respective block of the M blocks in the second block group.

[0017] In a possible implementation, a first block in the second block group is adjacent to the first block group, and the obtaining the M per-block syntax elements includes: obtaining a per-block syntax element of the first block by decoding the bitstream based on a probability model of the first block, where the probability model is determined based on transform of the first block group.

[0018] In some cases, it may be of a high probability that a transform type of a block may be the same as a transform type of a neighboring block. In these cases, group-level syntax element(s) available near to the current group-level syntax element may be taken into account to estimate probability of having specific transform-type(s) within a current group of blocks.

[0019] Since the first block in the second block group is adjacent to the first block group, it may be of a high probability that a transform type of the first block is the same as the transform type of the first block group. In this case, the decoder may perform entropy decoding on the per-block syntax element of the first block using probability model of the first block, which may help to improve the efficiency of entropy decoding.

[0020] In a possible implementation, the method further includes: obtaining a second syntax element based on the bitstream, where the second syntax element indicates the first transform.

[0021] In this case, the decoder may know the specific transform type of the first transform, and may perform the inverse first transform accordingly.

[0022] In a possible implementation, reconstructing an image based on the M inverse transform blocks for each block group includes: reconstructing the image by performing a second inverse transform on (MxN) inverse transform blocks.

[0023] In this case, the decoder may perform multiple inverse transforms, corresponding to multiple transforms performed by the encoder. The multiple inverse transforms may be considered as a combination of at least one inverse transform which may be performed sequentially.

[0024] In a possible implementation, the second inverse transform includes an inverse wavelet transform. In this case, the decoder may perform an inverse wavelet transform on the inverse transform blocks to reconstruct the image.

[0025] In a possible implementation, the blocks include image blocks.

[0026] In a possible implementation, the inverse quantization is determined based on one or more of: the inverse transform or the second inverse transform.

[0027] In this case, a dequantization strength may depend on the inverse transform type, which means that some of inverse transform types correspond to stronger dequantization strength, while other inverse transform typescorrespond to weaker dequantization strength. Alternatively, the dequantization strength may not depend on the inverse transform type. The quantization strength value may be determined by the decoder or the decoder may parse and decode the information about the quantization strength value from the bitstream.

[0028] In a possible implementation, the method may be applied to certain color channel(s) or color component(s). For example, each block includes luma samples, and the method may be applied to the luma samples.

[0029] In this case, flexibility and decoding performance may be may be improved. In addition, transform type of other color channel(s) or color component(s) may not be signaled, thereby further reducing the bit overhead.

[0030] According to a second aspect, an encoding method may be applied to an encoder side, for example, an encoder or a module in an encoder, a circuit or a chip that is responsible for an encoding function in an encoder, or a logical module or software that can implement all or some functions of an encoder. For example, the method is applied to an encoder. The method includes: obtaining transform coefficients of blocks in each block group of N block groups by performing a transform on blocks in a respective block group, where each block group of the N block groups includes M blocks, and N and M are positive integers; determining N first syntax elements, where each first syntax element in the N first syntax elements indicates whether transform coefficients of all blocks in a respective block group of N block groups are obtained by a first transform; and encoding the N first syntax elements.

[0031] In a possible implementation, one first syntax element in the N first syntax elements indicates that transform coefficients of all blocks in a first block group of the N block groups are obtained by the first transform.

[0032] In a possible implementation, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further includes: determining M per-block syntax elements, where each per-block syntax element of the M per-block syntax elements indicates whether transform coefficients of a respective block in the second block group are obtained by the first transform; and encoding the M per-block syntax elements.

[0033] In a possible implementation, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further includes: determining per-subgroup syntax elements, where each per-subgroup syntax element indicates whether transform coefficients of blocks in a respective subgroup in the second block group are obtained by the first transform, and encoding the per-subgroup syntax elements.

[0034] In a possible implementation, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further includes: determining M per-block syntax elements, where each per-block syntax element of the M per-block syntax elements indicates a transform of a respective block in the second block group; and encoding the M per-block syntax elements.

[0035] In a possible implementation, a first block in the second block group is adjacent to the first block group, and the encoding the M per-block syntax elements includes: encoding a per-block syntax element of the first block based on a probability model of the first block, where the probability model is determined based on transform of the first block group.

[0036] In a possible implementation, the method further includes: determining a second syntax element, where the second syntax element indicates the first transform; and encoding the second syntax element.

[0037] In a possible implementation, before the obtaining transform coefficients of blocks in each block group of N block groups by performing a transform on blocks in a respective block group, further including: performing a second transform on blocks in each block group of the N block groups.

[0038] In a possible implementation, the second transform includes a wavelet transform.

[0039] In a possible implementation, the blocks include image blocks.

[0040] In a possible implementation, the method further includes: obtaining quantized coefficients by performing a quantization on the transform coefficients of each block group, where quantization is determined based on the transform.

[0041] In a possible implementation, the method further includes: obtaining quantized coefficients by performing a quantization on the transform coefficients of each block group, where quantization is determined based on one or more of: the transform or the second transform.

[0042] In a possible implementation, each block includes luma samples.

[0043] According to a third aspect, an apparatus is described. The apparatus is configured to receive or restore a bitstream, where the bitstream is obtained by any one of the first aspect, the second aspect, or possible designs of the first aspect or the second aspect.

[0044] According to a fourth aspect, an apparatus is described. The apparatus has a function of implementing the first aspect or any possible design of the first aspect. For example, the communication apparatus includes a corresponding module, unit, or means (means) for performing operations in the first aspect or any possible design of the first aspect. The module, unit, or means may be specifically implemented by using software, may be implemented by using hardware, or may be implemented by using software in combination with hardware.

[0045] According to a fifth aspect, another apparatus is described. The apparatus includes a memory and one or more processors. The memory is configured to store a part or all of a necessary computer program or instructions for implementing a function in the first aspect or any possible design of the first aspect. The one or more processors may execute the computer program or the instructions, and when the computer program or the instructions is / are executed, the apparatus is enabled to implement the method in the first aspect, or any possible design or implementation of the first aspect.

[0046] According to a sixth aspect, an apparatus is described. The apparatus has a function of implementing the second aspect or any possible design of the second aspect. For example, the communication apparatus includes a corresponding module, unit, or means (means) for performing operations in the second aspect or any possible design of the second aspect. The module, unit, or means may be specifically implemented by using software, may be implemented by using hardware, or may be implemented by using software in combination with hardware.

[0047] According to a seventh aspect, another apparatus is described. The apparatus includes a memory and one or more processors. The memory is configured to store a part or all of a necessary computer program or instructions for implementing a function in the second aspect or any possible design of the second aspect. The one or more processors may execute the computer program or the instructions, and when the computer program or the instructions is / are executed, the apparatus is enabled to implement the method in the second aspect, or any possible design or implementation of the second aspect.

[0048] According to an eighth aspect, a computer-readable storage medium is described. The computer-readable storage medium stores computer-readable instructions, and when a computer reads and executes the computer-readable instructions, the computer is enabled to perform the method in any one of the first aspect, the second aspect, or possible designs of the first aspect to the second aspect.

[0049] According to a ninth aspect, a computer program product is described. When a computer reads and executes the computer program product, the computer is enabled to perform the method in any one of the first aspect, the second aspect, or possible designs of the first aspect to the second aspect.

[0050] According to a tenth aspect, a system is described. The system may include an apparatus for realizing the first aspect or any possible design of the first aspect and an apparatus for realizing the second aspect or any possible design of the second aspect.

[0051] This application encompasses various implementations, including not only method implementations, but also other implementations such as apparatus implementations and implementations related to computer readable storage media or computer program products. Implementations may incorporate, individually or in combinations, the features disclosed herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0052] For a better understanding of the present disclosure, as well as other aspects and further features thereof, reference is made to the following description which is to be used in conjunction with the accompanying drawings.

[0053] FIG. 1A is a block diagram of an example of a video coding system for implementing some implementations of the present disclosure.

[0054] FIG. IB is a block diagram of an example of another video coding system for implementing some implementations of the present disclosure.

[0055] FIG. 2 is a block diagram of an example structure of an encoder for implementing some implementations of the present disclosure.

[0056] FIG. 3 is a block diagram of an example structure of a decoder for implementing some implementations of the present disclosure.

[0057] FIG. 4 is a block diagram of an example of a video coding device for implementing some implementations of the present disclosure.

[0058] FIG. 5 is a block diagram of an example of another video coding device for implementing some implementations of the present disclosure.

[0059] FIG. 6 is a schematic flowchart of a video encoding method according to some implementations of the present disclosure.

[0060] FIG. 7 is a schematic illustration of an example wavelet transform according to some implementations.

[0061] FIG. 8 is a schematic illustration of a two-level dyadic wavelet transform according to some implementations.

[0062] FIG. 9 is a schematic illustration of another example wavelet transform according to some implementations.

[0063] FIG. 10 is a schematic illustration of transform types of different cases according to some implementations.

[0064] FIG. 11 is a schematic illustration of hierarchical signaling scheme according to some implementations.

[0065] FIG. 12 is a schematic illustration of group signaling according to some implementations.

[0066] FIG. 13 is a schematic illustration of an encoding process according to some implementations.

[0067] FIG. 14 is another schematic illustration of an encoding process according to some implementations.

[0068] FIG. 15 is a schematic flowchart of a video decoding method according to some implementations.

[0069] FIG. 16 is a schematic illustration of a simulation according to some implementations.

[0070] FIG. 17 is a schematic illustration of another simulation according to some implementations.

[0071] FIG. 18 is a block diagram of an example structure of an apparatus for implementing some implementations of the present disclosure.

[0072] FIG. 19 is another block diagram of an example structure of an apparatus for implementing some implementations of the present disclosure.DETAILED DESCRIPTION

[0073] The following describes embodiments of the present disclosure with reference to the accompanying drawings in embodiments of the present disclosure. In the following description, specific aspects of embodiments of the present disclosure or specific aspects in which embodiments of the present disclosure may be used are shown by the accompanying drawings. It is understood that embodiments of the present disclosure may be used in other aspects, and may include structural or logical changes not depicted in the accompanying drawings. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims. For example, it is understood that the disclosure with reference to the described method may also be applied to a corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, a corresponding device may include one or more units such as functional units for performing the described one or more method steps (for example, one unit performs the one or more steps, or a plurality of units, each of which performs one or more of the plurality of steps), even if such one or more units are not explicitly described or illustrated in the accompanying drawings. In addition, for example, if a specific apparatus is described based on one or more units such as a functional unit, a corresponding method may include one step for implementing functionality of one or more units (for example, one step for implementing functionality of one or more units, or a plurality of steps, each of which is for implementing functionality of one or more units in a plurality of units), even if such one or more of steps are not explicitly described or illustrated in the accompanying drawings. Further, it is understood that features of example embodiments and / or aspects described herein may be combined with each other, unless expressly stated otherwise.

[0074] In embodiments of the present disclosure, “at least one” means one or more, and “a plurality of’ means two or more. “And / or” describes an association relationship between associated objects, and represents that three relationships may exist. For example, A and / or B may represent the following cases: only A exists, both A and B exist, and only B exists, where A and B may be singular or plural. The character “ / ” generally indicates an “or” relationship between the associated objects. “At least one item (piece) of the following” or a similar expression thereof means any combination of these items, including a singular item (piece) or any combination of plural items (pieces). For example, at least one of a, b, or c may indicate: a; b; c; a and b; a and c; b and c; or a, b, and c, where a, b, and c may be singular or plural.

[0075] Embodiments of the present disclosure provide various technologies that may be used at a decoder and / or an encoder for processing image and / or video bitstreams to reduce bit overhead for images and / or videos. For brevity, the term “video” is used herein to include both a sequence of pictures (sometimes called video) and individual images.

[0076] The solutions described in embodiments of the present disclosure may be applicable to digital video applications, for example, a broadcast digital television, video transmission over the Internet and / or mobile networks, real-time conversational applications such as video chat and / or video conferencing, high-density digital versatile discs (DVDs) that are also referred to as BLU-RAY discs, video content capturing and editing systems, or security applications of camcorders. The solutions described in embodiments of the present disclosure may be also applicable to video coding standards like moving picture experts group 1 (MPEG-1) video, MPEG-2 video, international telecommunication union telecommunication standardization sector (ITU-T) H.262 / MPEG-2, ITU-T H.263, ITU-T H.264 / MPEG-4 part 10: advanced video coding (AVC), ITU-T H.265 / high efficiency video coding (HEVC), ITU-T H.266 / versatile video coding (VVC), audio video coding standard 3 (AVS3), VP8, VP9, or extensions, for example, scalability and / or 3D (three-dimensional) extensions, of these standards. The solutions described in embodiments of the present disclosure may be further applicable to future video coding standards or video codec.

[0077] The following first briefly describes some concepts that may be mentioned in embodiments of the present disclosure.

[0078] Video coding usually indicates processing of a sequence of pictures, where the sequence of pictures forms a video or a video sequence. In the field of video coding, the terms “picture”, “frame”, or “image” may be used as synonyms. Video coding used in this specification indicates video encoding or video decoding. Video encoding is performed on a source side, and usually includes processing (for example, through compressing) an original video picture to reduce an amount of data for representing the video picture, for more efficient storage and / or transmission. Video decoding is performed on a destination side, and usually includes inverse processing relative to an encoder, to reconstruct the video picture. Video picture “coding” in embodiments is understood as “encoding” or “decoding” of a video sequence. A combination of an encoding part and a decoding part is also referred to as coding (encoding and decoding).

[0079] A video sequence includes a series of pictures, a picture is further partitioned into slices, and a slice is further partitioned into blocks. In video coding, coding processing is performed per block. In some video coding standards, a concept “block” is further extended. For example, a macroblock (MB) is introduced in the H.264 standard. The macroblock may be further split into a plurality of prediction blocks for predictive coding. In the high efficiency video coding (HEVC) standard, a plurality of block units are functionally divided by using basic concepts such as a coding unit (CU), a prediction unit (PU), and a transform unit (TU), and are described by employing a tree-based structure. For example, a CU may be split into smaller CUs based on a quadtree, and a smaller CU may continue to be split to generate a quadtree structure. The CU is a basic unit for splitting and coding a to-be-coded image. A PU and a TU also have a similar tree structure. The PU may correspond to a prediction block and is a basic unit of predictive coding. The CU is further split into a plurality of PUs in a split mode. The TU may correspond to a transform block, and is a basic unit for transforming a prediction residual. However, in essence, all of the CU, the PU, and the TU are conceptually blocks (or image blocks). For a concept of an image block (for example, a first image block and a second image block) in embodiments of this disclosure, refer to description herein for details.

[0080] For example, in HEVC, a coding tree unit (CTU) is split into a plurality of CUs by using a quadtree structure represented as a coding tree. A decision on whether to code a picture area using inter-picture (temporal) or intra-picture (spatial) prediction is made at a CU level. Each CU may be further split into one, two, or four PUsbased on a PU splitting type. Inside one PU, a same prediction process is applied, and related information is transmitted to a decoder on a PU basis. After a residual block is obtained by applying the prediction process based on the PU splitting type, the CU may be partitioned into transform units (TUs) based on another quadtree structure similar to the coding tree used for the CU. In the recent development of video compression technologies, a quadtree plus binary tree (QTBT) partition frame is used for partitioning an encoding block. In a QTBT block structure, a CU may have a square or rectangular shape.

[0081] In this specification, for ease of description and understanding, a to-be-processed image block in a current coding image may be referred to as a current block, for example, in encoding, the to-be-processed image block refers to a block currently being encoded, and in decoding, the to-be-processed image block refers to a block currently being decoded. A decoded image block that is in a reference image and that is for predicting the current block is referred to as a reference block. In other words, the reference block is a block that provides a reference signal for the current block, where the reference signal indicates a pixel value in the image block. A block that provides a prediction signal for a current block in a reference image may be referred to as a prediction block. The prediction signal indicates a pixel value, a sampling value, or a sampling signal in the prediction block. For example, after a plurality of reference blocks are traversed, an optimal reference block is found, and the optimal reference block will provide prediction for the current block, and may be referred to as a prediction block.

[0082] In a case of lossless video coding, original video pictures can be reconstructed. In other words, reconstructed video pictures have same quality as the original video pictures (assuming that no transmission loss or other data loss occurs during storage or transmission). In a case of lossy video coding, further compression is performed through, for example, quantization, to reduce an amount of data for representing video pictures, and the video pictures may not be completely reconstructed on a decoder side. In other words, quality of reconstructed video pictures is lower or poorer than that of the original video pictures.

[0083] Several H.261 video coding standards are for “lossy hybrid video codecs” (in other words, spatial and temporal prediction in a sample domain is combined with two-dimensional (2D) transform coding for applying quantization in a transform domain). Each picture of a video sequence is usually split into a set of non-overlapping blocks, and coding is usually performed at a block level. In other words, at an encoder side, a video is usually processed, namely, encoded, at a block (video block) level, for example, using spatial (intra-picture) prediction and temporal (inter-picture) prediction to generate a prediction block, subtracting the prediction block from a current block (a block that is currently being processed or to be processed) to obtain a residual block, and transforming the residual block and quantizing the residual block in the transform domain to reduce an amount of data to be transmitted (compressed). At a decoder side, inverse processing relative to the encoder is applied to the encoded or compressed block to reconstruct the current block for representation. Furthermore, the encoder duplicates a decoder processing loop, so that the encoder and the decoder generate same prediction (for example, intra prediction and inter prediction) and / or reconstruction, for processing, that is, for coding subsequent blocks.

[0084] The following describes a system architecture to which embodiments of the present disclosure are applied. FIG. 1A is a schematic block diagram of an example of a video encoding and decoding system 10 to which one or more embodiments of the present disclosure is applied. As shown in FIG. 1 A, the video encoding and decoding system 10 may include a source device 12 and a destination device 14. The source device 12 generates encoded video data, and therefore the source device 12 may be referred to as a video encoding apparatus. The destination device 14 may decode the encoded video data generated by the source device 12, and therefore the destinationdevice 14 may be referred to as a video decoding apparatus. In various implementation solutions, the source device 12, the destination device 14, or both the source device 12 and the destination device 14 may include one or more processors and a memory coupled to the one or more processors. The memory may include but is not limited to a random-access memory (RAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or any other medium that can be used to store desired program code in a form of instructions or a data structure accessible to a computer, as described in this specification. The source device 12 and the destination device 14 may include various apparatuses, including a desktop computer, a mobile computing apparatus, a notebook (for example, a laptop) computer, a tablet computer, a set-top box, a telephone handset such as a “smart” phone, a television, a camera, a display apparatus, a digital media player, a video game console, a vehicle-mounted computer, a wireless communication device, or the like.

[0085] Although FIG. 1A depicts the source device 12 and the destination device 14 as separate devices, in one or more device embodiments, both the source device 12 and the destination device 14 or functionalities of both the source device 12 and the destination device 14, that is, the source device 12 or a corresponding functionality and the destination device 14 or a corresponding functionality, may alternatively be included. In such one or more embodiments, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality may be implemented by using same hardware and / or software or by using separate hardware and / or software or any combination thereof.

[0086] A communication connection between the source device 12 and the destination device 14 may be implemented through a link 13, and the destination device 14 may receive encoded video data from the source device 12 through the link 13. The link 13 may include one or more media or apparatuses capable of moving the encoded video data from the source device 12 to the destination device 14. In an example, the link 13 may include one or more communication media that enable the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. In this example, the source device 12 may modulate the encoded video data according to a communication standard (for example, a wireless communication protocol), and may transmit modulated video data to the destination device 14. The one or more communication media may include a wireless communication medium and / or a wired communication medium, for example, a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form a part of a packetbased network, and the packet-based network is, for example, a local area network, a wide area network, or a global network (for example, the internet). The one or more communication media may include a router, a switch, a base station, or another device that facilitates communication from the source device 12 to the destination device 14.

[0087] The source device 12 includes an encoder 20. For example, the source device 12 may further include a picture source 16, a picture preprocessor 18, and a communication interface 22. In a specific implementation form, the encoder 20, the picture source 16, the picture preprocessor 18, and the communication interface 22 may be hardware components in the source device 12, or may be software programs in the source device 12. The following describes the components separately.

[0088] The picture source 16 may include or be any type ofpicture capturing device configured to, for example, capture a real-world picture, and / or any type of device for generating a picture or comment (for screen content encoding, some text on a screen is also considered as a part of a to-be-encoded picture or image), for example, a computer graphics processor configured to generate a computer animation picture, or any type of deviceconfigured to obtain and / or provide a real-world picture or a computer animation picture (for example, screen content or a virtual reality (VR) picture), and / or any combination thereof (for example, an augmented reality (AR) picture). The picture source 16 may be a camera configured to capture a picture or a memory configured to store a picture. The picture source 16 may further include any type of (internal or external) interface through which a previously captured or generated picture is stored and / or a picture is obtained or received. When the picture source 16 is a camera, the picture source 16 may be, for example, a local camera, or an integrated camera integrated into the source device. When the picture source 16 is a memory, the picture source 16 may be a local memory or, for example, an integrated memory integrated into the source device. When the picture source 16 includes an interface, the interface may be, for example, an external interface for receiving a picture from an external video source. The external video source is, for example, an external picture capturing device such as a camera, an external memory, or an external picture generation device. The external picture generation device is, for example, an external computer graphics processor, a computer, or a server. The interface may be any type of interface, for example, a wired or wireless interface or an optical interface, according to any proprietary or standardized interface protocol.

[0089] A picture may be considered as a two-dimensional array or matrix of picture elements. A pixel in the array may also be referred to as a sample. A quantity of samples in horizontal and vertical directions (or axes) of the array or the picture defines a size and / or resolution of the picture. For representation of a color, three color components are usually employed. To be specific, the picture maybe represented as or include three sample arrays. For example, in a red, green, and blue (RGB) format or color space, a picture includes a corresponding red, green, and blue sample array. However, in video coding, each sample is usually represented in a luma / chroma format or color space. For example, a picture in a YUV format includes a luma component indicated by Y (or sometimes L) and two chroma components indicated by U and V. The luma component Y represents luma or gray level intensity (for example, both are the same in a gray-scale picture), and the two chroma components U and V represent chroma or color information components. Accordingly, the picture in the YUV format includes a luma sample array of luma sample values (Y) and two chroma sample arrays of chroma values (U and V). A picture in an RGB format may be transformed or converted to a picture in a YUV format and vice versa. This process is also referred to as color conversion or transform. If a picture is monochrome, the picture may include only a luma sample array. In one or more embodiments of the present disclosure, a picture transmitted by the picture source 16 to the picture processor such as a picture preprocessor 18 may also be referred to as raw picture data 17.

[0090] The picture preprocessor 18 is configured to receive raw picture data 17 and perform preprocessing on the raw picture data 17 to obtain a preprocessed picture 19 or preprocessed picture data 19. For example, the picture preprocessor 18 may perform preprocessing: clipping or trimming, color format transformation (for example, from the RGB format to the YUV format), color correction, or denoising.

[0091] The encoder 20 is configured to receive the preprocessed picture data 19, and process the preprocessed picture data 19 in a related prediction mode, to provide the encoded picture data 21 (the following further describes structural details of the encoder 20 based on FIG. 2, FIG. 4, or FIG. 5). In some embodiments, the encoder 20 may be configured to perform embodiments described below, to implement the video encoding method described in the present disclosure.

[0092] The communication interface 22 may be configured to receive the encoded picture data 21, and transmit the encoded picture data 21 to the destination device 14 or any other device (for example, a memory) through the link 13 for storage or direct reconstruction. The any other device may be any device configured for decoding orstorage. The communication interface 22 may be, for example configured to encapsulate the encoded picture data 21 into an appropriate format, for example, a data packet, for transmission through the link 13.

[0093] The destination device 14 includes a decoder 30. For example, the destination device 14 may further include a communication interface 28, a picture post-processor 32, and a display device 34. The following describes the components separately.

[0094] The communication interface 28 may be configured to receive the encoded picture data 21 from the source device 12 or any other source. The any other source is, for example, a storage device. The storage device is, for example, an encoded picture data storage device. The communication interface 28 may be configured to transmit or receive the encoded picture data 21 through the link 13 between the source device 12 and the destination device 14 or through any type of network. The link 13 is, for example, a direct wired or wireless connection, and the any type of network is, for example, a wired or wireless network or any combination thereof, or any type of private or public network, or any combination thereof. The communication interface 28 may be, for example configured to decapsulate the data packet transmitted through the communication interface 22, to obtain the encoded picture data 21.

[0095] Both the communication interface 28 and the communication interface 22 may be configured as unidirectional communication interfaces or bidirectional communication interfaces, and may be configured to, for example, send and receive messages to establish a connection, and acknowledge and exchange any other information related to a communication link and / or data transmission such as encoded picture data transmission.

[0096] The decoder 30 is configured to receive the encoded picture data 21 and provide the decoded picture data 31 or the decoded picture 31 (the following further describes structural details of the decoder 30 based on FIG. 3, FIG. 4, or FIG. 5). In some embodiments, the decoder 30 may be configured to perform embodiments described below, to implement the video decoding method described in the present disclosure.

[0097] The picture post-processor 32 is configured to post-process the decoded picture data 31 (also referred to as reconstructed picture data) to obtain post-processed picture data 33. The picture post-processor 32 may perform post-processing of color format transformation (for example, from a YUV format to an RGB format), color correction, trimming, re-sampling, or any other processing, and may be further configured to transmit the postprocessed picture data 33 to the display device 34.

[0098] The display device 34 is configured to receive the post-processed picture data 33 to display a picture, for example, to a user or a viewer. The display device 34 may be or include any type of display configured to present a reconstructed picture, for example, an integrated or external display or monitor. For example, the display may include a liquid-crystal display (LCD), an organic light-emitting diode (LED) (OLED) display, a plasma display, a projector, a micro-LED display, a liquid-crystal on silicon (LCoS), a digital light processor (DLP), or any type of other display.

[0099] Although FIG. 1A depicts the source device 12 and the destination device 14 as separate devices, in one or more device embodiments, both the source device 12 and the destination device 14 or functionalities of both the source device 12 and the destination device 14, that is, the source device 12 or a corresponding functionality and the destination device 14 or a corresponding functionality may alternatively be included. In such one or more embodiments, the source device 12 or the corresponding functionality and the destination device 14 or the corresponding functionality may be implemented by using same hardware and / or software or by using separate hardware and / or software or any combination thereof.

[0100] As will be apparent for a person skilled in the art based on the description, existence and (accurate) division of functionalities of the different units or functionalities of the source device 12 and / or the destination device 14 shown in FIG. 1A may vary with an actual device and application. The source device 12 and the destination device 14 each may include any one of a wide range of devices, including any type of handheld or stationary device, for example, a notebook or laptop computer, a mobile phone, a smartphone, a pad or a tablet computer, a video camera, a desktop computer, a set-top box, a television, a camera, a vehicle-mounted device, a display device, a digital media player, a video game console, a video streaming transmission device (such as a content service server or a content distribution server), a broadcast receiver device, and a broadcast transmitter device, and may not use or may use any type of operating system.

[0101] The encoder 20 and the decoder 30 each may be implemented as any one of various appropriate circuits, for example, one or more microprocessors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), discrete logic, hardware, or any combinations thereof. If the technologies are implemented partially by using software, a device may store software instructions in an appropriate and non-transitory computer-readable storage medium and may execute instructions by using hardware such as one or more processors, to perform the technologies of this disclosure. Any of the foregoing content (including hardware, software, a combination of hardware and software, and the like) may be considered as one or more processors.

[0102] In some cases, the video encoding and decoding system 10 shown in FIG. 1A is merely an example, and the technologies of the present disclosure may be applied to a video coding setting (for example, video encoding or video decoding) that does not need to include any data communication between encoding and decoding devices. In another example, data may be retrieved from a local memory, streamed over a network, or the like. A video encoding device may encode data and store data into the memory, and / or a video decoding device may retrieve and decode data from the memory. In some examples, encoding and decoding are performed by devices that do not communicate with each other but simply encode data into the memory and / or retrieve data from the memory and decode the data.

[0103] It is understood that the encoder 20 may be deployed on a terminal device or a server on a cloud side, the decoder 30 may be deployed on a terminal device or a server on a cloud side, or the encoder 20 and the decoder 30 may be jointly deployed on a terminal device or a server on a cloud side.

[0104] In a scenario, the encoder 20 and the decoder 30 may be deployed on a terminal device. The encoder 20 may perform coding compression on a video on the terminal device, perform secondary compression (or incremental storage compression) in the video encoding method provided in embodiments of this disclosure, and store compressed data. When the video needs to be played back, the stored compressed data may be decoded.

[0105] In a scenario, the encoder 20 and the decoder 30 may be deployed on a plurality of terminal devices. The encoder 20 may perform coding compression on a video on the terminal device, perform secondary compression (or incremental storage compression) in the video encoding method provided in embodiments of this disclosure, and transmit compressed data to another terminal device. When the other device needs to play back the video, the other device may decode the stored compressed data.

[0106] In a scenario, the encoder 20 and the decoder 30 may be deployed on a terminal device and a server on a cloud side. The encoder 20 may perform coding compression on a video on the terminal device, performsecondary compression (or incremental storage compression) in the video encoding method provided in embodiments of this disclosure, and transmit compressed data to the server.

[0107] FIG. IB is an illustrative diagram of an example of a video coding system 40 including the encoder 20 in FIG. 2 and / or the decoder 30 in FIG. 3 according to one or more example embodiments. The video coding system 40 can implement a combination of various technologies in embodiments of the present disclosure. In the illustrated implementations, the video coding system 40 may include an imaging device 41, the encoder 20, the decoder 30 (and / or a video encoder / decoder implemented by using a logic circuit (not shown) of a processing unit 46), an antenna 42, one or more processors 43, one or more memories 44, and / or a display device 45.

[0108] As shown in FIG. IB, the imaging device 41, the antenna 42, the processing unit 46, the logic circuit, the encoder 20, the decoder 30, the processor 43, the memory 44, and / or the display device 45 can communicate with each other. As described, although the video coding system 40 is depicted by using the encoder 20 and the decoder 30, the video coding system 40 may include only the encoder 20 or only the decoder 30 in different examples.

[0109] In some examples, the antenna 42 may be configured to transmit or receive encoded data of video data. Further, in some examples, the display device 45 may be configured to present the video data. In some examples, the logic circuit may be implemented by using the processing unit 46. The processing unit 46 may include ASIC logic, a graphics processor, a general-purpose processor, or the like. The video coding system 40 may alternatively include the optional processor 43. The optional processor 43 may similarly include ASIC logic, a graphics processor, a general-purpose processor, or the like. In some examples, the logic circuit may be implemented by hardware, for example, video coding dedicated hardware, and the processor 43 may be implemented by general-purpose software, an operating system, or the like. In addition, the memory 44 may be any type of memory, for example, a volatile memory (for example, a static RAM (SRAM) or a dynamic RAM (DRAM)) or a non-volatile memory (for example, a flash memory). In a non-limitative example, the memory 44 may be implemented by using a cache memory. In some examples, the logic circuit may access the memory 44 (for example, for implementation of an image buffer). In other examples, the logic circuit and / or the processing unit 46 may include a memory (for example, a cache) for implementation of an image buffer or the like.

[0110] In some examples, the encoder 20 implemented by using the logic circuit may include an image buffer (for example, implemented by using the processing unit 46 or the memory 44) and a graphics processing unit (for example, implemented by using the processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the encoder 20 implemented by using the logic circuit, to implement various modules that are described with reference to FIG. 2 and / or any other encoder system or subsystem described in this specification. The logic circuit may be configured to perform various operations described in this specification.

[0111] In some examples, the decoder 30 may be implemented by using the logic circuit in a similar manner, to implement various modules that are described with reference to the decoder 30 in FIG. 3 and / or any other decoder system or subsystem described in this specification. In some examples, the decoder 30 implemented by using the logic circuit may include an image buffer (implemented by using the processing unit 2820 or the memory 44) and a graphics processing unit (for example, implemented by using the processing unit 46). The graphics processing unit may be communicatively coupled to the image buffer. The graphics processing unit may include the decoder 30 implemented by using the logic circuit, to implement various modules that are described with reference to FIG.3 and / or any other decoder system or subsystem described in this specification.

[0112] In some examples, the antenna 42 may be configured to receive encoded data of video data. As described, the encoded data may include data, an indicator, an index value, mode selection data, or the like related to video frame encoding described in this specification, for example, data related to coding partitioning (for example, a transform coefficient or a quantized transform coefficient, an optional indicator (as described), and / or data defining coding partitioning). The video coding system 40 may further include the decoder 30 that is coupled to the antenna 42 and that is configured to decode the encoded data. The display device 45 is configured to present a video frame.

[0113] It is understood that in some embodiments of the present disclosure, for the example described with reference to the encoder 20, the decoder 30 may be configured to perform a reverse process. With regard to a signaling syntax element, the decoder 30 may be configured to receive and parse such a syntax element and correspondingly decode related video data. In some examples, the encoder 20 may entropy encode the syntax element into encoded video encoded data. In such examples, the decoder 30 may parse such a syntax element and correspondingly decode related video data.

[0114] FIG. 2 is a schematic / conceptual block diagram of an example of an encoder 20 configured to implement one or more embodiments of the present disclosure. In the example of FIG. 2, the encoder 20 includes a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a buffer 216, a loop filter unit 220, a decoded picture buffer (DPB) 230, a prediction processing unit 260, and an entropy encoding unit 270. The prediction processing unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a mode selection unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit. The encoder 20 shown in FIG. 2 may also be referred to as a hybrid video encoder or a video encoder based on a hybrid video codec.

[0115] For example, the residual calculation unit 204, the transform processing unit 206, the quantization unit 208, the prediction processing unit 260, and the entropy encoding unit 270 form a forward signal path of the encoder 20, whereas, for example, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the buffer 216, the loop filter 220, the DPB 230, and the prediction processing unit 260 form a backward signal path of the encoder, where the backward signal path of the encoder corresponds to a signal path of a decoder (refer to a decoder 30 in FIG. 3).

[0116] The encoder 20 receives, for example, via an input 202, a picture 201 or an image block 203 of the picture 201, for example, a picture in a sequence of pictures forming a video or a video sequence. The image block 203 may also be referred to as a current encoding block or a to-be-processed image block, and the picture 201 may be referred to as a current picture or a to-be-encoded picture (especially when the current picture is distinguished from another picture in video encoding, for example, the other picture is a previously encoded and / or decoded picture in a same video sequence, that is, a video sequence including the current picture).

[0117] One or more embodiments of the encoder 20 may include a partitioning unit (not depicted in FIG. 2) configured to partition the picture 201 into a plurality of blocks such as the image block 203. The picture 201 is usually partitioned into a plurality of non-overlapping blocks. The partitioning unit may be configured to use a same block size for all pictures in a video sequence and a corresponding grid defining the block size, or change a block size between pictures or subsets or picture groups and partition each picture into corresponding blocks.

[0118] In one example, the prediction processing unit 260 of the encoder 20 may be configured to perform any combination of the partitioning technologies described above.

[0119] Like the picture 201, the image block 203 is also or may be considered as a two-dimensional array or matrix of samples with sample values, although of a smaller size than the picture 201. In other words, the image block 203 may include, for example, one sample array (for example, a luma array in a case of a monochrome picture 201), three sample arrays (for example, one luma array and two chroma arrays in a case of a color picture), or any other quantity and / or type of arrays depending on an applied color format. A quantity of samples in horizontal and vertical directions (or axes) of the image block 203 defines a size of the image block 203.

[0120] The encoder 20 shown in FIG. 2 is configured to encode the picture 201 block by block, for example, perform encoding and prediction on each image block 203.

[0121] The residual calculation unit 204 is configured to calculate a residual block 205 based on the picture image block 203 and a prediction block 265 (further details about the prediction block 265 are provided below), for example, obtain the residual block 205 in a sample domain by subtracting sample values of the prediction block 265 from sample values of the picture image block 203 sample by sample (pixel by pixel).

[0122] The transform processing unit 206 is configured to apply a transform, for example, a discrete cosine transform (DCT) or a discrete sine transform (DST), to sample values of the residual block 205 to obtain transform coefficients 207 in a transform domain. The transform coefficients 207 may also be referred to as transform residual coefficients and represent the residual block 205 in the transform domain.

[0123] The transform processing unit 206 may be configured to apply integer approximations of DCT / DST, such as transform specified in HEVC / H.265. Compared with an orthogonal DCT transform, such integer approximations are usually scaled by a factor. To preserve a norm of a residual block which is processed by using forward transform and inverse transform, applying an additional scaling factor is a part of a transform process. The scaling factor is usually chosen based on some constraints, for example, the scaling factor being a power of two for a shift operation, a bit depth of the transform coefficient, or a tradeoff between accuracy and implementation costs. A specific scaling factor is, for example, specified for an inverse transform, for example, by the inverse transform processing unit 212 on the decoder side 30 (and the corresponding inverse transform, for example, by the inverse transform processing unit 212 on the encoder side 20), and a corresponding scaling factor for the forward transform, for example, by the transform processing unit 206 on the encoder side 20 may be specified accordingly.

[0124] It is noted that the transform may not be applied in a case of a transform-skip mode. Moreover, the transform may be applied to block-level basis, frame-level basis, or sub-picture-level basis, which may be depend on the actual need.

[0125] The quantization unit 208 is configured to quantize the transform coefficients 207 to obtain quantized transform coefficients 209, for example, through scalar quantization or vector quantization. The quantized transform coefficients 209 may also be referred to as the quantized residual coefficients 209. A quantization process may reduce a bit depth related to a part or all of the transform coefficients 207. For example, an n-bit transform coefficient may be rounded down to an m-bit transform coefficient during quantization, where n is greater than m. A quantization degree may be modified by adjusting a quantization parameter (QP). For example, for scalar quantization, different scales may be applied to achieve finer or coarser quantization. A smaller quantization step size corresponds to finer quantization, and a larger quantization step size corresponds to coarserquantization. An appropriate quantization step size may be indicated by the QP. For example, the quantization parameter may be an index to a predefined set of appropriate quantization step sizes. For example, a smaller quantization parameter may correspond to finer quantization (a smaller quantization step size) and a larger quantization parameter may correspond to coarser quantization (a larger quantization step size), or vice versa. The quantization may include division by a quantization step size and corresponding quantization or inverse quantization, for example, performed through the inverse quantization 210, or may include multiplication by a quantization step size. Embodiments according to some standards such as HEVC may use a quantization parameter to determine the quantization step size. Generally, the quantization step size may be calculated based on a quantization parameter by using a fixed point approximation of an equation including division. Additional scaling factors may be introduced for quantization and dequantization, to restore the norm of the residual block, where the norm of the residual block may be modified because of a scale used in the fixed point approximation of the equation for the quantization step size and the quantization parameter. In one example implementation, a scale of the inverse transform may be combined with a scale of dequantization. Alternatively, a customized quantization table may be used and signaled from an encoder to a decoder, for example, in encoded data. The quantization is a lossy operation, where the loss increases with increasing of the quantization step size.

[0126] The inverse quantization unit 210 is configured to apply the inverse quantization of the quantization unit 208 to quantized coefficients to obtain dequantized coefficients 211, for example, apply, based on or by using a same quantization step size as the quantization unit 208, the inverse of a quantization scheme applied by the quantization unit 208. The dequantized coefficients 211 may also be referred to as dequantized residual coefficients 211, and correspond, although usually different from the transform coefficients due to a loss caused by quantization, to the transform coefficients 207.

[0127] The inverse transform processing unit 212 is configured to apply the inverse transform of the transform applied by the transform processing unit 206, for example, an inverse DCT or an inverse DST, to obtain an inverse transform block 213 in the sample domain. The inverse transform block 213 may also be referred to as an inverse transform dequantized block 213 or an inverse transform residual block 213.

[0128] The reconstruction unit 214 (for example, an addition 214) is configured to add the inverse transform block 213 (that is, a reconstructed residual block 213) to the prediction block 265 to obtain a reconstructed block 215 in the sample domain, for example, by adding sample values of the reconstructed residual block 213 and the sample values of the prediction block 265.

[0129] For example, a buffer unit 216 (or a “buffer” 216), for example, a line buffer 216, is configured to buffer or store the reconstructed block 215 and a corresponding sample value, for example, for intra prediction. In other embodiments, the encoder may be configured to use unfiltered reconstructed blocks and / or corresponding sample values stored in the buffer unit 216 for any type of estimation and / or prediction, for example, intra prediction.

[0130] For example, in some embodiments, the encoder 20 may be configured, so that the buffer unit 216 is used for storing the reconstructed block 215 for intra prediction 254 and also used for the loop filter unit 220, and / or so that, for example, the buffer unit 216 and the DPB unit 230 form one buffer. In other embodiments, filtered blocks 221 and / or blocks or samples from the DPB 230 (the blocks or samples are not shown in FIG. 2) are used as an input or a basis for intra prediction 254.

[0131] The loop filter unit 220 (or “loop filter” 220) is configured to filter the reconstructed block 215 to obtain the filtered block 221, to smooth pixel transition or improve video quality. For example, the loop filter unit 220 isintended to represent one or more loop filters including a de-blocking filter, a sample -adaptive offset (SAO) filter, and another filter, for example, a bilateral filter, an adaptive loop filter (ALF), a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit is shown in FIG. 2 as an in-loop filter, the loop filter unit 220 may be implemented as a post- loop filter in other configurations. The filtered block 221 may also be referred to as a filtered reconstructed block 221. The DPB 230 may store a reconstructed encoded block after the loop filter unit 220 performs a filtering operation on the reconstructed encoded block.

[0132] In one or more embodiments, the encoder 20 (correspondingly, the loop filter unit 220) may be configured to output a loop filter parameter (such as sample adaptive offset information), for example, directly or after entropy encoding performed by the entropy encoding unit 270 or any other entropy encoding unit, so that, for example, the decoder 30 can receive the same loop filter parameter and apply the same loop filter parameter to decoding.

[0133] The DPB 230 may be a reference picture memory that stores reference picture data for use in video data encoding by the encoder 20. The DPB 230 may be formed by any one of a variety of memory devices, such as a DRAM (including a synchronous DRAM (SDRAM), a magnetoresistive RAM (MRAM), and a resistive RAM (RRAM)), or another type of memory devices. The DPB 230 and the buffer 216 may be provided by a same memory device or separate memory devices. In an example, the DPB 230 is configured to store the filtered block 221. The DPB 230 may be further configured to store other previously filtered blocks, for example, previously reconstructed and filtered blocks 221, of the same current picture or of different pictures, for example, previously reconstructed pictures, and may provide complete previously reconstructed, namely, decoded, pictures (and corresponding reference blocks and samples) and / or a partially reconstructed current picture (and corresponding reference blocks and samples), for example, for inter prediction. In an example, if the reconstructed block 215 is reconstructed without in-loop filtering, the DPB 230 is configured to store the reconstructed block 215.

[0134] The prediction processing unit 260, also referred to as a block prediction processing unit 260, is configured to receive or obtain the image block 203 (a current image block 203 of the current picture 201) and reconstructed picture data, for example, reference samples of the same (current) picture from the buffer 216 and / or reference picture data 231 of one or more previously decoded pictures from the DPB 230, and to process such data for prediction, namely, to provide the prediction block 265 that may be an inter prediction block 245 or an intra prediction block 255.

[0135] The mode selection unit 262 may be configured to select a prediction mode (for example, an intra or inter prediction mode) and / or a corresponding prediction block 245 or 255 to be used as the prediction block 265, for calculation of the residual block 205 and for reconstruction of the reconstructed block 215.

[0136] In one or more embodiments, the mode selection unit 262 may be configured to select the prediction mode (for example, from prediction modes supported by the prediction processing unit 260). The prediction mode provides an optimal match or a minimum residual (the minimum residual means better compression for transmission or storage), or provides minimum signaling overheads (the minimum signaling overheads mean better compression for transmission or storage), or considers or balances both. The mode selection unit 262 may be configured to determine the prediction mode based on rate-distortion optimization (RDO), that is, select a prediction mode that provides minimum rate-distortion optimization or select a prediction mode for which related rate distortion satisfies at least a prediction mode selection criterion.

[0137] In the following, prediction processing performed (for example, by using the prediction processing unit 260) and mode selection performed (for example, by using the mode selection unit 262) by an example of the encoder 20 are described in more detail.

[0138] As described above, the encoder 20 is configured to determine or select the optimal or optimum prediction mode from a set of (pre-determined) prediction modes. The set of prediction modes may include, for example, an intra prediction mode and / or an inter prediction mode.

[0139] In a possible implementation, a set of intra prediction modes may include a plurality of different intra prediction modes, for example, non-directional modes such as a direct current (DC) (or mean) mode and a planar mode, or directional modes as defined in H.265, or may include 67 different intra prediction modes, for example, non-directional modes such as a DC (or mean) mode and a planar mode, or directional modes as defined in developing H.266.

[0140] In a possible implementation, a set of inter prediction modes depends on available reference pictures (namely, for example, at least some decoded pictures stored in the DBP 230, as described above) and other inter prediction parameters, for example, depends on whether an entire reference picture or only a part of the reference picture, for example, a search window region around a region of the current block, is for searching for an optimal matching reference block, and / or for example, depends on whether pixel interpolation such as half-pel and / or quarter-pel interpolation is applied. The set of inter prediction modes may include, for example, an advanced motion vector prediction (AMVP) mode and a merge mode. In specific implementation, the set of inter prediction modes may include an affine motion model-based prediction mode described in some embodiments of the present disclosure, for example, an affine motion model-based AMVP mode (Affine AMVP mode) or an affine motion model-based merge mode (Affine merge mode), a control point-based AMVP mode (inherited control point motion vector prediction method or constructed control point motion vector prediction method) and a control point-based merge mode (inherited control point motion vector prediction method or constructed control point motion vector prediction method), and an advanced temporal motion vector prediction (ATMVP) method, a planar method, and the like, or a subblock-based merge mode formed by combining the affine motion model-based merge mode, the ATMVP and / or the planar method, or the like. In some embodiments of the present disclosure, inter prediction for the to-be-processed image block may be applied to unidirectional prediction (forward or backward), bidirectional prediction (forward and backward), or multi-frame prediction. When inter prediction is applied to bidirectional prediction, bidirectional prediction block level generalized bi-prediction (GBi), referred to as a weighted prediction method, may be used. In an example, the intra prediction unit 254 may be configured to execute any combination of inter prediction technologies described below.

[0141] In addition to the foregoing prediction modes, a skip mode and / or a direct mode may be also applied in the embodiments of the present disclosure.

[0142] The prediction processing unit 260 may be further configured to partition the image block 203 into smaller block partitions or subblocks, for example, by iteratively using quadtree (QT) partitioning, binary-tree (BT) partitioning, triple-tree (TT) partitioning, or any combination thereof, and perform, for example, prediction on each of the block partitions or subblocks. Mode selection includes selection of a tree structure of the partitioned image block 203 and selection of a prediction mode used for each of the block partitions or subblocks.

[0143] The inter prediction unit 244 may include a motion estimation (ME) unit (not shown in FIG. 2) and a motion compensation (MC) unit. The motion estimation unit is configured to receive or obtain the picture imageblock 203 (the current picture image block 203 of the current picture 201) and the decoded picture 231, or at least one or more previously reconstructed blocks, such as one or more reconstructed blocks of other / different previously decoded pictures 231, to perform motion estimation based on the determined inter prediction mode. For example, a video sequence may include the current picture and the previously decoded pictures 31, or in other words, the current picture and the previously decoded pictures 31 may be a part of or form a sequence of pictures forming a video sequence.

[0144] For example, the encoder 20 may be configured to select a reference block from a plurality of reference blocks of a same picture or different pictures in a plurality of other pictures (reference images), and to provide a reference picture and / or an offset (spatial offset) between a position of the reference block (X and Y coordinates) and a position of a current block to the motion estimation unit (not shown in FIG. 2) as an inter prediction parameter. This offset is also referred to as a motion vector (MV).

[0145] The motion compensation unit is configured to obtain the inter prediction parameter, and perform inter prediction based on or by using the inter prediction parameter, to obtain the inter prediction block 245. Motion compensation performed by the motion compensation unit may include taking out or generating a prediction block (predictor) based on a motion / block vector determined by using motion estimation (by possibly performing interpolation of sub-pixel accuracy). Interpolation filtering may generate additional pixel samples from known pixel samples. This potentially increases a quantity of candidate prediction blocks that may be used for encoding a picture block. Upon receiving a motion vector for a PU of the current picture block, the motion compensation unit 246 may locate a prediction block to which the motion vector points in one reference picture list. The motion compensation unit 246 may further generate a syntax element associated with a block and a video slice, so that the decoder 30 uses the syntax element to decode the picture block in the video slice.

[0146] Further, the inter prediction unit 244 may transmit the syntax element to the entropy encoding unit 270, where the syntax element, for example, includes an inter prediction parameter (for example, indication information of an inter prediction mode that is used for prediction of the current block and that is selected after a plurality of inter prediction modes are traversed), and an index number of a candidate motion vector list, and for example includes a reference frame index. In a possible application scenario, if only one inter prediction mode exists, the inter prediction parameter may be alternatively not carried in the syntax element. In this case, the decoder side 30 may perform decoding directly in a default prediction mode. It can be understood that the inter prediction unit 244 may be configured to perform any combination of inter prediction technologies.

[0147] The intra prediction unit 254 is configured to obtain, for example, receive, the picture block 203 (the current picture block) and one or more previously reconstructed blocks, for example, reconstructed neighbor blocks, of the same picture for intra estimation. The encoder 20 may be, for example configured to select an intra prediction mode from a plurality of (predetermined) intra prediction modes.

[0148] In one or more embodiments, the encoder 20 may be configured to select the intra prediction mode based on an optimization criterion, for example, based on a minimum residual (for example, an intra prediction mode providing the prediction block 255 that is most similar to the current picture block 203) or minimum rate distortion.

[0149] The intra prediction unit 254 is further configured to determine the intra prediction block 255 based on, for example, an intra prediction parameter of the selected intra prediction mode. In any case, after selecting an intra prediction mode for a block, the intra prediction unit 254 is further configured to provide the intra prediction parameter, that is, information indicating the selected intra prediction mode for the block, to the entropy encodingunit 270. In an example, the intra prediction unit 254 may be configured to perform any combination of intra prediction technologies.

[0150] Further, the intra prediction unit 254 may transmit the syntax element to the entropy encoding unit 270. The syntax element includes the intra prediction parameter (such as indication information of selection of an intra prediction mode used for prediction of the current block after traversal of a plurality of intra prediction modes). In a possible application scenario, if only one intra prediction mode exists, the intra prediction parameter may be alternatively not carried in the syntax element. In this case, the decoder side 30 may perform decoding directly in a default prediction mode.

[0151] The entropy encoding unit 270 is configured to apply an entropy encoding algorithm or scheme (for example, a variable length coding (VLC) scheme, a context adaptive VLC (CAVLC) scheme, an arithmetic coding scheme, context adaptive binary arithmetic coding (CABAC), syntax-based CABAC (SB AC), probability interval partitioning entropy (PIPE) coding, or another entropy encoding method or technology) to a single one or all (or none) of the quantized residual coefficients 209, the inter prediction parameter, the intra prediction parameter, and / or the loop filter parameter to obtain encoded picture data 21 that can be output by an output 272, for example, in a form of an encoded data 21. The encoded data may be transmitted to the decoder 30 or archived for later transmission or retrieval by the decoder 30. The entropy encoding unit 270 may be further configured to entropy encode another syntax element for a current video slice being encoded.

[0152] Another structural variation of the encoder 20 may be used to encode a video stream. For example, a non-transform based encoder 20 may quantize a residual signal directly without the transform processing unit 206 for some blocks or frames. In another implementation, the encoder 20 may have the quantization unit 208 and the inverse quantization unit 210 combined into a single unit.

[0153] It is understood that another structural change of the encoder 20 may be used to encode a video stream. For example, for some image blocks or image frames, the encoder 20 may directly quantize a residual signal without processing by the transform processing unit 206, and correspondingly without processing by the inverse transform processing unit 212. Alternatively, for some image blocks or image frames, the encoder 20 does not generate residual data, and correspondingly does not need processing by the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212. Alternatively, the encoder 20 may store a reconstructed image block directly as a reference block without processing by the filter 220. Alternatively, the quantization unit 208 and the inverse quantization unit 210 in the encoder 20 may be combined together. The loop filter 220 is optional, and in a case of lossless compression encoding, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, and the inverse transform processing unit 212 are optional. It is understood that in different application scenarios, the inter prediction unit 244 and the intra prediction unit 254 may be used selectively.

[0154] FIG. 3 is a schematic / conceptual block diagram of an example of a decoder 30 configured to implement one or more embodiments of the present disclosure. The decoder 30 is configured to receive, for example, encoded picture data (for example, the encoded data) 21 encoded by the encoder 20 to obtain a decoded picture 331. During decoding, the decoder 30 receives video data from the encoder 20, for example, an encoded video encoded data representing a picture block of an encoded video slice and an associated syntax element.

[0155] In the example of FIG. 3, the decoder 30 includes an entropy decoding unit 304, an inverse quantization unit 310, an inverse transform processing unit 312, a reconstruction unit 314 (for example, a summer 314), abuffer 316, a loop filter 320, a DPB 330, and a prediction processing unit 360. The prediction processing unit 360 may include an inter prediction unit 344, an intra prediction unit 354, and a mode selection unit 362. In some examples, the decoder 30 may perform a decoding process substantially inverse to an encoding process performed by the encoder 20 described above with reference to FIG. 2.

[0156] The entropy decoding unit 304 is configured to perform entropy decoding on the encoded picture data 21 to obtain, for example, quantized coefficients 309 and / or decoded coding parameters, for example, any one or all of an inter prediction parameter, an intra prediction parameter, a loop filter parameter, and / or another syntax element (decoded). The entropy decoding unit 304 is further configured to forward the inter prediction parameter, the intra prediction parameter, and / or the other syntax element to the prediction processing unit 360. The decoder 30 may receive a syntax element at a video slice level and / or a video block level.

[0157] The inverse quantization unit 310 may have a same function as the inverse quantization unit 210, the inverse transform processing unit 312 may have a same function as the inverse transform processing unit 212, the reconstruction unit 314 may have a same function as the reconstruction unit 214, the buffer 316 may have a same function as the buffer 216, the loop filter 320 may have a same function as the loop filter 220, and the DPB 330 may have a same function as the DPB 230.

[0158] The prediction processing unit 360 may include an inter prediction unit 344 and an intra prediction unit 354. The inter prediction unit 344 may resemble the inter prediction unit 244 in function, and the intra prediction unit 354 may resemble the intra prediction unit 254 in function. The prediction processing unit 360 is usually configured to perform block prediction and / or obtain a prediction block 365 from the encoded data 21, and (explicitly or implicitly) receive or obtain a prediction-related parameter and / or information about a selected prediction mode, for example, from the entropy decoding unit 304.

[0159] When the video slice is encoded into an intra encoded (I) slice, the intra prediction unit 354 of the prediction processing unit 360 is configured to generate the prediction block 365 for a picture block of a current video slice based on a signaled intra prediction mode and data that is from a previously decoded block of a current frame or picture. When the video frame is encoded into an inter encoded (that is, B or P) slice, the inter prediction unit 344 (for example, a motion compensation unit) of the prediction processing unit 360 is configured to generate the prediction block 365 for a video block of a current video slice based on a motion vector and another syntax element that is received from the entropy decoding unit 304. For inter prediction, the prediction block may be generated from one of reference pictures in a reference picture list. The decoder 30 may construct reference frame lists: a list 0 and a list 1, based on a reference picture stored in the DPB 330 by using a default construction technology.

[0160] The prediction processing unit 360 is configured to determine prediction information for a video block of the current video slice by parsing the motion vector and the other syntax element, and use the prediction information to generate the prediction block for the current video block being decoded. In one example of the present disclosure, the prediction processing unit 360 uses some of the received syntax elements to determine a prediction mode (for example, intra or inter prediction) for encoding video blocks of the video slice, an inter prediction slice type (for example, a B slice, a P slice, or a generalized P / B (GPB) slice), construction information for one or more of the reference picture lists for the slice, a motion vector for each inter-encoded video block of the slice, an inter prediction status for each inter-encoded video block of the slice, and other information, to decode the video blocks in the current video slice. In another example of this disclosure, a syntax element received by thedecoder 30 from encoded data includes a syntax element in one or more of an adaptive parameter set (APS), a sequence parameter set (SPS), a picture parameter set (PPS), or a slice header.

[0161] The inverse quantization unit 310 may be configured to inversely quantize (namely, dequantize) quantized transform coefficients provided in the encoded data and decoded by the entropy decoding unit 304. An inverse quantization process may include using a quantization parameter calculated by the encoder 20 for each video block in the video slice, to determine a degree of quantization to be applied and also determine a degree of inverse quantization to be applied.

[0162] The inverse transform processing unit 312 is configured to apply an inverse transform (for example, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to transform coefficients to generate residual blocks in a pixel domain.

[0163] The reconstruction unit 314 (for example, the summer 314) is configured to add an inverse transform block 313 (that is, a reconstructed residual block 313) to the prediction block 365 to obtain a reconstructed block 315 in a sample domain, for example, by adding sample values of the reconstructed residual block 313 and sample values of the prediction block 365.

[0164] The loop filter unit 320 (in a coding loop or after the coding loop) is configured to filter the reconstructed block 315 to obtain a filtered block 321, to smooth pixel transition or improve video quality. In an example, the loop filter unit 320 may be configured to perform any combination of filtering technologies described below. The loop filter unit 320 is intended to represent one or more loop filters including a de-blocking filter, a SAO filter, and another filter, for example, a bilateral filter, an ALF, a sharpening or smoothing filter, or a collaborative filter. Although the loop filter unit 320 is shown in FIG. 3 as an in-loop filter, the loop filter unit 320 may be implemented as a post-loop filter in other configurations.

[0165] The decoded video blocks 321 in a given frame or picture are then stored in the DPB 330 that stores reference pictures used for subsequent motion compensation.

[0166] The decoder 30 is configured to, for example, output the decoded picture 31 by using an output 332, for presentation to a user or viewing by a user.

[0167] Another variation of the decoder 30 may be used to decode compressed encoded data. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, a non-transform based decoder 30 may inversely quantize a residual signal directly without the inverse transform processing unit 312 for some blocks or frames. In another implementation, the decoder 30 may have the inverse quantization unit 310 and the inverse transform processing unit 312 that are combined into a single unit.

[0168] It is understood that, in the encoder 20 and the decoder 30 in the present disclosure, a processing result of a specific phase may be further processed and then output to the next phase. For example, after interpolation filtering, motion vector derivation or loop filtering, a further operation, such as clip or shift, may be performed on the processing result of a corresponding phase.

[0169] FIG. 4 is a schematic diagram of a structure of a video coding device 400 (for example, a video encoding device 400 or a video decoding device 400) according to one or more embodiments of the present disclosure. The video coding device 400 is applicable to implementation of one or more embodiments described in this specification. In some embodiments, the video coding device 400 may be a video decoder (for example, the decoder 30 in FIG. 1 A) or a video encoder (for example, the encoder 20 in FIG. 1 A). In other some embodiments,the video coding device 400 may be one or more components of the decoder 30 in FIG. 1A or the encoder 20 in FIG. 1A.

[0170] The video coding device 400 includes an ingress port 410 and a receiver unit (Rx) 420 for receiving data, a processor, a logic unit, or a central processing unit (CPU) 430 for processing the data, a transmitter unit (Tx) 440 and an egress port 450 for transmitting the data, and a memory 460 for storing the data. The video coding device 400 may further include an optical-to-electrical conversion component and an electrical-to-optical (EO) component that are coupled to the ingress port 410, the receiver unit 420, the transmitter unit 440, and the egress port 450, for egress or ingress of optical or electrical signals.

[0171] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (for example, multi-core processors), FPGAS, ASICs, and DSPs. The processor 430 communicates with the ingress port 410, the receiver unit 420, the transmitter unit 440, the egress port 450, and the memory 460. The processor 430 includes a coding module 470 (for example, an encoding module 470 or a decoding module 470). The encoding / decoding module 470 implements the embodiments disclosed in this specification, to implement methods provided in the embodiments of the present disclosure. For example, the encoding / decoding module 470 implements, processes, or provides various coding operations. Therefore, the encoding / decoding module 470 substantially improves functions of the video coding device 400 and affects transform of the video coding device 400 to a different state. Alternatively, the encoding / decoding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0172] The memory 460 includes one or more disks, tape drives, and solid-state drives (SSDs) and may be used as an overflow data storage device, to store programs when such programs are selected for execution, and to store instructions and data that are read during program execution. The memory 460 may be volatile and / or non-volatile, and may be a ROM, a RAM), a ternary content-addressable memory (TCAM), and / or a SRAM.

[0173] FIG. 5 is simplified block diagram of an apparatus 500 that can be used as any one or two of the source device 12 and the destination device M in FIG. 1A according to one or more example embodiments. The apparatus 500 may implement the technologies of the present disclosure. In other words, FIG. 5 is a schematic block diagram of one or more implementations of an encoding device or a decoding device (or a coding device 500) according to one or more embodiments of the present disclosure. The coding device 500 may include a processor 510, a memory 530, and a bus system 550. The processor and the memory are connected through the bus system. The memory is configured to store instructions. The processor is configured to execute the instructions stored in the memory. The memory of the coding device stores program code, and the processor may invoke the program code stored in the memory to execute various video encoding or decoding methods described in the present disclosure. To avoid repetition, details are not described herein again.

[0174] In some embodiments of the present disclosure, the processor 510 may be a CPU, or the processor 510 may be another general-purpose processor, a DSP, an ASIC, an FPGA or another programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or the like. The general-purpose processor may be a microprocessor, or the processor may be another processor or the like.

[0175] The memory 530 may include a ROM device or a RAM device. Any other proper type of storage device may also be used as the memory 530. The memory 530 may include code and data 531 accessed by the processor 510 by using a bus 550. The memory 530 may further include an operating system 533 and an application program 535, and the application program 535 includes at least one program that allows the processor 510 to execute thevideo encoding or decoding method described in the present disclosure. For example, the application program 535 may include applications 1 to N, which further include a video encoding or decoding application (referred to as a video coding application) performing the video encoding or decoding method described in the present disclosure.

[0176] In addition to a data bus, the bus system 550 may further include a power bus, a control bus, a status signal bus, and the like. However, for clear description, various types of buses in the figure are marked as the bus system 550.

[0177] For example, the coding device 500 may further include one or more output devices, for example, a display 570. In an example, the display 570 may be a touch display that combines a display and a touch unit that operably senses a touch input. The display 570 may be connected to the processor 510 through the bus 550.

[0178] Although the processor 510 and the memory 530 of the apparatus 500 are depicted in FIG. 5 as integrated into a single unit, another configuration may be used. Running of the processor 510 may be distributed in a plurality of machines that can be directly coupled (each machine has one or more processors), or in a local area or another network. The memory 530 may be distributed in a plurality of machines, such as network-based memories or memories in a plurality of machines running the apparatus 500. Although only a single bus is depicted herein, the bus 550 of the apparatus 500 may be formed by a plurality of buses. Further, the memory 530 may be directly coupled to another component of the apparatus 500 or may be accessed through a network, and may include a single integrated unit such as a storage card, or a plurality of units such as a plurality of storage cards. Therefore, the apparatus 500 may be implemented in a plurality of configurations.

[0179] In order to facilitate the understanding of solutions of embodiments the present disclosure, some terms mentioned herein are first introduced. Terms used in embodiments of the present disclosure are merely used to explain specific embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0180] 1. Transform

[0181] In video coding, the term “transform” may refer to an operation that is used to convert a spatial domain (or time domain) representation of video data into a frequency domain representation of the vide data, or into features / characteristics of the video data, where the frequency domain representation of the vide data or the features / characteristics of the video data, may be easier to analyze and process the video data such as compress the video data. The “transform” is designed to exploit statistical redundancies in video content more effectively, and thus effectively represent the video data and process such as compress the video data. This operation is a component of many video compression standards, for example, H.264 / AVC or H.265 / HEVC. The following are some transforms used in video coding.

[0182] DCT: as mentioned above, DCT is a transform used widely in video coding. DCT is effective for image or video compression because it has a strong energy compaction property, meaning that most of the image or video data may tend to be concentrated in a few low-frequency components and energy compaction may be produced for images or residuals (or differences) between images. DCT is used in standards such as JPEG for still images or H.264 / AVC for video.

[0183] DST: as mentioned above, DST is a transform sometimes used in video coding, particularly for some types of content or in specific contexts where it may outperform DCT. DST is useful for handling data with specific characteristics, such as symmetry properties that are better captured by sine functions.

[0184] Wavelet Transform: wavelet transform is used in some video coding applications for spatial scalability and to provide a multi-resolution analysis of the video data. Wavelet transform may decompose data or signal(s)using wavelet functions as a basis, allowing for the analysis of data or signal(s) in both time and frequency domains simultaneously. Wavelet transform is suitable for analyzing data or signal(s) with abrupt changes, such as images or audio data (or signal(s)).

[0185] Lapped Transform: lapped transform aims at reducing or even eliminating block artifacts by the use of overlapping basis functions, which may extend over more than one block.

[0186] Trainable Transform and machine learning based (ML-based) method: trainable transformation & machine learning based (ML-based) method may refer to learning the representation of video or image data through machine learning. This transform aims to use a data-driven approach to optimize transform parameters to better fit a specific task

[0187] Lazy transform: lazy transform refers to a technique where the transform stage of the encoding process is skipped or bypassed for certain blocks or regions of the video frame. The term lazy transform is also known as identity transform or bypass transform.

[0188] 2. Quantization

[0189] Quantization is a technology used in video encoding to reduce the amount of data that needs to be stored or transmitted. Quantization may work by approximating a range of values into a smaller set of values. In essence, quantization simplifies the data by reducing the precision of the data. When a video is encoded, the video is broken down into smaller blocks of pixels. Quantization may reduce the precision of the pixel values within these blocks. By performing quantization, the encoder can eliminate some of the less important data, leading to a smaller file size.

[0190] 3. Multiple Transforms

[0191] Different types of content (e.g., natural scenes, text, or graphics) of videos or images may have varying frequency characteristics. A single transform may not be optimal for all types of content. In some solutions for image or video compression, multiple transforms may be used to improve compression efficiency and quality. The concept of using multiple transforms in codecs has evolved over several generations of video compression standards. The multiple transforms may be included in standards such as H.261, MPEG-1, MPEG-2, and H.264. For example, the multiple transforms may include a DCT and a rotation transform (ROT). DCT coefficients may be obtained by performing the DCT on the image or video, and the ROT may be applied on top of the DCT coefficients and the ROT may be used to rotate some of the DCT coefficients. Multiple transforms may leverage strengths of different transforms to better handle various types of content and signal characteristics. Moreover, multiple transforms may optimize the representation of different frequency components of the image or video, thereby reducing the amount of bit overhead required to represent the original content with minimal loss of quality.

[0192] 4. Transform-Skip Mode (TSM)

[0193] TSM is a coding technology used in some video coding standards such as H.265 / HEVC or H.266 / VVC to improve coding efficiency, especially for screen content. This mode allows the encoder to bypass some transform steps (e.g., DCT) for some blocks, which may be beneficial for content with high-frequency components like text and graphics.

[0194] 5. Transform Quantizer Bypass (TQB)

[0195] TQB allows the encoder to bypass the transform and quantization steps, directly encoding the prediction residuals. This can lead to reductions in computational complexity and latency. TQB may be applied for high-throughput and low-latency applications, such as real-time video streaming and video conferencing.

[0196] The above describes transforms, multiple transforms, and skipping or bypassing on the transform, and some solutions proposed for selection or signaling for the multiple transforms will be described below.

[0197] Solution 1 : Transform Skip Mode in HE VC

[0198] According to HEVC specification, there is transform_skip_flag syntax element, which specifies whether a transform is applied to the associated transform block or not. According to the specification (see 7.4.9.11, Residual coding semantics), transform_skip_flag [ xO ][ yO ][ cldx ] is provided, where transform_skip_flag [ xO ][ yO ][ cldx ] equal to 1 specifies that no transform is applied to the current transform block; transform_skip_flag [ xO ][ yO ][ cldx ] equal to 0 specifies that the decision whether transform is applied to the current transform block or not depends on other syntax elements; and when transform_skip_flag[ xO ][ yO ][ cldx ] is not present, it is inferred to be equal to 0. The array indices xO, yO specify the location (xO, yO) of the top-left luma sample of the considered transform block relative to the top-left luma sample of the picture. The array index cldx specifies an indicator for the colour component; it is equal to 0 for luma, equal to 1 for Cb and equal to 2 for Cr.

[0199] According to the syntax table, in particular chapter 7.3.8.11 (Residual coding syntax), TU includes residual coding part, which includes transform_skip_flag. As described in chapter 7.3.8.11, residual coding is applied for all color components including luma and chroma, which means transform_skip_flag is signaled always, under specific conditions: if(transform_skip_enabled_flag && !cu_transquant_bypass_flag && (log2TrafoSize <= Log2MaxTransformSkipSize)). That is, this syntax element is signaled for all transform-sizes less then predefined size.

[0200] Solution 2: Multiple Transform Selection (MTS) in VVC

[0201] VVC uses large block-size transforms, up to 64x64, which are beneficial for high-resolution videos like 1080p and 4K. For blocks sized 64 in width or height, high-frequency coefficients are zeroed out, retaining only the lower-frequency ones. For instance, in an MxN block, if M is 64, only the left 32 columns of coefficients are kept; if N is 64, only the top 32 rows are retained. In transform skip mode, the entire block is used without zeroing any values, and transform shift is disabled. The VVC test model (VTM) allows configurable maximum transform sizes in SPS, giving encoders the flexibility to choose between 32-length or 64-length transforms based on specific needs.

[0202] In addition to DCT type II which has been employed in HEVC, an MTS scheme is used for residual coding both inter and intra coded blocks. The MTS scheme may select transforms from different transform types including DCT-II, DCT -VIII, and DST-VII. Table 1 illustrates basis function for different transform types for N-point input, where i represents one of transform functions, j represent coefficient of z'-th transform function, and N represents number of coefficients.Table 1: Basis Function for Different Transform Types for N-Point Input Transform Type Basis function 7, J = 0, 1,.... N- 1DCT-II / 7r ■ (21 + 1) ■ (2 / + 1) DCT-Vin ■ COS -I 4 (x ■ (2i + 1) ■ O' + 1)' DST-VII I - sin -27V + 1 \ 27V + 1

[0203] Table 2 illustrates 5 possible combinations of multiple transforms adopted in the VVC standard.Table 2: 5 Possible Combinations of Multiple TransformsMTS index Horizontal Verticaltransform transform0 DCT2 DCT21 DST-7 DST-72 DCT-8 DST-73 DST-7 DCT-84 DCT-8 DCT-8

[0204] When MTS is enabled at SPS, a CU level index is signaled to indicate which transform matrix is used for the horizontal and the vertical transform. The MTS signaling is skipped when one of the conditions is applied: one condition is that DC-only non-zero coefficient in a block, and another condition is that the last significant coefficient of the luma TB is located inside the MTS zero-out region.

[0205] Similar to HEVC, residuals can be coded with transform-skip mode. To avoid the redundancy of syntax coding, the transform skip flag is not signaled when the CU level MTS CU flag is not equal to zero. Implicit MTS transform is set to DCT2 dependent on other syntax elements (e.g. low frequencies non-separable transform (LFNST) or matrix weighted intra prediction - MIP are used for the current CU). In addition, there is an implicit MTS derivation which is used if MTS is enabled for inter coded blocks.

[0206]

[0207]

[0208]

[0209]

[0210] Solution 3: Signaling in Transform Skip Mode

[0211] Solution 3 introduces a dependency between a syntax element indicating whether a transform skip mode is applied to a transform block and another syntax element indicating if a pulse-code modulation is used. This solution helps improve encoding efficiency of transform skip syntax element by taking use of the dependency.

[0212] As described above, in current standards such as HEVC or VVC, spatial transform(s) may be provided and an appropriate spatial transform may be adaptively selected for a given image or video, where transform for different image blocks such as TUs may or may not be the same. In such case, some syntax elements may be used for signaling the selected transform(s) for the image blocks. However, in such case, some syntax elements are signaled per image block, resulting in high signaling overhead and low encoding efficiency.

[0213] In various aspects and / or implementations of the present disclosure, solutions are proposed to solve at least one of the above problems. In some implementations of the present disclosure, a syntax element may be used for signaling a transform type of an image block group. The Image block group may include multiple image blocks, and each image block is a basic unit for processing and is a part of an image. Compared with the solutions where per image block signaling is used, signaling overhead may be reduced and encoding efficiency may be improved.

[0214] In contrast to some approaches that includes a wavelet-base codec considered as a baseline, in some embodiments, at least another transform, applied on top of the wavelet transform, is adaptively enabled or disabled.

[0215] In some embodiments, the proposed method is applied to only some of color components, for example, but not limited to luma.

[0216] In some embodiments, quantization level for coefficients is determined on the transform type.

[0217] In some embodiments, if coefficients of a block are quantized to zero, the transform type of the block may not be signaled, and it may be inferred that the corresponding transform is skipped.

[0218] In some embodiments, if several levels of hierarchy of said signaling is used, dependencies in this hierarchy are used to adjust probabilities of specific signaling values on lower levels or higher levels of said hierarchy which is further used by an entropy encoding engine to effectively encode the syntax elements.

[0219] Reference is now made to FIG. 6, which illustrates an example flowchart of an encoding method in accordance with some implementations. The method 600 may be performed by an encoder or a module, a circuit or a chip in the encoder, or a logical module or software that can implement all or some functions of the encoder. For example, the method 600 is performed by the encoder and includes the following steps. The encoder may be any encoder described in FIG. 1A, FIG. IB, FIG. 2, FIG. 4, or FIG. 5.

[0220] Step 601 : the encoder obtains transform coefficients of blocks in each block group of N block groups by performing a transform on blocks in a respective block group, where each block group in the N block groups includes M blocks, and N and M are positive integers.

[0221] In some implementations, each block may include a video (or image) block or a prediction video (or image) block. The video block and image block may be interchangeable in the present disclosure. The encoder may obtain or receive an input video frame. The encoder may obtain a plurality of video blocks by partitioningthe input video frame into the plurality of video blocks. The encoder may perform the prediction on the plurality of video blocks to obtain the plurality of prediction video blocks.

[0222] The input video frame may include samples of at least one channel, such as luma samples or chroma samples. Accordingly, each block also includes luma samples or chroma samples. That is, the transform performed in step 601 may be applicable for channel(s). In one or more implementations, the transform may be applicable for luma channel only. For example, the transform-skip mode may be applied for luma channel only. In another implementation, the transform may be applicable for chroma only. The channel(s) may be preconfigured. Alternatively, the channel(s) may be dynamically selected and identifier of the channel(s) may be signaled in the bitstream.

[0223] The input video frame may be a two-dimension (2D) frame, so that the plurality of blocks may be distributed in a 2D space corresponding to the input video frame. The plurality of blocks distributed in the 2D space may be grouped in a certain order. In an example, the order may be a line-scan order (e.g. from left-to-right, or from top to bottom). In another example, the order may be an order which traverses the 2D space according to a predetermined pattern (e.g. Z-order or nested Z-order) according to a packing structure of blocks included in the input 2D frame. In one or more implementations, the order is predetermined. In another implementation, the order may be determined adaptively according to elements (e.g. blocks, partition information) that are already encoded. The information about the order may be signaled in the bitstream.

[0224] The plurality of blocks may be tightly packed to each other and may not overlapped with each other. The plurality of blocks may be grouped equally or unequally, that is, the number of blocks in each block group may be the same or different. Each block group may include multiple blocks.

[0225] The blocks may be of same size or different sizes. In a case where the blocks are of different sizes, the block sizes may be selected adaptively, and information about the block sizes may be signaled in the bitstream.

[0226] In one or more implementations, the encoder may perform multiple transforms. The multiple transforms may include a first-level transform and a second-level transform, and the first-level transform is performed before the second-level transform. It is noted that for the video block or the prediction video block, the transform on step 601 is the first-level transform, or the seco nd- level transform, or the first-level transform and the second second-level transform.

[0227] In an example, when the transform on step 601 is the first-level transform, each block may be performed with anther transform after step 601. Alternatively, when the transform on step 601 is the first-level transform, there is no second-level transform.

[0228] In another example, when the transform on step 601 are the first-level transform and the second second-level transform, step 601 may be performed twice.

[0229] In still another example, when the transform on step 601 is the second- level transform, each block may be performed with anther transform before step 601. That is, the method further includes that: the encoder may perform a second transform on blocks in each block group of the N block groups. In this case, the first-level transform may include the second transform and the first-level transform may include the transform on step 601. The encoder may perform the second transform on blocks in each block group of the N block groups, and obtain the transform coefficients of the blocks in each block group of N block groups where the blocks in each block group of N block groups have been performed with the second transform.

[0230] In some implementations, the second transform includes a wavelet transform. When the second transform includes the wavelet transform, the wavelet transform may be performed on the input frame directly and then the transformed input frame may be partitioned to obtain the plurality of blocks (or prediction blocks).

[0231] FIG. 7 illustrates an example of wavelet transform. After performing the wavelet transform, a video frame is decomposed into different frequency components including a low-low (LL) subband which is the upper left part showing a cat, a high-low (HL) subband which is marked with "Vertical", a low-high (LH) subband which is marked with "Horizontal", and a high-high (HH) subband which is marked with "Diagonal".

[0232] The LL subband includes the low-frequency information of the image, which represents the main trends and overall structure. It is obtained by applying low-pass filters both horizontally and vertically. The HL subband subband includes the high-frequency information in the horizontal direction while retaining low-frequency information in the vertical direction. It is achieved by applying a high-pass filter horizontally and a low-pass filter vertically. The LH subband includes the high-frequency information in the vertical direction while retaining low-frequency information in the horizontal direction. It is achieved by applying a high-pass filter vertically and a low-pass filter horizontally. The HH subband includes high-frequency information in both the horizontal and vertical directions. It is obtained by applying high-pass filters both horizontally and vertically.

[0233] FIG. 8 illustrates an example of two-level dyadic wavelet transform. In this example, the wavelet transform is performed recursively. The two-level dyadic wavelet transform includes a level 0 wavelet transform and a level 1 wavelet transform. Level 0 wavelet transform is performed on the video frame. By performing level 0 wavelet transform, the video frame is decomposed into four frequency components including a LL subband, a HL subband, a LH subband, and a HH subband. Level 1 wavelet transform is then performed on the LL subband which is obtained by performing the level 0 wavelet transform. As such, the LL subband is further decomposed into four frequency components including a LL subband, a HL subband, a LH subband, and a HH subband.

[0234] In can be understood that the wavelet transform can be performed recursively for several times, and the times is not limited herein.

[0235] FIG. 9 illustrates annother example of two-level dyadic wavelet transform according to some implementations. Similar to FIG. 8, by performing level 0 wavelet transform, the video frame is decomposed into four frequency components including a LL subband, a HL subband, a LH subband, and a HH subband, and by performing level 1 wavelet transform on the LL subband which is obtained by performing the level 0 wavelet transform, this LL subband is further decomposed into four frequency components including a LL subband, a HL subband, a LH subband, and a HH subband. Furthermore, as shown in FIG. 9, the HL subband obtained by performing level 0 wavelet transform is further divided into several non-overlapping blocks. It is noted that other subbands obtained by wavelet transform of any level may also be divided into blocks.

[0236] The wavelet transform may be 2D wavelet transform such as separable 2D wavelet transform or non-separable 2D wavelet transform. In one or more implementations, the wavelet transform may be applied recursively. In this case, a wavelet transform may be applied to low-pass part of a result obtained by previous wavelet transform.

[0237] For example, the first-level transform is the second transform that is the wavelet transform, and the second-level transform is Hadamard transform. In this example, the input frame may be firstly partitioned into sub-parts such as sub-pictures. The encoder may obtain transformed sub-pictures by performing the wavelet transform on the sub-pictures. The transformed sub-pictures may also be referred to as transformed blocks. Thetransformed sub-pictures may be further partitioned into blocks and processed by a Hadamard transform. The first syntax element may indicate whether transform coefficients of all blocks in a respective block group are obtained by the Hadamard transform.

[0238] In some cases, if Hadamard transform is applied as the second- level transform unconditionally, it may not be efficient in terms of compression due-to lack of adaptation. In view of this, the encoder may determine whether the Hadamard transform will be performed. In this case, the first syntax element may indicate whether the Hadamard transform is performed on a respective block group.

[0239] It is understood that the second-level may be performed on part or all of the transformed block obtained by the first-level transform. Moreover, in addition to the first-level transform and the second-level transform, other transforms may be performed by the encoder.

[0240] FIG. 10 illustrates example transform types of different cases. In the example shown in FIG. 10, four blocks are grouped together, forming a block group. Blocks marked with grids are blocks on which the encoder performs wavelet transform, Hadamard transform, and quantization!. Blocks marked with dots are blocks on which the encoder performs wavelet transform and quantization2. Blocks marked with stripes are blocks whose coefficients are all zero.

[0241] Case 1 - four consecutive blocks are grouped for the purpose of signaling of same transform-type.

[0242] For example, in block group 1, four consecutive blocks are of the same transform-type that is a combination of wavelet transform and Hadamard transform. In block group 2, four consecutive blocks are of the same transform-type that is a wavelet transform.

[0243] In Case 1, 1 bit per four blocks may indicate whether the four blocks have the certain transform type. Alternatively, 1 bit per four blocks may indicate whether the four blocks have the same transform type, and another syntax element may indicate the specific type of the same transform type. In Case 1, 1 bit per 4 blocks may be supplemented with an additional bit indicating whether all 8 blocks in group 1 and group 2 have same transformtype or not.

[0244] Case 2 - four consecutive blocks do not have the same transform-type, which is shown in group 3. In case 2, per-block syntax element may be used to indicate transform type of a respective block.

[0245] Case 3 - coefficients of four consecutive blocks are all zero, which is shown in group 4. In this case, there may not be a syntax element signaling the transform type.

[0246] In some cases, multiple transforms including the wavelet transform and the Hadamard transform may work well in some color channels while may not work well on other color channels. In an example, after performing the wavelet transform on an image or part of an image, the encoder may decide and indicate whether it will further perform Hadamard transform on certain color channel such as luma. In this case, the encoder may not signal whether it will further perform Hadamard transform on certain color channel such as chroma, thereby reducing bit overhead.

[0247] Step 602: the encoder determines N first syntax elements, where each first syntax element in the N first syntax elements indicates whether transform coefficients of all blocks in a respective block group of N block groups are obtained by a first transform.

[0248] In some implementations, the first transform may include a spatial transform such as DCT, DST, Hadamard transform, wavelet transform (i.e. wavelet decomposition), lapped transforms, trainable transforms, or identity transform.

[0249] The spatial transform may be applied firstly horizontally and then vertically, or firstly applied vertically and then horizontally on the block. Alternatively, the first transform may include transform-skip which means that the transform is skipped on the block and the block is not transformed. The transform-skip may also be considered as an identity transform (i.e., lazy transform) which bypasses the input signal through without changing the input signal.

[0250] The first syntax element may be considered as a group-level syntax element because it indicates information for a block group. The first syntax element may be a 1 -bit indication. For example, the first syntax element has a value of 0 may specify that the first transform is performed on all blocks in a respective block group; while the first syntax element has a value of 1 may specify that the first transform is performed on not all blocks in a respective block group (for example, all of parts of blocks in the respective block group is performed with the first transform). Alternatively, the first syntax element has a value of 1 may specify that the first transform is performed on all blocks in a respective block group; while the first syntax element has a value of 0 specifies that the first transform is performed not on all blocks in a respective block group.

[0251] Step 603: the encoder encodes the N first syntax elements.

[0252] After step 602, the encoder may encode the N first syntax elements into a bitstream.

[0253] Compared with some solutions in which per-block signaling is used to indicate transform type of each block, in these implementations, a group-based signaling is used to indicate transform type of a group of blocks, thereby reducing bit overhead for signaling the transform type and not sacrificing image quality.

[0254] In some implementations, step 601 may include that: the encoder obtains transform coefficients of all blocks in a first block group of the N block groups by performing the first transform on all the blocks in the first block group. In this case, one first syntax element in the N first syntax elements indicates that transform coefficients of all the blocks in the first block group of the N block groups are obtained by the first transform.

[0255] The encoder may determine to perform the first transform on all the blocks in the first block group. In this case, the one first syntax element in the N first syntax elements may indicate that transform coefficients of all the blocks in the first block group of the N block groups are obtained by the first transform. If all blocks in a block group are of the same transform type, the transform type may be signaled by using only one syntax element.

[0256] The encoder may perform transform of a same type on a group of blocks. For example, the encoder may perform the first transform on the group of blocks, otherwise, the encoder may perform transform-skip on the group of blocks. In this example, the transform type of the group of blocks may be signaled by using only one syntax element.

[0257] In some implementations, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method 600 further includes that: the encoder determines per-subgroup syntax elements, where each per-subgroup syntax element indicates whether transform coefficients of all blocks in a respective subgroup in the second block group are obtained by the first transform, and the encoder encodes the per-subgroup syntax elements.

[0258] A subgroup may be a subset of the group and include one or more blocks. A block group may include multiple subgroups. Different subgroups may be of the same size. In other words, different subgroups may include a same quantity of blocks. For example, a block group include 4 blocks, and the block group may be divided into two subgroups, where each subgroup includes two blocks.

[0259] In this case, hierarchical signaling of grouped transform-type selection is applied. That is, one syntax element may be used to indicate whether a group of blocks are of the same transform-type. If the group of blocks are not of the same transform type, fine granularity syntax elements may be used to indicate transform type for a smaller group of blocks. If the smaller group of blocks are not of the same transform type, the further fine granularity syntax elements may be used to indicate transform type for a further smaller group of blocks until transform types for all blocks are clear. Or, if the smaller group of blocks are not of the same transform type, reference may be made as the following description related to the second block group, where the block group may be replaced with the block subgroup.

[0260] In some implementations, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method 600 further includes that: the encoder determines M per-block syntax elements, where each per-block syntax element of the M per-block syntax element indicates whether transform coefficients of a respective block in the second block group are obtained by the first transform, and the encoder encodes the M per-block syntax elements.

[0261] In this case, hierarchical signaling of grouped transform-type selection is also applied. The fine granularity syntax elements are per-block syntax elements. As such, the encoder may inform the decoder of the transform type for each block.

[0262] In some other implementations, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further includes that: the encoder determines M per-block syntax elements, where each per-block syntax element of the M per-block syntax elements indicates a transform of a respective block in the second block group, and the encoder encodes the M per-block syntax elements.

[0263] In this case, each per-block syntax element may indicate a specific transform type of a respective block instead of indicating whether a transform of a block is the first transform.

[0264] It is noted that, the first block group is before the second block group or the first block group is after the second block group.

[0265] In some implementations, the first block group is before the second block group. For example, the first block group is previously adjacent to the second block group. In this case, a first block in the second block group is adjacent to the first block group, and the encoding the M per-block syntax elements may include: encoding a per-block syntax element of the first block based on a probability model of the first block, where the probability model is determined based on transform of the first block group.

[0266] In some cases, it may be of a high probability that a transform type of a block may be the same as a transform type of a neighboring block. In these cases, group-level syntax element(s) available near to the current group-level syntax element may be taken into account to estimate probability of having specific transform-type(s) within a current group of blocks.

[0267] Since the first block in the second block group is adjacent to the first block group, it may be of a high probability that a transform type of the first block is the same as the transform type of the first block group. In this case, the encoder may encode the per-block syntax element of the first block using probability model of the first block, which may help to improve compression efficiency.

[0268] FIG. 11 illustrates an example hierarchical signaling scheme according to some implementations. FIG.11 shows four block groups, each block group including four blocks. Blocks marked with grids are blocks on which the encoder performs the first transform, and blocks marked with dots are blocks on which the encoder performs transform-skip. Syntax elements VO, VI, V2, and V3 each indicate whether the encoder performs the first transform on all blocks in a respective block group.

[0269] V0=0 indicates that the encoder performs the first transform on all blocks in block group A.

[0270] Vl=l indicates that the encoder performs the first transform not on all blocks in block group B, for example, block 5 and block 8 are performed with the first transform, but block 6 and block 7 are not performed with the first transform.

[0271] V2=0 indicates that the encoder performs the first transform on all blocks in block group C.

[0272] V3=l indicates that the encoder performs the first transform not on all blocks in block group D, for example, block 13 is performed with the first transform, but blocks 14-16 are not performed with the first transform.

[0273] For block group B and block group D, per-block syntax elements may be used to indicate whether the encoder performs the first transform on a respective block.

[0274] Since block 5 is adjacent to block group A in which all the blocks have the same transform type, block 5 is likely to be of the same transform-type as the block group A. In this case, the encoder may encode the per-block syntax element of the block 5 using a probability model of the block group A, which may help to improve compression efficiency.

[0275] Similarly, since block 8 is adjacent to block group C in which all the blocks have the same transform type, block 8 is likely to be of the same transform-type as the block group C. In this case, the encoder may encode the per-block syntax element of the block 8 using a probability model of the block group C.

[0276] In addition, the encoder may encode the per-block syntax element of the block 13 using a probability model of the block group C.

[0277] FIG. 12 illustrates another example of group signaling:

[0278] In this case, a syntax element bl indicates the transform type (e.g. transform-skip) of blocks 1~8. If blocks 1~8 are all of the same type, bl may be enough to indicate the transform type of blocks 1~8 and finer signaling may not be needed. Similarly, a syntax element b2 indicates the transform type of blocks 9-16. Moreover, a syntax element b3 indicates the transform type of blocks 5-12 which have overlapped region with blocks indicated by bl and b2. Similarly, a syntax element b4 indicates the transform type of blocks 13—16 and 1-4 which have overlapped region with blocks indicated by bl and b2. If bl & b2 are all homogenous (e.g. blocks 1-8 are of the same transform type, and blocks 9-16 are of the same transform type), b3 and b4 may not be signaled. This cascaded scheme allows to compactly transmit transform-type syntax elements for multiple blocks.

[0279] In some implementations, the method 600 further includes that: the encoder determines a second syntax element, where the second syntax element indicates the first transform, and the encoder encodes the second syntax element.

[0280] In this case, the second syntax element may indicate the first transform, that is, the second syntax element may indicate the type of the first transform. In this way, the encoder may inform the decoder of the specific type of the first transform.

[0281] In some implementations, in a case where all coefficients (or pixels depending on a structure of a video frame) of a group of blocks are zeros, group signaling of a transform type may not be encoded into the bitstream. In this case, a transform type is inferred a predefined type. The predefined type may be transform-skip or other transform types.

[0282] In the case where the blocks are video blocks, the method 600 may further include: the encoder obtains quantized coefficients by performing a quantization on the transform coefficients of each block group, where quantization is determined based on the transform.

[0283] In this case, a quantization strength may depend on a transform type, which means each transform type may correspond to a certain quantization strength. In addition, quantization process of quantization with different quantization strength may be different. As such, an appropriate quantization may be selected for a given block according to the transform type of the given block, thereby improving performance of the quantization and enhancing the flexibility of encoding.

[0284] In one or more implementations, a quantization factor may include a scaling factor of a transform to implement efficient and low-complexity transformation and quantization. In an example, the transformation and quantization can be considered as a fuse operation of two steps including quantization and transform-scaling. In this example, the encoder may dynamically determine the quantization according to the transform type. In this case, the quantization may be completely or partially fused with a transform, thereby reducing the computation complexity.

[0285] In some cases, characteristics of the content of video blocks being encoded may be different. In such case, transform and quantization for different video blocks may vary. By selecting an appropriate transform and quantization for each video block, codecs may achieve better compression ratios and maintain higher visual quality. Adaptive algorithms may be used to select transform and quantization based on the characteristics of the content of each video block being encoded.

[0286] FIG. 13 illustrates an example schematic of an encoding process according to some implementations. On the left side of FIG. 13, there is a thumbnail of a video frame, indicating the starting point of the encoding process. The video frame may be divided into multiple video blocks (not shown in FIG. 13). In the central part of FIG. 13, a series of boxes labeled “Transform l + QI”, “Transform_2 + Q2”, and so on, up to “Transform n + Qn” are different transforms and quantizations that may be selected for each video block, where QI, Q2, ..., Qn represent the transform and Transform l, Transform_2, ..., Transform n represent the quantization. Rate-distortion optimization (RDO) is used to select the transform and quantization for each video block. RDO may help to find the balance between encoding efficiency and video quality. The encoding engine performs encoding and input a bitstream.

[0287] In this example, quantization may be determined based on the transform. For example, QI is determined based on Transform l, Q2 is determined based on Transform_2, ..., and Qn is determined based on Transform n.

[0288] In some implementations, the method 600 may ftirther include: the encoder obtains quantized coefficients by performing a quantization on the transform coefficients of each block group, where the quantization is determined based on one or more of: the transform or the second transform.

[0289] In this case, a quantization strength may depend on a transform type, which means each transform type may correspond to a certain quantization strength. In addition, quantization process of quantization with different quantization strength may be different. As such, an appropriate quantization may be selected for a given blockaccording to the transform type of the given block, thereby improving performance of the quantization and enhancing the flexibility of encoding.

[0290] FIG. 14 illustrates another example schematic of an encoding process according to some implementations. The example shown in FIG. 14 is similar to that shown in FIG. 13, and the difference is that multiple transforms are involved. The multiple transforms may be considered as forming a new transform that is the combination of the multiple transforms. In this example, quantization may be determined based on the transform. For example, QI is determined based on Transform l, Q2 is determined based on Transform_2, Q3 is determined based on both Transform l and Transform_2,..., and Qn is determined based on Transform n.

[0291] The above describes operations of the encoder and the following describes operations of the decoder.

[0292] In some implementation, the decoder may start with an input bitstream and form decoded information or decoded frame as an output. The input bitstream may be generated by the encoder based on the method 600.

[0293] Reference is now made to FIG. 15, which illustrates an example flowchart of a decoding method in accordance with some implementations. The method 1500 may be performed by a decoder or a module, a circuit or a chip in the decoder, or a logical module or software that can implement all or some functions of the decoder. For example, the method 1500 is performed by the decoder and includes the following steps. The decoder may be any decoder described in FIG. 1A, FIG. IB, FIG. 3, FIG. 4, or FIG. 5.

[0294] Step 1501 : the decoder obtains N first syntax elements and quantized coefficients of each block group of N block groups based on a bitstream, where each first syntax element in the N first syntax elements indicates whether transform coefficients of all blocks in a respective block group of the N block groups are obtained by a first transform, each block group of the N block groups includes M blocks, and N and M are positive integers.

[0295] Step 1502: the decoder obtains dequantized coefficients of each block group by performing an inverse quantization on the quantized coefficients of each block group.

[0296] Step 1503: the decoder obtains M inverse transform blocks for each block group by performing an inverse transform on the dequantized coefficients of each block group, where the inverse transform is determined based on a respective first syntax element in the N first syntax elements.

[0297] Step 1504: the decoder reconstructs an image based on the M inverse transform blocks for each block group.

[0298] Compared with some solutions in which per-block signaling is used to indicate transform type of each block, in these implementations, a group-based signaling is used to indicate transform type of a group of blocks, thereby reducing bit overhead for signaling the transform type and not sacrificing image quality.

[0299] In some implementations, the blocks include image blocks. The decoder may obtain the block group by scanning or placing the blocks in a certain order as the encoder. In an implementation, the order may be predetermined. In another implementation, the order may be determined adaptively by the encoder and signaled in the bitstream. In the case where the order is determined adaptively, the decoder may parse and decode a syntax element indicating the order from the bitstream.

[0300] The decoder may obtain the first syntax elements and quantized coefficients by parsing process. Input of the parsing process may be bits from a bitstream which may be a compressed bitstream representing compressed frame(s). Output of the parsing process may include syntax elements such as the first syntax elements or other syntax elements indicating other information about a frame. The parsing process may be implemented according a certain syntax structure. The syntax structure refers to a specified order of zero or more syntax elements that arepresented in. A decoding process including the inverse quantization and the inverse transform may further be performed to reconstruct pixel data. Output of the decoding process may include one or more decoded frames or sub-frames. It is noted that the generation or the details of the first syntax elements may refer to the related description at the encoder.

[0301] The parsing process may be performed according to a syntax table that may specify the structure and format of the encoded video bitstream. The syntax table may specify how the video data are organized and how different elements within the bitstream are interpreted by the decoder. For example, the syntax table reflects syntax element parsing as well as statements, loops, invocations of functions, conditions. In addition to the syntax table, semantics defines the scope, restrictions and conditions that are imposed on the syntax elements, ensuring that the parsed data are valid and correctly interpreted.

[0302] In the parsing process, the decoder parses syntax elements from the compressed bitstream. In an implementation, the syntax elements may include information related to partitioning of a frame into blocks. In another implementation, the syntax elements may include information regarding values of compressed coefficients. For example, the syntax elements may include significance flags indicating whether a group of coefficients are all zeros. The group of coefficients may be coefficients of a group of blocks.

[0303] In a case where there is at least one non-zero coefficient in a group of coefficients, the decoder may parse respective syntax elements including the first syntax elements. The decoder may traverse blocks one by one or according to certain order, and the first syntax elements may be parsed and decoded for every group.

[0304] The parsing and decoding order of the first syntax elements and the significance flags and may vary in different cases. In an implementation, the significance flags are parsed and decoded prior to the first syntax elements. In another implementation, the first syntax elements are parsed and decoded prior to the significance flags. In yet another implementation, the first syntax elements and the significance flags are parsed and decoded simultaneously.

[0305] After parsing and decoding the first syntax elements, in step 1503, the decoder may perform an inverse transform on the blocks according to the first syntax elements.

[0306] In some implementations, one first syntax element in the N first syntax elements indicates that transform coefficients of all blocks in a first block group of the N block groups are obtained by the first transform, and the performing an inverse transform on the dequantized coefficients of each block group includes: obtaining an inverse transform block of each block in the first block group, by performing an inverse first transform on dequantized coefficients of a respective block in the first block group.

[0307] In this case, all blocks in the first block group are of the same transform type which is the first transform type. The decoder may perform the inverse transform on each block in the first block group, thereby obtaining inverse transform coefficients for each block in the first block group.

[0308] In some implementations, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further includes: obtaining M per-subgroup syntax elements based on the bitstream, where each per-subgroup syntax element of the M per-subgroup syntax elements indicates whether transform coefficients of a respective subgroup in the second block group are obtained by the first transform. In this case, step 1503 includes: obtaining an inverse transform block of each block in the second block group based on the M per-subgroup syntax elements.

[0309] In this case where blocks in the same block group are not of the same transform type, fine granularity syntax elements are further parsed and decoded for smaller group of blocks, and each smaller group of blocks are subset of an original group of blocks.

[0310] In some implementations, the subgroup may include one block. In this case, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further includes: obtaining M per-block syntax elements based on the bitstream, where each per-block syntax element of the M per-block syntax elements indicates whether transform coefficients of a respective block in the second block group are obtained by the first transform. In this case, step 1503 includes: obtaining an inverse transform block of each block in the second block group based on the M per-block syntax elements.

[0311] In some implementations, L per-block syntax elements in the M per-block syntax elements each indicates transform coefficients of a respective block in the second block group are obtained by the first transform, L is a non-negative integer less than M, and the obtaining an inverse transform block of each block in the second block group based on the M per-block syntax elements includes: obtaining an inverse transform block of each block in L blocks of the second block group by performing an inverse first transform on dequantized coefficients of a respective block in the L blocks.

[0312] In some implementations, the obtaining an inverse transform block of each block in the second block group based on the M per-block syntax elements further includes: obtaining an inverse transform block of each block in (M-L) blocks of the second block group by performing an inverse identity transform on dequantized coefficients of a respective block in the (M-L) blocks.

[0313] In some cases, (M-L) per-block syntax elements in the M per-block syntax elements each indicates transform coefficients of a respective block in the second block group are not obtained by the first transform, the decoder may infer that the respective (M-L) blocks are obtained by identity transform which means coefficients are passed through without modifying. In this case, the decoder may perform an inverse identity transform on dequantized coefficients of a respective block in the (M-L) blocks. In other words, the decoder may not perform an inverse transform that modifies the dequantized coefficients.

[0314] In some implementations, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further includes: obtaining M per-block syntax elements based on the bitstream, where each per-block syntax element of the M per-block syntax elements indicates a transform of a respective block in the second block group. The performing an inverse transform on the dequantized coefficients of each block group includes: obtaining an inverse transform block of each block of the M blocks in the second block group by performing an inverse transform corresponding to a transform indicated by a respective per-block syntax element on dequantized coefficients of a respective block of the M blocks in the second block group.

[0315] In this case, the M per-block syntax elements each may indicate a specific transform type of a respective block in the second block group. The decoder may perform an inverse transform on dequantized coefficients of a respective block, and type of the inverse transform is inverse to the type of the specific transform type indicated by each of the M per-block syntax elements.

[0316] In some implementations, a first block in the second block group is adjacent to the first block group, and the obtaining the M per-block syntax elements includes: obtaining a per-block syntax element of the first blockby decoding the bitstream based on a probability model of the first block, where the probability model is determined based on transform of the first block group.

[0317] In some cases, it may be of a high probability that a transform type of a block may be the same as a transform type of a neighboring block. In these cases, group-level syntax element(s) available near to the current group-level syntax element may be taken into account to estimate probability of having specific transform-type(s) within a current group of blocks.

[0318] Since the first block in the second block group is adjacent to the first block group, it may be of a high probability that a transform type of the first block is the same as the transform type of the first block group. In this case, the decoder may perform entropy decoding on the per-block syntax element of the first block using probability model of the first block, which may help to improve the efficiency of entropy decoding.

[0319] In some implementations, the method 1500 further includes: obtaining a second syntax element based on the bitstream, where the second syntax element indicates the first transform.

[0320] In this case, the decoder may know the specific transform type of the first transform, and may perform the inverse first transform accordingly.

[0321] In some implementations, reconstructing an image based on the M inverse transform blocks for each block group includes: reconstructing the image by performing a second inverse transform on (MxN) inverse transform blocks.

[0322] In these embodiments, the decoder may perform multiple inverse transforms, corresponding to multiple transforms performed by the encoder. The multiple inverse transforms may be considered as a combination of at least one inverse transform which may be performed sequentially. In an example, the decoder may obtain inverse transform blocks by performing inverse transforms according to one or more of the first syntax elements (each of which indicates the transform type of a respective group), the per-subgroup syntax elements (each of which indicates the transform type of a respective subgroup), or the per-block syntax elements (each of which indicates the transform type of a respective block). Subsequently, the decoder may perform the second inverse transform on the inverse transform blocks obtained in step 1503 to reconstruct the image.

[0323] In some implementations, the second inverse transform includes an inverse wavelet transform. In this case, the decoder may perform an inverse wavelet transform on the inverse transform blocks obtained in step 1503 to reconstruct the image.

[0324] In an implementation, the first inverse transform is an inverse Hadamard transform, and the second inverse transform is an inverse wavelet transform. The first syntax element indicates whether transform coefficients of blocks in a respective block group are obtained by a Hadamard transform. In this case, the decoder may know whether to perform the inverse Hadamard transform on a respective block. In a case where the first syntax element indicates transform coefficients of blocks in a respective block group are obtained by a Hadamard transform, the decoder may obtain inverse transform blocks by performing the inverse Hadamard transform on the respective block group. After that, the decoder may perform the inverse wavelet transform on these inverse transform blocks to reconstruct the image.

[0325] In another example, all blocks in a block group are configured to be of the same transform type, and the first syntax element indicates whether transform coefficients of blocks in a respective block group are obtained by a Hadamard transform and otherwise the Hadamard transform is inferred to be skipped. If the first syntax element indicates transform coefficients of blocks in a respective block group are obtained by a Hadamardtransform, the decoder may obtain inverse transform blocks by performing the inverse Hadamard transform on the respective block group. After that, the decoder may perform the inverse wavelet transform on these inverse transform blocks to reconstruct the image. On the other hand, if the first syntax element indicates transform coefficients of blocks in a respective block group are not obtained by a Hadamard transform, the decoder may not perform the Hadamard transform (i.e., the decoder may perform identity transform) on the respective block group. In this case, the decoder may perform the inverse wavelet transform on the dequantized coefficients to reconstruct the image.

[0326] In some implementations, the inverse quantization is determined based on one or more of: the inverse transform or the second inverse transform. The inverse quantization may also be referred to as dequantization.

[0327] In some cases, a dequantization strength may depend on the inverse transform type, which means that some of inverse transform types correspond to stronger dequantization strength, while other inverse transform types correspond to weaker dequantization strength. Alternatively, the dequantization strength may not depend on the inverse transform type. The quantization strength value may be determined by the decoder or the decoder may parse and decode the information about the quantization strength value from the bitstream.

[0328] The dequantization factor may include a scaling factor of an inverse transform to implement efficient and low-complexity inverse transform and dequantization. The inverse transform and dequantization may be considered as a fuse operation of two steps that are de-quantization and inverse transform-scale. In some cases, the dequantization may be fully or partially fused with an inverse transform. For example, a certain dequantization may be fully or partially fused with a certain inverse transform to simplify computations, and the correspondence between the quantization strength and the inverse transform type may be preconfigured or signaled in the bitstream.

[0329] In some implementations, the transform and / or the second transform may be applied to certain color channel(s) of the blocks. The certain color channel(s) may include one or more of: luminance (i.e. luma), chroma, or different combinations of three color channels such as YUV or RGB.

[0330] In an implementation, the first syntax element is used to indicate a transform type for a certain color component of a respective block group. The certain color may be preconfigured or signaled in the bitstream.

[0331] In another implementation, each color component may correspond to a first syntax. In this case, each color component may correspond to a syntax element that indicates a transform type for this color component of a respective block group. For example, a one-bit syntax element per-color component is used to indicate whether the first transform performed on a respective color component of a respective block group. The term “color component” may also be referred to as “color channel”.

[0332] In an example, the blocks include luma samples, and the transform and / or the second transform may be applied to luma samples of the blocks.

[0333] The following describe some simulation results.

[0334] In a simulation, it is considered wavelet-based codec where Hadamard transform is applied on top of wavelet transform (without limiting the scope of this disclosure, for high-frequency bands). Two alternatives are provided to the codec: wavelet-only or wavelet + Hadamard. 4 blocks share same transform-type and this alternative is applicable only for luma channel.

[0335] As a result, if same quantization is used rate -quality curves, a simulation result of an example with group transform-skip mode, signaled per 4 blocks, and luma channel only is shown in FIG. 16.

[0336] In this simulation, test codec - it’s a codec, which includes adaptive selection between wavelet & wavelet + Hadamard transforms, this selection is done per group of 4 blocks. From this figure, it is obvious that the test codec generates slightly better rate-quality curves for luma channel (Y, dashed-black curve goes over solid-black curve); however, chroma components degraded significantly. It happens because of bitrate is given for the all color components together; but, adaptive selection is applied for luma only.

[0337] FIG. 17 illustrates an example of simulation with group transform-skip mode, signaled per 4 blocks, luma channel only, and the quantization parameter (QP) is increased for Wavelet-only mode

[0338] In another simulation, quantization is further adjusted according to the transform type. In this simulation, the overall bitrate gets aligned with anchor, and the test codec demonstrates clear benefits for luma channel as expected, and there is no drop for chroma. As a result, beneficial effect of the synergy of several steps bring gain ~ -1.2% BD-Rate (YUV PSNR) when this tool was firstly adopted in the reference code. Tables 4 and 5 each illustrates result of the simulation:Table 4Complexity, Enc Complexity, Dec BD-Rate-Y BD-Rate-U BD-Rate-V AVE 104.03% 102.29% -1.505% -0.02% -0.02%MIN 95.73% 91.83% -4.652% -2.49% -2.48%MAX 114.07% 109.87% 0.183% 1.29% 1.29%

[0339] In Table 4, “AVE” represents the average value, “MIN” represents the minimum value, “MAX” represents the maximum value. “Complexity, Enc” represents complexity of the encoder, “Complexity, Dec” represents complexity of the decoder. “BD-Rate- Y”, “BD-Rate-U” and “BD-Rate-V” represent Bjontegaard delta rate for Y, U, and V component, respectively.Table 5Average -1.2%BD-RATE, YUV- Median -1.1%PSNR MIN 0.0%MAX -4.3%

[0340] In Table 5, “BD-Rate, YUV PSNR” represents BD-Rate accounting the weighted average of the peak signal-to-noise ratio (PSNR) for the three color components of YUV.

[0341] The above describes the encoding and decoding methods according to one or more implementations of the present disclosure. The following describes the apparatuses according to one or more implementations of the present disclosure.

[0342] FIG. 18 shows a schematic structural diagram of an apparatus 1800 in a first implementation manner. The apparatus 1800 may be applied to the decoder or a module, a circuit or a chip in the decoder. The apparatus 1800 may include one or more of: an obtaining module 1801 and a reconstructing module 1802.

[0343] The obtaining module 1801 is configured to obtain N first syntax elements and quantized coefficients of each block group of N block groups based on a bitstream, where each first syntax element in the N first syntaxelements indicates whether transform coefficients of all blocks in a respective block group of the N block groups are obtained by a first transform, each block group of the N block groups includes M blocks, and N and M are positive integers; obtain dequantized coefficients of each block group by performing an inverse quantization on the quantized coefficients of each block group; obtain M inverse transform blocks for each block group by performing an inverse transform on the dequantized coefficients of each block group, where the inverse transform is determined based on a respective first syntax element in the N first syntax elements.

[0344] The reconstructing module 1802 is configured to reconstruct an image based on the M inverse transform blocks for each block group.

[0345] In some implementations, one first syntax element in the N first syntax elements indicates that transform coefficients of all blocks in a first block group of the N block groups are obtained by the first transform, and the obtaining module 1801 is configured to obtain an inverse transform block of each block in the first block group by performing an inverse first transform on dequantized coefficients of a respective block in the first block group.

[0346] In some implementations, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the obtaining module 1801 is further configured to: obtain M per-block syntax elements based on the bitstream, where each per-block syntax element of the M per-block syntax elements indicates whether a respective block in the second block group is obtained by the first transform; and obtain an inverse transform block of each block in the second block group based on the M per-block syntax elements.

[0347] In some implementations, L per-block syntax elements in the M per-block syntax elements each indicates a respective block in the second block group is obtained by the first transform, L is a non-negative integer less than M, and the obtaining module 1801 is configured to obtain an inverse transform block of each block in L blocks of the second block group by performing an inverse first transform on dequantized coefficients of a respective block in the L blocks.

[0348] In some implementations, the obtaining module 1801 is configured to obtain an inverse transform block of each block in (M-L) blocks of the second block group by performing an inverse identity transform on dequantized coefficients of a respective block in the (M-L) blocks.

[0349] In some implementations, another first syntax element in the N first syntax elements indicates that not all transform coefficients of blocks in a second block group of the N block groups are obtained by the first transform, and the obtaining module 1801 is further configured to obtain M per-block syntax elements based on the bitstream, where each per-block syntax element of the M per-block syntax elements indicates a transform of a respective block in the second block group; and obtain an inverse transform block of each block of the M blocks in the second block group by performing an inverse transform corresponding to a transform indicated by a respective per-block syntax element on dequantized coefficients of a respective block of the M blocks in the second block group.

[0350] In some implementations, a first block in the second block group is adjacent to the first block group, and the obtaining module 1801 is configured to obtain a per-block syntax element of the first block by decoding the bitstream based on a probability model of the first block, where the probability model is determined based on transform of the first block group.

[0351] In some implementations, the obtaining module 1801 is further configured to obtain a second syntax element based on the bitstream, where the second syntax element indicates the first transform.

[0352] In some implementations, the reconstructing module 1802 is configured to reconstruct the image by performing a second inverse transform on (M*N) inverse transform blocks.

[0353] In some implementations, the second inverse transform includes an inverse wavelet transform.

[0354] In some implementations, the blocks include image blocks.

[0355] In some implementations, the inverse quantization is determined based on one or more of: the inverse transform or the second inverse transform.

[0356] In some implementations, each block includes luma samples.

[0357] It should be noted that the apparatus provided by the aspects and / or implementations of the present disclosure in the first implementation manner can realize all the method steps related to the decoder in the method implementations and can achieve the same technical effects, and the same parts and beneficial effects between this implementation and the method implementations are not repeated here in detail.

[0358] FIG. 19 shows a schematic structural diagram of an apparatus 1900 in a second implementation manner. The apparatus 1900 may be applied to the encoder or a module, a circuit or a chip in the encoder. The apparatus 1900 may include one or more of: an obtaining module 1901, determining module 1902, and an encoding module 1903.

[0359] The obtaining module 1901 is configured to: obtain transform coefficients of blocks in each block group of N block groups by performing a transform on blocks in a respective block group, where each block group of the N block groups includes M blocks, and N and M are positive integers.

[0360] The determining module 1902 is configured to: determine N first syntax elements, where each first syntax element in the N first syntax elements indicates whether transform coefficients of all blocks in a respective block group of N block groups are obtained by a first transform.

[0361] The encoding module 1903 is configured to: encode the N first syntax elements.

[0362] In some implementations, one first syntax element in the N first syntax elements indicates that transform coefficients of all blocks in a first block group of the N block groups are obtained by the first transform.

[0363] In some implementations, an another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the determining module 1902 is configured to: determine M per-block syntax elements, where each per-block syntax element of the M per-block syntax elements indicates whether transform coefficients of a respective block in the second block group are obtained by the first transform. The encoding module 1903 is further configured to: encode the M per-block syntax elements.

[0364] In some implementations, another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform the determining module 1902 is configured to : determine M per-block syntax elements, where each per-block syntax element of the M per-block syntax elements indicates a transform of a respective block in the second block group. The encoding module 1903 is further configured to: encode the M per-block syntax elements.

[0365] In some implementations, a first block in the second block group is adjacent to the first block group, and the encoding module 1903 is configured to: encode a per-block syntax element of the first block based on a probability model of the first block, where the probability model is determined based on transform of the first block group.

[0366] In some implementations, the determining module 1902 is further configured to: determine a second syntax element, where the second syntax element indicates the first transform, and the encoding module 1 03 is further configured to: encode the second syntax element.

[0367] In some implementations, the apparatus 1900 further include a transform module 1904, and before the obtaining transform coefficients of blocks in each block group of N block groups by performing a transform on blocks in a respective block group, transform module 1904 is configured to: perform a second transform on blocks in each block group of the N block groups.

[0368] In some implementations, the second transform includes a wavelet transform.

[0369] In some implementations, the blocks include image blocks.

[0370] In some implementations, the obtaining module 1901 is further configured to: obtain quantized coefficients by performing a quantization on the transform coefficients of each block group, where quantization is determined based on the transform.

[0371] In some implementations, the obtaining module 1901 is further configured to: obtain quantized coefficients by performing a quantization on the transform coefficients of each block group, where quantization is determined based on one or more of: the transform or the second transform.

[0372] In some implementations, each block includes luma samples.

[0373] It should be noted that the apparatus provided by the aspects and / or implementations of the present disclosure in the second implementation manner can realize all the method steps related to the encoder in the method implementations and can achieve the same technical effects, and the same parts and beneficial effects between this implementation and the method implementations are not repeated here in detail.

[0374] Referring back to FIG. 4, in a third implementation manner, the video coding device 400 may be applied to a decoding apparatus and realize the above method implementations performed by the decoder. The processor 430 executes computer executable instructions stored in the memory 460 for executing the above methods implemented by the decoder. It should be noted that, the memory 460 may be included or excluded from the coding device 400, depending on actual needs.

[0375] Referring back to FIG. 4, in a fourth implementation manner, the video coding device 400 may be applied to an encoding apparatus and realize the above method implementations performed by the encoder. The processor 430 executes computer executable instructions stored in the memory 460 for executing the above methods implemented by the encoder. It should be noted that, the memory 460 may be included or excluded from the coding device 400, depending on actual needs.

[0376] The present disclosure encompasses various implementations, including not only method implementations, but also other implementations such as apparatus implementations and implementations related to non-transitory computer readable storage media. Implementations may incorporate, individually or in combinations, the features disclosed herein.

[0377] Although this disclosure refers to illustrative implementations, this is not intended to be construed in a limiting sense. Various modifications and combinations of the illustrative implementations, as well as other implementations of the disclosure, will be apparent to persons skilled in the art upon reference to the description.

[0378] Features disclosed herein in the context of any particular implementations may also or instead be implemented in other implementations. Method implementations, for example, may also or instead be implemented in apparatus, system, and / or computer program product implementations. In addition, althoughimplementations are described primarily in the context of methods and apparatus, other implementations are also contemplated, as instructions stored on one or more non-transitory computer-readable media, for example. Such media could store programming or instructions to perform any of various methods consistent with the present disclosure.

[0379] Some implementations of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium). The computer-readable storage medium has stored thereon program instructions that, when run on a network device / terminal device, cause the network device / terminal device to execute one or more steps of the method for beam management as described in any one of the above implementations.

[0380] For example, the computer-readable storage medium includes, but is not limited to, a magnetic storage device (e.g., a hard disk, a floppy disk or a magnetic tape), an optical disk (e.g., a compact disk (CD), or a DVD), a smart card, and a flash memory device (e.g., an erasable programmable read-only memory (EPROM), a card, a stick or a key driver). Various computer-readable storage media described in the implementations of the present disclosure may represent one or more devices and / or other machine-readable storage media, which are used for storing information. The term “computer-readable storage medium” may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0381] Some implementations of the present disclosure further provide a computer program product. The computer program product includes program instructions carried on a non-transitory computer-readable storage medium. When executed on a network device / terminal device, the computer program instructions cause the network device / terminal device to perform one or more steps of the method for data transmission as described in the above implementations.

[0382] Beneficial effects of the computer-readable storage medium and the computer program product are the same as the beneficial effects of the method for data transmission as described in some of the above implementations, and details will not be repeated here.

[0383] The foregoing descriptions are merely specific implementations of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or replacements within the technical scope of the present disclosure shall be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

[0384] In some aspects of the present disclosure, there is provided a computer program including instructions. The instructions, when executed by a processor, may cause the processor to implement a method of the present disclosure.

[0385] In some aspects of the present disclosure, there is provided an integrated circuit. The integrated circuit includes one or more logic circuits for executing the steps of the method for data transmission of the present disclosure.

[0386] In some aspects of the present disclosure, there is provided an apparatus including means (e.g., at least one processor) to implement a method of the present disclosure. The apparatus may be device (that is, a terminal device or a network device) or a module or component in the device. The at least one processor may execute instructions stored in a computer-readable medium to implement the method.

[0387] The apparatus may be a communication device or an apparatus implemented in a communication device. For example, the apparatus implemented in a communication device may be an integrated circuit, which in somecontexts may be known by other colloquial names, such as chip, modem, modem chip, baseband chip, or baseband processor. In some implementations, one or more integrated circuits can be packaged into a system-on-chip, a system-in-package, or a multi-chip module. The apparatus may include one or more integrated circuits or include one or more integrated circuits and other discrete components.

[0388] It will be appreciated that any module, component, or device disclosed herein that executes instructions may include, or otherwise have access to, a non-transitory computer / processor readable storage medium or media for storage of information, such as computer / processor readable instructions, data structures, program modules and / or other data. A non-exhaustive list of examples of non-transitory computer / processor readable storage media includes magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, optical disks such as compact disc read-only memory (CD-ROM), digital video discs or digital versatile discs (i.e., DVDs), Blu-ray Disc™, or other optical storage, volatile and non-volatile, removable and non-removable media implemented in any method or technology, random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology. Any such non-transitory computer / processor storage media may be part of a device / apparatus or accessible or connectable thereto. Computer / processor readable / executable instructions to implement a method, an application or a module described herein may be stored or otherwise held by such non-transitory computer / processor readable storage media.

[0389] It could be noted that the message in the disclosure could be replaced with information, which may be carried in one single message, or be carried in more than one separate message.

[0390] The terms “apparatus” and “device” are used exchangeable.

[0391] The terms “first”, “second”, and “third” are used for descriptive purposes only, and are not to be construed as indicating or implying the relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined with “first”, “second” or “third” may explicitly or implicitly include one or more of the features.

[0392] In the present disclosure, the terms “a” or “an” are defined to mean “at least one”, that is, these terms do not exclude a plural number of items, unless stated otherwise.

[0393] In the present disclosure, terms such as “substantially”, “generally” and “about”, which modify a value, condition or characteristic of a feature of an example implementation, is understood to mean that the value, condition or characteristic is defined within tolerances that are acceptable for the proper operation of the example implementation for its intended application.

[0394] In the present disclosure, unless stated otherwise, the terms “connected” and “coupled”, and derivatives and variants thereof, refer herein to any structural or functional connection or coupling, either direct or indirect, between two or more elements. For example, the connection or coupling between the elements can be acoustical, mechanical, optical, electrical, thermal, logical, or any combinations thereof.

[0395] In the present disclosure, expressions such as “match”, “matching” and “matched”, including variants and derivatives thereof, are intended to refer herein to a condition in which two or more elements are either the same or within some predetermined tolerance of each other. That is, these terms are meant to encompass not only “exactly” or “identically” matching the two elements but also “substantially”, “approximately” or “subjectively” matching the two or more elements, as well as providing a higher or best match among a plurality of matching possibilities.

[0396] In the present disclosure, the expression “based on” is intended to mean “based at least partly on”, that is, this expression can mean “based solely on” or “based partially on”, and so may not be interpreted in a limited manner. More particularly, the expression “based on” could also be understood as meaning “depending on”, “representative of’, “indicative of’, “associated with” or similar expressions.

[0397] In the present disclosure, the terms “system” and “network” may be used interchangeably in different implementations of this application. “At least one” means one or more, and “a plurality of’ means two or more. The term “and / or” describes an association relationship of associated objects, and indicates that three relationships may exist. For example, A and / or B may indicate the following three cases: Only A exists, both A and B exist, and only B exists, where A and B may be singular or plural. The character “ / ” indicates an “or” relationship between associated objects. “At least one of the following items (pieces)” or a similar expression thereof indicates any combination of these items, including a single item (piece) or any combination of a plurality of items (pieces). For example, “at least one of A, B, or C” includes: only A; only B; only C; A and B; A and C; B and C; or A, B, and C, and “at least one of A, B, and C” may also be understood as including: only A; only B; only C; A and B; A and C; B and C; or A, B, and C. In addition, unless otherwise specified, ordinal numbers such as “first” and “second” in implementations of this application are used to distinguish between a plurality of objects, and are not used to limit a sequence, a time sequence, priorities, or importance of the plurality of objects.

[0398] A person skilled in the art may understand that implementations of this application may be provided as a method, an apparatus (or system), computer-readable storage medium, or a computer program product. Therefore, this application may use a form of a hardware-only implementation, a software-only implementation, or one or more implementations with a combination of software and hardware. Moreover, this application may use a form of a computer program product that is implemented on one or more computer-usable storage media (including but not limited to a disk memory, an optical memory, and the like) that include computer-usable program code.

[0399] This application is described with reference to the flowcharts and / or block diagrams of the method, the device (system), and the computer program product according to this application. It is understood that computer program instructions may be used to implement each process and / or each block in the flowcharts and / or the block diagrams and a combination of a process and / or a block in the flowcharts and / or the block diagrams. The computer program instructions may be provided for a general-purpose computer, a dedicated computer, an embedded processor, or a processor of another programmable data processing device and enable a machine to execute the instructions. When executed by any computer or the processor of a programmable data processing device, the instructions cause the apparatus to implement specific functions as described in one or more procedures in the flowcharts and / or one or more blocks in the block diagrams. The computer program instructions may alternatively be stored in a computer-readable memory that can indicate a computer or another programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate an artifact that includes an instruction apparatus. The instruction apparatus implements a specific function in one or more procedures in the flowcharts and / or one or more blocks in the block diagrams.

[0400] The computer program instructions may alternatively be loaded onto a computer or another programmable data processing device, so that a series of operations and steps are performed on the computer or the another programmable device, so that computer-implemented processing is generated. Therefore, theinstructions executed on the computer or on another programmable device provide steps for implementing specific functions as described in one or more procedures in the flowcharts and / or one or more blocks in the block diagrams.

[0401] It is clear that a person skilled in the art can make various modifications and variations to this application without departing from the scope of this disclosure. This disclosure is intended to cover these modifications and variations of this application provided that they fall within the scope of protection defined by the following claims and their equivalent technologies.

Claims

CLAIMS1. A decoding method, comprising:obtaining N first syntax elements and quantized coefficients of each block group of N block groups based on a bitstream, wherein each first syntax element in the N first syntax elements indicates whether transform coefficients of all blocks in a respective block group of the N block groups are obtained by a first transform, each block group of the N block groups comprises M blocks, and N and M are positive integers;obtaining dequantized coefficients of each block group by performing an inverse quantization on the quantized coefficients of each block group;obtaining M inverse transform blocks for each block group by performing an inverse transform on the dequantized coefficients of each block group, wherein the inverse transform is determined based on a respective first syntax element in the N first syntax elements; andreconstructing an image based on the M inverse transform blocks for each block group.

2. The method of claim 1, wherein one first syntax element in the N first syntax elements indicates that transform coefficients of all blocks in a first block group of the N block groups are obtained by the first transform, and the performing an inverse transform on the dequantized coefficients of each block group comprises:obtaining an inverse transform block of each block in the first block group, by performing an inverse first transform on dequantized coefficients of a respective block in the first block group.

3. The method of claim 1 or 2, wherein another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further comprises:obtaining M per-block syntax elements based on the bitstream, wherein each per-block syntax element of the M per-block syntax elements indicates whether transform coefficients of a respective block in the second block group are obtained by the first transform;wherein the performing an inverse transform on the dequantized coefficients of each block group comprises:obtaining an inverse transform block of each block in the second block group based on the M per-block syntax elements.

4. The method of claim 3, wherein L per-block syntax elements in the M per-block syntax elements each indicates transform coefficients of a respective block in the second block group are obtained by the first transform, L is a non-negative integer less than M, and the obtaining an inverse transform block of each block in the second block group based on the M per-block syntax elements comprises:obtaining an inverse transform block of each block in L blocks of the second block group by performing an inverse first transform on dequantized coefficients of a respective block in the L blocks.

5. The method of claim 3 or 4, wherein the obtaining an inverse transform block of each block in the second block group based on the M per-block syntax elements further comprises:obtaining an inverse transform block of each block in (M-L) blocks of the second block group by performing an inverse identity transform on dequantized coefficients of a respective block in the (M-L) blocks.

6. The method of claim 1 or 2, wherein another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further comprises:obtaining M per-block syntax elements based on the bitstream, wherein each per-block syntax element of the M per-block syntax elements indicates a transform of a respective block in the second block group;wherein the performing an inverse transform on the dequantized coefficients of each block group comprises:obtaining an inverse transform block of each block of the M blocks in the second block group by performing an inverse transform corresponding to a transform indicated by a respective per-block syntax element on dequantized coefficients of a respective block of the M blocks in the second block group.

7. The method of any one of claims 3 to 6, wherein a first block in the second block group is adjacent to the first block group, and the obtaining the M per-block syntax elements comprises:obtaining a per-block syntax element of the first block by decoding the bitstream based on a probability model of the first block, wherein the probability model is determined based on transform of the first block group.

8. The method of any one of claims 1 to 7, further comprising:obtaining a second syntax element based on the bitstream, wherein the second syntax element indicates the first transform.

9. The method of any one of claims 1 to 8, wherein reconstructing an image based on the M inverse transform blocks for each block group comprises:reconstructing the image by performing a second inverse transform on (MxN) inverse transform blocks.

10. The method of claim 9, wherein the second inverse transform comprises an inverse wavelet transform.

11. The method of any one of claims 1 to 10, wherein the blocks comprise image blocks.

12. The method of any one of claims 9 to 11, wherein the inverse quantization is determined based on one or more of: the inverse transform or the second inverse transform.

13. The method of any one of claims 1 to 12, wherein each block comprises luma samples.

14. An encoding method, comprising:obtaining transform coefficients of blocks in each block group of N block groups by performing a transform on blocks in a respective block group, wherein each block group of the N block groups comprises M blocks, and N and M are positive integers;determining N first syntax elements, wherein each first syntax element in the N first syntax elements indicates whether transform coefficients of all blocks in a respective block group of N block groups are obtained by a first transform; andencoding the N first syntax elements.

15. The method of claim 14, wherein one first syntax element in the N first syntax elements indicates that transform coefficients of all blocks in a first block group of the N block groups are obtained by the first transform.

16. The method of claim 14 or 15, wherein another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further comprises:determining M per-block syntax elements, wherein each per-block syntax element of the M per-block syntax elements indicates whether transform coefficients of a respective block in the second block group are obtained by the first transform; andencoding the M per-block syntax elements.

17. The method of claim 14 or 15, wherein another first syntax element in the N first syntax elements indicates that transform coefficients of not all blocks in a second block group of the N block groups are obtained by the first transform, and the method further comprises:determining M per-block syntax elements, wherein each per-block syntax element of the M per-block syntax elements indicates a transform of a respective block in the second block group; andencoding the M per-block syntax elements.

18. The method of claim 16 or 17, wherein a first block in the second block group is adjacent to the first block group, and the encoding the M per-block syntax elements comprises:encoding a per-block syntax element of the first block based on a probability model of the first block, wherein the probability model is determined based on transform of the first block group.

19. The method of any one of claims 14 to 18, further comprising:determining a second syntax element, wherein the second syntax element indicates the first transform; and encoding the second syntax element.

20. The method of any one of claims 14 to 19, before the obtaining transform coefficients of blocks in each block group of N block groups by performing a transform on blocks in a respective block group, further comprising:performing a second transform on blocks in each block group of the N block groups.

21. The method of claim 20, wherein the second transform comprises a wavelet transform.

22. The method of any one of claims 14 to 21, wherein the blocks comprise image blocks.

23. The method of any one of claims 20 to 22, further comprising:obtaining quantized coefficients by performing a quantization on the transform coefficients of each block group, wherein quantization is determined based on the transform.

24. The method of claim 23, further comprising:obtaining quantized coefficients by performing a quantization on the transform coefficients of each block group, wherein quantization is determined based on one or more of: the transform or the second transform.

25. The method of any one of claims 14 to 24, wherein each block comprises luma samples.

26. An apparatus for receiving or restoring a bitstream, wherein the bitstream is obtained by method of any one of claims 1 to 25.

27. An apparatus, configured to perform the method according to any one of claims 1 to 13, or any one of claims 14 to 25.

28. A computer-readable storage medium having instructions stored thereon which, when executed by an apparatus, cause the apparatus to perform the method of any one of claims 1 to 13, or any one of claims 14 to 25.

29. A computer program product storing instructions which, when executed, cause an apparatus to perform the method of any one claims 1 to 13, or any one of claims 14 to 25.

Images