Signaled conversion set

Abstract

This disclosure relates in general to video coding / decoding, and in particular to improvements for signaled transformation sets. One method includes the steps of: receiving a coded video bitstream; extracting a set of transformation coefficients for a current block based on the coded video bitstream; determining a transformation set index for the current block in intra-predictive mode based on syntax elements explicitly signaled in the coded video bitstream; identifying a transformation set according to the transformation set index, where the transformation set index indicates a transformation set among multiple transformation sets; performing an inverse transformation using the set of transformation coefficients and the identified transformation set to obtain a residual block for the current block; and the device reconstructing the current block based on the residual block.

Description

Technical Field

[0001] [Incorporation by Reference] This application claims the benefit of priority based on U.S. Provisional Patent Application No. 63 / 470,763, filed on June 2, 2023, the entire disclosure of which is incorporated herein by reference. This application is also based on and claims the benefit of priority of U.S. Non-Provisional Patent Application No. 18 / 497,778, filed on October 30, 2023, the entire disclosure of which is incorporated herein by reference.

[0002] [Technical Field] This disclosure describes a set of advanced video / streaming coding / decoding techniques. More specifically, the disclosed techniques include signaling a set of transforms for residual blocks.

Background Art

[0003] Uncompressed digital video can include a series of pictures and can have specific bitrate requirements for storage, data processing, and transmission bandwidth in streaming applications. One purpose of video coding and decoding can be the reduction of redundancy in an uncompressed input video signal through various compression techniques.

Summary of the Invention

Means for Solving the Problems

[0004] This disclosure describes various embodiments of methods, apparatuses, and computer-readable storage media for improving the signaling of a set of transforms for residual blocks.

[0005] According to one aspect, an embodiment of the present disclosure provides a method for decoding the current block of the current frame in a coded video bitstream. The method includes the step of a device receiving a coded video bitstream. The device includes a memory storing instructions and a processor communicating with the memory. The method also includes the step of the device extracting a set of transformation coefficients for the current block based on the coded video bitstream; determining a transformation set index for the current block in intra-predictive mode based on syntax elements explicitly signaled in the coded video bitstream, wherein the transformation set index indicates a transformation set among a plurality of transformation sets; the device identifying a transformation set according to the transformation set index; the device performing an inverse transformation using the set of transformation coefficients and the identified transformation set to obtain a residual block for the current block; and the device reconstructing the current block based on the residual block.

[0006] In other embodiments, embodiments of the present disclosure provide a device for processing the current block of the current frame in a coded video bitstream. The device includes a memory storing instructions and a processor communicating with the memory. When the processor executes an instruction, the processor is configured to cause the device to perform the methods described above for video decoding and / or encoding.

[0007] In other embodiments, embodiments of the present disclosure provide a non-temporary computer-readable medium that stores instructions causing a computer to perform the above-described method for video decoding and / or encoding when performed by the computer for video decoding and / or encoding.

[0008] The above-described and other embodiments, as well as their implementations, will be described in more detail in the drawings, description, and claims.

[0009] Further features, properties, and various advantages of the disclosed subject matter will become clearer from the detailed description and accompanying drawings below. [Brief explanation of the drawing]

[0010] [Figure 1] This figure schematically shows a simplified block diagram of a communication system (100) according to an exemplary embodiment.

[0011] [Figure 2] This figure schematically shows a simplified block diagram of a communication system (200) according to an exemplary embodiment.

[0012] [Figure 3] This figure schematically shows a simplified block diagram of a video decoder according to an exemplary embodiment.

[0013] [Figure 4] This figure schematically shows a simplified block diagram of a video encoder according to an exemplary embodiment.

[0014] [Figure 5] This figure shows a block diagram of a video encoder according to another exemplary embodiment.

[0015] [Figure 6] This figure shows a block diagram of a video decoder according to another exemplary embodiment.

[0016] [Figure 7] This figure shows a coding block partitioning scheme according to an exemplary embodiment of the present disclosure.

[0017] [Figure 8] This figure shows another method of coding block partitioning according to the exemplary embodiments of the present disclosure.

[0018] [Figure 9] A diagram showing another method of coding block partitioning according to an exemplary embodiment of the present disclosure.

[0019] [Figure 10] A diagram showing exemplary fine angles in directional intra prediction.

[0020] [Figure 11] A diagram showing a nominal angle in directional intra prediction.

[0021] [Figure 12] A diagram schematically showing the use of secondary conversion in encoding and decoding processes.

[0022] [Figure 13] A diagram showing low-frequency non-separable conversion processing according to an exemplary embodiment of the present disclosure.

[0023] [Figure 14] A diagram showing an exemplary logic flow for a method in the present disclosure.

[0024] [Figure 15] A diagram schematically showing a computer system according to an exemplary embodiment of the present disclosure.

Embodiments for Carrying Out the Invention

[0025] The present invention is described in detail below with reference to the accompanying drawings, which form part of the invention and illustrate specific examples of embodiments. However, it should be noted that the present invention may be embodied in various different forms, and therefore the protected, i.e., claimed subject matter is intended to be construed as not being limited to any of the embodiments described below. It should also be noted that the present invention may be embodied as a method, apparatus, component, or system. Accordingly, embodiments of the present invention may take the form of, for example, hardware, software, firmware, or any combination thereof.

[0026] Throughout this specification and the claims, terms may have nuances implied or suggested in context beyond their expressly stated meanings. The phrases “in one embodiment” or “in some embodiments” as used herein do not necessarily refer to the same embodiment, and the phrases “in other embodiments” or “in other embodiments” as used herein do not necessarily refer to different embodiments. Similarly, the phrases “in one implementation” or “in some implementations” as used herein do not necessarily refer to the same implementation, and the phrases “in other implementations” or “in other implementations” as used herein do not necessarily refer to different implementations. For example, the claimed subject matter is intended to include, in whole or in part, exemplary embodiment / implementation combinations.

[0027] In general, terms can be understood, at least in part, from their usage in context. For example, terms such as “and,” “or,” or “and / or” as used herein may have various meanings that may, at least in part, depend on the context in which such terms are used. Typically, when “or” is used to relate a list such as A, B, or C, it is intended to mean A, B, and C in an inclusive sense, as well as A, B, or C in an exclusive sense. Furthermore, the terms “one or more” or “at least one” as used herein may, at least in part depending on the context, be used to describe any feature, structure, or characteristic in the singular, or to describe a combination of features, structures, or characteristics in the plural. Similarly, terms such as “a,” “an,” or “the” can also be understood, at least in part depending on the context, to convey either a singular or plural usage. Furthermore, the terms "based on" or "determined by" can be understood not to necessarily convey that there is an exclusive set of factors, but rather, they can allow for the existence of additional factors that are not necessarily explicitly explained, also at least partially depending on the context.

[0028] As shown in Figure 1, terminal devices may be implemented as servers, personal computers, and smartphones, but the applicability of the fundamental principles of this disclosure is not limited thereto. Embodiments of this disclosure may be implemented in desktop computers, laptop computers, tablet computers, media players, wearable computers, dedicated video conferencing equipment, and / or similar devices. Network (150) represents any number or type of network that transmits encoded video data between terminal devices, including, for example, wired and / or wireless communication networks. Communication network (150) may exchange data via circuit switching, packet switching, and / or other types of channels. Typical networks include telecommunications networks, local area networks, wide area networks, and / or the Internet.

[0029] Figure 2 shows the arrangement of a video encoder and video decoder in a video streaming environment as an example of an application for the disclosed subject matter. The disclosed subject matter may also be equally applicable to other video applications, such as video conferencing, digital television broadcasting, games, virtual reality, and the storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0030] As shown in Figure 2, the video streaming system may include a video capture subsystem (213) which may include a video source (201), such as a digital camera, for creating a stream of uncompressed video pictures or images (202). In one example, the stream of video pictures (202) includes samples recorded by the digital camera of the video source (201). The stream of video pictures (202), shown in bold to emphasize the high data volume compared to encoded video data (204) (or coded video bitstream), may be processed by an electronic device (220) which includes a video encoder (203) coupled to the video source (201). The video encoder (203) may include hardware, software, or a combination thereof for enabling or implementing embodiments of the subject matter disclosed, as will be detailed below. The encoded video data (204) (or encoded video bitstream (204)), shown in thin lines to highlight the lower data volume compared to the uncompressed video picture stream (202), may be stored on a streaming server (205) for future use or directly on a downstream video device (not shown). One or more streaming client subsystems, such as client subsystems (206) and (208) in Figure 2, can access the streaming server (205) to retrieve copies (207) and (209) of the encoded video data (204). The client subsystem (206) may include a video decoder (210) within an electronic device (230), for example. The video decoder (210) decodes the incoming copy (207) of the encoded video data to create a departure stream of a video picture (211) that is uncompressed and can be rendered on a display (212) (e.g., a display screen) or other rendering device (not shown).

[0031] Figure 3 shows a block diagram of a video decoder (310) of an electronic device (330) according to any embodiment of the present disclosure described below. The electronic device (330) may include a receiver (331) (e.g., a receiving circuit). The video decoder (310) may be used in place of the video decoder (210) in the example of Figure 2.

[0032] As shown in Figure 3, the receiver (331) may receive one or more coded video sequences from the channel (301). A buffer memory (315) may be placed between the receiver (331) and the entropy decoder / parser (320) (hereinafter, "Parser (320)"). The Parser (320) may reconstruct symbols (321) from the coded video sequences. These categories of symbols include information used to manage the operation of the video decoder (310), and potentially information for controlling rendering devices such as the display (312) (e.g., a display screen). The Parser (320) may parse / entropy decode the coded video sequences. The Parser (320) may extract from the coded video sequences a set of subgroup parameters relating to at least one of the subgroups of pixels in the video decoder. Subgroups may include Groups of Pictures (GOP), pictures, tiles, slices, macroblocks, Coding Units (CU), blocks, Transform Units (TU), Prediction Units (PU), etc. The parser (320) may also extract information from the coded video sequence, such as transformation coefficients (e.g., Fourier transform coefficients), quantization parameter values, and motion vectors. The reconstruction of the symbol (321) may involve multiple different processing or function units. The units involved, and how they are involved, may be controlled by the parser (320) based on subgroup control information analyzed from the coded video sequence.

[0033] The first unit may include a scaler / inverse unit (351). The scaler / inverse unit (351) may receive control information from the parser (320) including the quantized transformation coefficients, information indicating which type of inverse transformation to use, block size, quantization coefficients / parameters, quantization scaling matrix, and position (lie) as symbol (321). The scaler / inverse unit (351) may output a block containing sample values ​​that can be input to the aggregator (355).

[0034] In some cases, the output samples of the scaler / inverse transform (351) may relate to intracoded blocks, i.e., blocks that do not use prediction information from previously reconstructed pictures, but can use prediction information from previously reconstructed portions of the current picture. Such prediction information may be provided by an intrapicture prediction unit (352). In some cases, the intrapicture prediction unit (352) may generate a block of the same size and shape as the block being reconstructed, using surrounding block information that has already been reconstructed and stored in the current picture buffer (358). The current picture buffer (358) buffers, for example, partially reconstructed current pictures and / or fully reconstructed current pictures. In some embodiments, the aggregator (355) may add the prediction information generated by the intraprediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351) on a sample-by-sample basis.

[0035] In other cases, the output samples of the scaler / inverse unit (351) may relate to an intercoded, potentially motion-compensated block. In such cases, the motion-compensated prediction unit (353) can access the reference picture memory (357) based on the motion vector and fetch samples to be used for interpicture prediction. After motion-compensating the fetched reference samples according to the symbols (321) associated with the block, these samples can be added by the aggregator (355) to the output of the scaler / inverse unit (351) (the output of unit 351 may be referred to as residual samples or residual signal) to generate output sample information.

[0036] The output samples of the aggregator (355) can undergo various loop filtering techniques in a loop filter unit (356) which includes several types of loop filters. The output of the loop filter unit (356) may be a sample stream that is output to a rendering device (312) and can be stored in a reference picture memory (357) for use in future interpicture prediction.

[0037] Figure 4 shows a block diagram of a video encoder (403) according to an exemplary embodiment of the present disclosure. The video encoder (403) may be included in an electronic device (420). The electronic device (420) may further include a transmitter (440) (e.g., a transmitting circuit). The video encoder (403) may be used instead of the video encoder (403) in the example of Figure 4.

[0038] The video encoder (403) may receive video samples from the video source (401). According to some exemplary embodiments, the video encoder (403) may encode and compress the pictures of the source video sequence into a coded video sequence (443) in real time or under any other time constraints required by the application. Implementing an appropriate coding speed constitutes one function of the controller (450). In some embodiments, the controller (450) may be functionally coupled to and control other functional units, as described below. Parameters set by the controller (450) may include rate control-related parameters (picture skip, quantizer, lambda value of rate distortion optimization technique, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc.

[0039] In some exemplary embodiments, the video encoder (403) may be configured to operate in a coding loop. The coding loop may include a source coder (430) and a (local) decoder (433) embedded in the video encoder (403). The decoder (433) reconstructs the symbols in a similar manner to that produced by a (remote) decoder, even if the embedded decoder 433 processes the video stream coded by the source coder 430 without entropy coding (since any compression between symbols in entropy coding and the coded video bitstream can be lossless in the video compression techniques considered in the disclosed subject). An observation that can be made at this point is that any decoder technique other than parsing / entropy decoding that may exist only in the decoder may necessarily need to exist in the corresponding encoder in substantially the same functional form. For this reason, the disclosed subject sometimes focuses on decoder operation comparable to the decoding portion of the encoder. Thus, the description of encoder techniques can be omitted, as it is the inverse of the comprehensively described decoder techniques. A more detailed description of the encoder is given below, in only certain areas or embodiments.

[0040] In operation in some exemplary implementations, the source coder (430) may perform motion-compensated predictive coding, predictively coding the input picture by referencing one or more previously coded pictures from a video sequence designated as “reference pictures”.

[0041] The local video decoder (433) can decode the coded video data of a picture that may be designated as a reference picture. The local video decoder (433) can reproduce the decoding process that may be performed on the reference picture by the video decoder and store the reconstructed reference picture in the reference picture cache (434). In this way, the video encoder (403) can locally store a copy of the reconstructed reference picture that has the same content as the reconstructed reference picture obtained by the far-end (remote) video decoder (if there are no transmission errors).

[0042] The predictor (435) can perform a predictive search for the coding engine (432). That is, for a new picture to be coded, the predictor (435) can search the reference picture memory (434) for sample data (as candidate reference pixel blocks) that can function as a suitable predictive reference for the new picture, or for some metadata such as reference picture motion vectors, block shapes, etc.

[0043] The controller (450) may manage the coding operations of the source coder (430), including, for example, setting parameters and subgroup parameters used to encode video data.

[0044] The outputs of all the aforementioned functional units may undergo entropy coding in the entropy coder 445. The transmitter (440) may buffer the coded video sequence created by the entropy coder (445) in preparation for transmission via a communication channel (460), which may be a hardware / software link to a storage device that stores the coded video data. The transmitter (440) may merge the coded video data from the video coder (403) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0045] The controller (450) can manage the operation of the video encoder (403). During coding, the controller (450) can assign each coded picture to a type of coded picture that can influence the coding techniques that may be applied to that picture. For example, a picture may often be assigned to one of the following picture types: intra-picture (I-picture), predictive picture (P-picture), bidirectional predictive picture (B-picture), or multiple predictive picture. A source picture can generally be spatially subdivided into multiple sample coding blocks, as will be further detailed below.

[0046] Figure 5 shows a diagram of a video encoder (503) according to another exemplary embodiment of the present disclosure. The video encoder (503) is configured to receive a processing block (e.g., a prediction block) of sample values ​​in a video picture in a sequence of video pictures and to encode the processing block into a coded picture which is part of a coded video sequence. The exemplary video encoder (503) may be used instead of the video encoder (403) in the example of Figure 4.

[0047] For example, the video encoder (503) receives a matrix of sample values ​​for a processing block. The video encoder (503) then determines, for example, whether the processing block is best coded using intra-mode, inter-mode, or bi-predictive mode, using rate distortion optimization (RDO).

[0048] In the example shown in Figure 5, the video encoder (503) includes an interencoder (530), an intraencoder (522), a residual calculator (523), a switch (526), ​​a residual encoder (524), a general-purpose controller (521), and an entropy encoder (525), all coupled together as shown in the exemplary configuration of Figure 5.

[0049] The interencoder (530) is configured to receive a sample of the current block (e.g., a processing block), compare the block with one or more reference blocks in the reference picture (e.g., blocks in the previous and subsequent pictures in display order), generate interprediction information (e.g., descriptions of redundant information by interencoding techniques, motion vectors, merge mode information), and compute interprediction results (e.g., predicted blocks) based on the interprediction information using any appropriate technique.

[0050] The intra encoder (522) is configured to receive a sample of the current block (e.g., a processing block), compare the block to an already coded block in the same picture, generate quantization coefficients after the transformation, and optionally also generate intra prediction information (e.g., intra prediction direction information by one or more intra encoding techniques).

[0051] The general-purpose controller (521) may be configured to determine general-purpose control data and, based on the general-purpose control data, control other components of the video encoder (503) to determine, for example, the prediction mode of a block and provide control signals to the switch (526) based on the prediction mode.

[0052] A residual calculator (523) may be configured to calculate the difference (residual data) between the received block and the prediction result for a block selected from an intra encoder (522) or interencoder (530). A residual encoder (524) may be configured to encode the residual data to generate transformation coefficients. The transformation coefficients are then quantized to obtain quantized transformation coefficients. In various exemplary embodiments, the video encoder (503) also includes a residual decoder (528). The residual decoder (528) is configured to perform an inverse transformation to generate decoded residual data. An entropy encoder (525) may be configured to format the bitstream to include the encoded block and perform entropy coding.

[0053] Figure 6 shows a diagram of an exemplary video decoder (610) according to another embodiment of the present disclosure. The video decoder (610) is configured to receive a coded picture which is part of a coded video sequence and to decode the coded picture to produce a reconstructed picture. In one example, the video decoder (610) may be used instead of the video decoder (410) in the example of Figure 4.

[0054] In the example shown in Figure 6, the video decoder (610) includes an entropy decoder (671), an interdecoder (680), a residual decoder (673), a reconfiguration module (674), and an intradecoder (672), which are coupled together as shown in the exemplary configuration of Figure 6.

[0055] An entropy decoder (671) may be configured to reconstruct from a coded picture some symbols representing the syntax elements that make up the coded picture. An interdecoder (680) may be configured to receive interprediction information and generate interprediction results based on the interprediction information. An intradecoder (672) may be configured to receive intraprediction information and generate prediction results based on the intraprediction information. A residual decoder (673) may be configured to perform inverse quantization to extract dequantized transformation coefficients and process the dequantized transformation coefficients to convert the residuals from the frequency domain to the spatial domain. A reconstruction module (674) may be configured to combine the residuals output by the residual decoder (673) and the prediction results (which may be output by the interprediction module or intraprediction module, depending on the case) in the spatial domain to form a reconstruction block as part of the reconstructed video that forms part of the reconstructed picture.

[0056] It should be noted that the video encoders (203), (403), and (503), as well as the video decoders (210), (310), and (610), may be implemented using any suitable technique. In some exemplary embodiments, the video encoders (203), (403), and (503), as well as the video decoders (210), (310), and (610), may be implemented using one or more integrated circuits. In other embodiments, the video encoders (203), (403), and (503), as well as the video decoders (210), (310), and (610), may be implemented using one or more processors that execute software instructions.

[0057] Next, considering block partitioning for coding and decoding, general partitioning may begin with a base block and may follow a predetermined set of rules, a specific pattern, a partition tree, or any partition structure or scheme. Partitioning can be hierarchical and recursive. After dividing or partitioning the base block according to one of the exemplary partitioning procedures or other procedures described below, or a combination thereof, a final set of partitions or coding blocks may be obtained. Each of these partitions may be at one of the various partitioning levels in the partitioning hierarchy and may take on various shapes. Each partition may be referred to as a coding block (CB). In the various exemplary partitioning implementations described further below, each obtained CB may be of any of the allowed sizes and partitioning levels. Such partitions are referred to as coding blocks because they may form units in which several basic coding / decoding decisions can be made, and coding / decoding parameters can be optimized, determined, and signaled in the encoded video bitstream. The highest or deepest level in the final partition represents the depth of the coding block partitioning structure of the tree. A coding block can be a lumen coding block or a chroma coding block. The CB tree structure for each color can be called a coding block tree (CBT). The coding blocks for all color channels can be collectively called coding units (CUs). The hierarchical structure for all color channels can be collectively called a coding tree unit (CTU). The partitioning patterns or structures for different color channels within a CTU may or may not be the same.

[0058] In some implementations, the partition tree schemes or structures used for lumana channels and chroma channels do not need to be the same. In other words, lumana channels and chroma channels may have separate coding tree structures or patterns. Furthermore, whether lumana channels and chroma channels use the same coding partition tree structure or different coding partition tree structures may depend on whether the slice being coded is a P slice, a B slice, or an I slice. For example, for an I slice, chroma channels and lumana channels may have separate coding partition tree structures or coding partition tree structure modes, while for P or B slices, lumana channels and chroma channels may share the same coding partition tree scheme. When separate coding partition tree structures or modes are applied, a lumana channel may be partitioned into CBs by one coding partition tree structure, and a chroma channel may be partitioned into chroma CBs by another coding partition tree structure.

[0059] Figure 7 shows an exemplary predefined 10-way partitioning structure / pattern that allows recursive partitioning to form a partitioning tree. The root block can start at a predefined level (e.g., from a base block at the 128x128 or 64x64 level). The exemplary partitioning structure in Figure 7 includes various 2:1 / 1:2 and 4:1 / 1:4 rectangular partitions. In some exemplary implementations, none of the rectangular partitions in Figure 7 are allowed to be further subdivided. The coding tree depth may be further defined to indicate the partitioning depth from the root node or root block. For example, the coding tree depth for the root node or root block may be set to 0, and after the root block is partitioned one more time according to Figure 7, the coding tree depth is increased by 1. In some implementations, only all square partitions within 710 may be allowed for recursive partitioning to the next level of the partitioning tree following the pattern in Figure 7.

[0060] In some other exemplary implementations for coding block partitioning, a quadtree structure may be used. Such quadtree partitioning can be applied hierarchically and recursively to any square partition. Whether the base block or intermediate block or partition is further quadtree-partitioned can be tailored to the various local characteristics of the base block or intermediate block / partition.

[0061] In several other examples, a triple partitioning scheme may be used to partition a base block or any intermediate block, as shown in Figure 8. The triple pattern can be implemented vertically, as shown in 802, or horizontally, as shown in 804. The exemplary partition ratio in Figure 8 is shown as 1:2:1, but other ratios can be predefined. In some implementations, two or more different ratios can be predefined. In some embodiments, the width and height of the partitions in the exemplary triple tree are always powers of 2 to avoid additional transformations.

[0062] The partitioning schemes described above can be combined in any way at different partitioning levels. For example, the quadtree and binary partitioning schemes described above can be combined to partition a base block into a quadtree-binary (QTBT) structure. In such a scheme, the base block or intermediate block / partition can be either quadtree partitioned or binary partitioned, if specified, according to a set of predefined conditions. A concrete example is shown in Figure 9, where the base block is first quadtree partitioned into four partitions, as indicated by 902, 904, 906, and 908. Each of the resulting partitions is then quadtree partitioned into four further partitions (e.g., 908), or binary partitioned into two further partitions at the next level (horizontally or vertically, for example, both symmetric, such as 902 or 906), or no partitioning at all (e.g., 904). As illustrated by the overall exemplary partitioning pattern in 910 and the corresponding tree structure / representation in 920, for square-shaped partitions, binary or quadtree partitions may be recursively allowed, where solid lines represent quadtree partitions and dashed lines represent binary partitions. A flag may be used for each binary partition node (non-leaf binary partition) to indicate whether the binary partition is horizontal or vertical. For example, as shown in 920, consistent with the partitioning structure in 910, a flag of "0" may represent a horizontal binary partition and a flag of "1" may represent a vertical binary partition. In the case of quadtree partitions, there is no need to indicate the partition type, as a quadtree partition always divides a block or partition both horizontally and vertically to produce four subblocks / partitions of equal size. In some implementations, a flag of "1" may represent a horizontal binary partition and a flag of "0" may represent a vertical binary partition.

[0063] In some exemplary implementations of QTBT, the quadtree and binary partitioning rule sets can be represented by the following predefined parameters and their associated corresponding functions. -CTU size: The size of the root node of the quadtree (the size of the base block). -MinQTSize: Minimum allowable quadtree leaf node size -MaxBTSize: Maximum allowable binary tree root node size -MaxBTDepth: Maximum allowable binary tree depth -MinBTSize: Minimum allowable size of a binary tree leaf node

[0064] In some exemplary implementations of the QTBT partitioning structure, the CTU size may be set as a 128x128 chromasample with two corresponding 64x64 blocks of chromasamples (when exemplary chroma subsampling is considered and used), MinQTSize may be set as 16x16, MaxBTSize may be set as 64x64, MinBTSize may be set as 4x4 (for both width and height), and MaxBTDepth may be set as 4. Quadratic partitioning may first be applied to the CTU to generate quadtree leaf nodes. A quadtree leaf node can have a size from its minimum allowable size (i.e., MinQTSize) which is 16x16 to 128x128 (i.e., CTU size). If a node is 128x128, it will not first be partitioned by a binary tree because its size exceeds MaxBTSize (i.e., 64x64). Otherwise, nodes that do not exceed MaxBTSize can be partitioned by a binary tree. In the example in Figure 9, the base block is 128x128. The base block can only be partitioned by a quadtree according to a predefined set of rules. The base block has a partitioning depth of 0. Each of the four resulting partitions is 64x64 and does not exceed MaxBTSize, and can be further partitioned by a quadtree or binary tree at level 1. The process continues. When the binary tree depth reaches MaxBTDepth (i.e., 4), further partitioning may not be considered. When a binary tree node has a width equal to MinBTSize (i.e., 4), further horizontal partitioning may not be considered. Similarly, when a binary tree node has a height equal to MinBTSize, further vertical partitioning is not considered.

[0065] In some exemplary implementations, the QTBT scheme described above may be configured to support the flexibility for lumens and chromens to have the same QTBT structure or different QTBT structures. For example, in the case of P-slice and B-slice, the lumens CTB and chromens CTB within a single CTU may share the same QTBT structure. However, for I-slice, the lumens CTB may be separated into CBs by a QTBT structure, and the chromens CTB may be separated into chromens CBs by another QTBT structure. This means that CUs may be used to refer to different color channels within an I-slice; for example, an I-slice may consist of a coding block for the lumens component or a coding block for two chromens, and a CU in a P-slice or B-slice may consist of coding blocks for all three color components.

[0066] The various CB partitioning schemes and further partitioning of CBs into PBs described above can be combined in any way. The following specific implementations are given as non-limiting examples.

[0067] In some implementations, a set of intra-prediction modes (replaceable as “intra-modes”) may include a predefined number of directional intra-prediction modes. These intra-prediction modes may correspond to a predefined number of directions along which out-of-block samples are selected as predictions for samples predicted within a particular block. In other specific exemplary implementations, eight main directional modes corresponding to angles from 45 to 207 degrees with respect to the horizontal axis may be supported and predefined. In some other implementations of intra-prediction, the directional intra-modes may be further extended to angle sets with finer granularity to further leverage more types of spatial redundancy in directional textures. For example, the 8-angle implementation described above may be configured to give eight nominal angles referred to as V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED, and D67_PRED, as shown in Figure 11, with a predefined number (e.g., seven) finer angles added for each nominal angle. In such extensions, a larger total number of directional angles (e.g., 56 in this example) corresponding to the same number of predefined directional intra-modes may be available for intra-prediction. The predicted angle may be represented by the nominal intra-angle plus an angular delta. In the specific example above, which has seven finer angular directions for each nominal angle, the angular delta may be -3 to 3 times the 3-degree step size. As shown in Figure 10, some angular scheme can be used that has 65 different predicted angles. In some implementations, eight nominal modes are first signaled along with five non-angular smoothing modes, and then an index is further signaled to indicate the angular delta for the corresponding nominal angle if the current mode is an angular mode. In some implementations, all 56 directional intra-prediction modes can be implemented using an integrated directional predictor that projects each pixel to a reference subpixel position and interpolates the reference pixel with a two-tap bilinear filter, in order to implement the directional prediction modes in a comprehensive manner.

[0068] Next, the residuals of either the intra-prediction block or the inter-prediction block are transformed, followed by the quantization of the transformation coefficients. For the purpose of transformation, both intra-coded and inter-coded blocks may be further partitioned into multiple transformation blocks before the transformation (the term "unit" is usually used to represent a set of three color channels, and although, for example, a "coding unit" would include a lumen-coding block and a chroma-coding block, it is sometimes used interchangeably as a "transformation unit"). In some implementations, the maximum partitioning depth of the coded block (or prediction block) may be specified (the term "coded block" may be used interchangeably with "coding block"). For example, such partitioning may not exceed two levels. Partitioning prediction blocks into transformation blocks may be handled differently between intra-prediction blocks and inter-prediction blocks. However, in some implementations, such partitioning may be similar between intra-prediction blocks and inter-prediction blocks.

[0069] In some exemplary implementations, for interconnected blocks, transformation unit partitioning may be performed recursively using a partitioning depth up to a predefined number of levels (e.g., 2 levels). Partitioning may stop or continue recursively at any level for any subpartition. For example, one block may be partitioned into four quadtree subblocks, one of which is further partitioned into four second-level transformation blocks, while the partitioning of the other subblocks stops after the first level, resulting in a total of seven transformation blocks of two different sizes. In some implementations, transformation partitioning may support 1:1 (square), 1:2 / 2:1, and 1:4 / 4:1 transformation block shapes and sizes ranging from 4x4 to 64x64. In some exemplary implementations, if the coding block is smaller than or equal to 64x64, transformation block partitioning may be applied only to the lumen component (in other words, the chroma transformation block is the same as the coding block under that condition). Alternatively, if the coding block width or height is greater than 64, both the lumen coding block and the chromen coding block may be implicitly divided into multiples of min(W, 64) × min(H, 64) and min(W, 32) × min(H, 32), respectively.

[0070] Each of the transformation blocks described above can then undergo a linear transformation. A linear transformation essentially moves the residuals within the transformation block from the spatial domain to the frequency domain. In some implementations of the actual linear transformation, multiple transformation sizes (ranging from 4 to 64 points for each of the two dimensions) and transformation shapes (square, rectangles with width / height ratios of 2:1 / 1:2 and 4:1 / 1:4) may be allowed to support the exemplary extended coding block partitions described above.

[0071] When considering actual linear transformations, in some exemplary implementations, the 2-D transformation process may involve the use of hybrid transformation kernels (which may consist of different 1-D transformations for each dimension of a coded residual transformation block). Exemplary 1-D transformation kernels may include, but are not limited to, a) 4-point, 8-point, 16-point, 32-point, and 64-point DCT-2, b) 4-point, 8-point, and 16-point asymmetric DSTs (DST-4, DST-7) and their inverted versions, and c) 4-point, 8-point, 16-point, and 32-point identity transformations. The selection of the transformation kernel to be used for each dimension may be based on a rate-distortion (RD) criterion.

[0072] In some exemplary implementations, the availability of a hybrid transformation kernel for a particular primary transformation implementation may be based on the transformation block size and prediction mode. For chroma components, transformation type selection may be performed implicitly. For example, in the case of intra-predictive residuals, the transformation type may be selected according to the intra-predictive mode. For inter-predictive residuals, the transformation type for the chroma block may be selected according to the transformation type selection of the rumor block located in the same place. Therefore, for chroma components, there is no signaling of the transformation type in the bitstream.

[0073] In some implementations, for the intra-predictive residual block of the chroma color component, a quadratic transformation method, i.e., an intra-quadratic transformation (IST), may be applied to the linear transformation coefficient block before quantization is applied in the encoder. Therefore, before the inverse linear transformation is applied in the decoder, an inverse quadratic transformation may be applied to the inversely quantized transformation coefficient block. IST is not applied to the chroma color component. The use of IST in the encoding and decoding process is shown in Figure 12.

[0074] In some implementations using IST, an inseparable transformation process is applied. To apply a forward inseparable transformation to a specific region of an input transformation coefficient block consisting of N samples, the N samples are first scanned using a coefficient scan order according to the relative coordinates of each sample in the input N samples to create an N×1 vector.

number

number

number

[0075] In some implementations for applying inverse non-separable transforms, first, an inversely quantized transform coefficient block is given as input, and then, based on the transform block size, a specific region of the inversely quantized transform coefficient block is identified.

number

number

number

number

number

[0076] In some implementations, the input to the forward IST is a coefficient vector consisting of low-frequency linear transformation coefficients in a zigzag scan. Depending on the block size, either a 16-point or 64-point inseparable quadratic transformation can be selected. When the minimum values ​​of the linear transformation width and height are less than 8, the 16-point IST is used, and the low-frequency linear transformation coefficients refer to the first 16 linear transformation coefficients in the zigzag scan order. When both the linear transformation width and height are 8 or greater, the 64-point IST is applied, and the low-frequency linear transformation coefficients refer to the first 64 linear transformation coefficients in the zigzag scan order. The 16-point inseparable transformation uses an 8x16 transformation kernel, and the 64-point inseparable transformation uses a 32x64 transformation kernel. Furthermore, when the IST is applied, high-frequency transformation coefficients not processed by the quadratic transformation are set to zero.

[0077] In some implementations, a total of 12 quadratic transformation sets (or IST sets) may be defined, each containing three quadratic transformation kernels. For each intracoded block, first, the nominal intra-prediction mode and the primary transformation type are identified, and then an IST set is selected based on Table 1. In some implementations, for Paeth prediction mode and recursive intra-prediction mode, ISTs are neither applied nor signaled. [Table 1]

[0078] In some implementations, given an IST set, there may be four encoder options: 1) no quadratic transformation, 2) quadratic transformation using the first transformation kernel from the given IST set, 3) a quadratic transformation kernel using the second transformation kernel from the given IST set, and / or 4) a quadratic transformation kernel using the third transformation kernel from the given IST set. In some implementations, the encoder signals the selection using the syntax element ist_idx. In the decoder, the value of the syntax element ist_idx is first parsed. Then, given the IST set and value associated with ist_idx, the quadratic transformation kernel is identified. This syntax element ist_idx is signaled for each rumor transformation block after the signaling of the primary transformation type. ist_idx signaling occurs when all of the following conditions are true: the current block is an intra-coded luma transform block; the linear transform type is DCT in both dimensions or ADST in both dimensions; the intra-prediction mode is neither Paeth prediction mode nor recursive intra-prediction mode; the transform partitioning depth is 0; and the end-of-block (EOB) position falls within the low-frequency transform coefficient region to which a quadratic transform is applicable. In some implementations, the entropy coding context of ist_idx is derived based on the transform block size.

[0079] In some implementations, a quadratic transformation may be performed on the linear transformation coefficients. For example, the Low Frequency Non-Separated Transform (LFNST), also known as the Reduced Quadratic Transform, can be applied between the forward linear transformation and quantization (in the encoder) and between the inverse quantization and inverse linear transformation (on the decoder side) to further decorrelate the linear transformation coefficients, as shown in Figure 13. Thus, the LFNST can proceed to the quadratic transformation by taking a portion of the linear transformation coefficients, e.g., the low-frequency portion (and therefore "reduced" from the full set of linear transformation coefficients in the transformation block). In an exemplary LFNST, a 4x4 non-separated transformation or an 8x8 non-separated transformation may be applied depending on the size of the transformation block. For example, a 4x4 LFNST may be applied to a small transformation block (e.g., min(width, height) < 8), and an 8x8 LFNST may be applied to a larger transformation block (e.g., min(width, height) > 8). For example, if an 8x8 transformation block undergoes a 4x4 LFNST, only the low-frequency 4x4 portion of the 8x8 linear transformation coefficients undergoes a further quadratic transformation.

[0080] As specifically shown in Figure 13, the transformation block can be 8×8 (or 16×16). Thus, the forward linear transformation 1305 of the transformation block yields an 8×8 (or 16×16) linear transformation coefficient matrix 1304, where each square unit represents a 2×2 (or 4×4) portion. The input to the forward LFNST may not be the entire 8×8 (or 16×16) linear transformation coefficients. For example, a 4×4 (or 8×8) LFNST may be used for the quadratic transformation. Thus, as shown in the shaded portion (upper left) 1306, only the 4×4 (or 8×8) low-frequency linear transformation coefficients of the linear transformation coefficient matrix 1304 may be used as input to the LFNST. The rest of the linear transformation coefficient matrix may not undergo the quadratic transformation. Therefore, after the quadratic transformation, the portion of the linear transformation coefficients that underwent LFNST becomes the quadratic transformation coefficient, while the remaining portion that did not undergo LFNST (for example, the unshaded portion of matrix 1304) retains the corresponding linear transformation coefficient. In some exemplary implementations, the remaining portion that did not undergo the quadratic transformation may all be set to zero coefficients.

[0081] An example of applying the non-separated transform used in LFNST is described below. To apply an exemplary 4×4 LFNST, the 4×4 input block X (representing the 4×4 low-frequency portion of the linear transform coefficient block, such as the shaded portion 1306 of the linear transform matrix 1304 in Figure 13) can be shown as follows:

number

[0082] This 2D input matrix is ​​first a vector in an illustrative order.

number

number

Number

Number

Number

[0083] The exemplary LFNST described above is based on a direct matrix multiplication technique to apply a non-separable transform so that it is implemented in a single pass without multiple iterations. In some further exemplary implementations, the dimension of the non-separable conversion matrix (T) for the exemplary 4×4 LFNST can be further reduced to minimize the computational complexity and memory space requirements for storing the conversion coefficients. Such an implementation can be referred to as a reduced non-separable transform (RST). More specifically, the main idea of RST is to map an N-dimensional vector (where N is 4×4 = 16 in the above example, but may be equal to 64 for an 8×8 block) to an R-dimensional vector in a different space, where N / R (R < N) represents the dimension reduction coefficient. Thus, instead of an N×N conversion matrix, the RST matrix becomes an R×N matrix as follows.

Number

[0084] In the matrix described above, the R rows of the transformation matrix are the R reduced basis vectors in the N-dimensional space. Thus, the transformation transforms an N-dimensional input vector into a reduced R-dimensional output vector. Therefore, as shown in Figure 13, the quadratic transformation coefficient (shaded square, 1308) transformed from the linear coefficient 1306 has its dimension reduced by the coefficient or N / R. The three squares around 1308 (1309, 1310, 1311) can be zero-padding.

[0085] The inverse transform matrix of an RST can be the transpose of its forward transform. In the case of an exemplary 8x8 LFNST (for the sake of more diverse explanation here, in contrast to the 4x4 LFNST described above), an exemplary reduction factor of 4 may be applied, and thus the 64x64 direct inseparable transform matrix is ​​reduced accordingly to a 16x64 direct matrix. Furthermore, in some implementations, only a portion, rather than the entirety, of the input linear coefficients may be linearized to the input vector for the LFNST. For example, only a portion of the exemplary 8x8 input linear transform coefficients may be linearized to the X vector described above. In a specific example, of the four 4x4 quadrants of the 8x8 linear transform coefficient matrix, the lower right (high frequency coefficients) quadrant may be excluded, and only the other three quadrants may be linearized to a 48x1 vector using a predefined scan order, rather than a 64x1 vector. In such implementations, the inseparable transform matrix may be further reduced from 16x64 to 16x48.

[0086] Therefore, the exemplary reduced 48×16 inverse RST matrix can be used on the decoder side to generate the top-left, top-right, and bottom-left 4×4 quadrants of the 8×8 core (linear) transformation coefficients. Specifically, when a further reduced 16×48 RST matrix is ​​applied instead of a 16×64 RST with the same transformation set configuration, the inseparable quadratic transformation will take as input 48 vectorized matrix elements from three 4×4 quadrant blocks of the 8×8 linear coefficient block, excluding the bottom-right 4×4 block. In such an implementation, the excluded bottom-right 4×4 linear transformation coefficients will be ignored in the quadratic transformation. This further reduced transformation transforms the 48×1 vector into a 16×1 output vector, which is then inversely scanned into a 4×4 matrix to fill 1308 in Figure 13. The three squares of quadratic transformation coefficients surrounding 1308 (1309, 1310, 1311) can be zero-padding.

[0087] With the help of dimensionality reduction in such RSTs, the memory usage required to store all LFNST matrices is reduced. In the example above, memory usage can be reduced from 10 KB to 8 KB with a fairly small performance degradation compared to an implementation without dimensionality reduction.

[0088] In some implementations, to reduce complexity, LFNST may be further restricted to apply only when all coefficients outside the linear transformation coefficient portion to which LFNST should be applied (e.g., outside the 1306 portion of 1304 in Figure 13) are not significant. Thus, all linear-only transformation coefficients (e.g., the unshaded portion of the linear coefficient matrix 1304) can be close to 0 when LFNST is applied. Such a constraint allows for adjustment of the signaling of LFNST indices at the last significant position and thus avoids any extra coefficient scans that might be required to check for significant coefficients at specific positions when this constraint is not applied. In some implementations, the worst-case handling of LFNST (with respect to pixel-by-pixel multiplication) may restrict inseparable transformations for 4x4 and 8x8 blocks to 8x16 and 8x48 transformations, respectively. In these cases, when LFNST is applied, the last significant scan position must be less than 8 for other sizes less than 16. For blocks with shapes of 4×N, N×4, and N>8, the above restriction means that LFNST is applied only once to the top-left 4×4 region. When LFNST is applied, all linear coefficients are zero, so in such cases the number of operations required for the linear transformation is reduced. From the encoder's perspective, coefficient quantization can be simplified when the LFNST transformation is tested. Rate-distortion-optimized quantization (RDO) must be performed on at most the first 16 coefficients (in scan order), and the remaining coefficients can be forced to zero.

[0089] In some exemplary implementations, the available RST kernels may be specified as several transformation sets, each containing several inseparable transformation matrices. For example, for use in LFNST, there may be a total of four transformation sets and two inseparable transformation matrices (kernels) for each transformation set. These kernels may be pre-trained offline and are therefore data-driven. Offline-trained transformation kernels may be stored in memory or hardcoded on the encoding or decoding device for use during encoding / decoding. The selection of a transformation set during encoding or decoding may be determined by the intra-prediction mode. Mappings from intra-prediction modes to transformation sets may be predefined. Examples of such predefined mappings are shown in Table 2. For example, when one of the three cross-component linear model (CCLM) modes (INTRA_LT_CCLM, INTRA_T_CCLM, or INTRA_L_CCLM) is used for the current block (i.e., 81 ≤ predModeIntra ≤ 83), transformation set 0 may be selected for the current chroma block. For each transformation set, the selected inseparable quadratic transformation candidates may be further specified by an explicitly signaled LFNST index. For example, the index may be signaled in the bitstream once per intraCU after the transformation coefficient. [Table 2]

[0090] In the exemplary implementation described above, LFNST is restricted to being applicable only when all coefficients outside the first coefficient subgroup or portion are not significant; therefore, LFNST index coding depends on the position of the last significant coefficient. Also, the LFNST index is context-coded, but independent of the intra-prediction mode, and only the first bin is context-coded. Furthermore, LFNST can be applied to intraCUs in both intra-slice and inter-slice, and to both lumens and chromens. When dual-tree is enabled, LFNST indices for lumens and chromens can be signaled separately. In the case of inter-slices (where dual-tree is disabled), a single LFNST index can be signaled and used for both lumens and chromens.

[0091] In some implementations, block-based 2-D data transformations can be performed using two methods: i) separable transformations and ii) inseparable transformations. In separable transformations, each column and row of a block is considered a 1-D signal, and a 1-D transformation is used to map the blocks of data to a set of coefficients. The 1-D transformations used in each direction may be the same or different. In the case of inseparable transformations, blocks are generally ordered as 1-D vectors by lexicographically ordering the columns or rows of the block. The drawback of this is that inseparable transformations may require more memory to hold the entries for the transformation matrix, and multiplication of large matrices can generally be too complex for hardware implementations. Therefore, in some implementations, separable transformations are attractive. However, since separable transformations only utilize correlation with columns or rows, they are costly, and therefore, the compression performance of separable transformations is lower around directional edges compared to inseparable transformations.

[0092] In some exemplary implementations, when the Intra Sub-Partitioning (ISP) mode is selected, LFNST may be disabled and the RST index may not be signaled because even if RST is applied to all feasible partition blocks, the performance improvement is likely to be minimal. Furthermore, disabling RST for ISP prediction residuals can reduce encoding complexity. In some further implementations, when the Multiple linear regression Intra Prediction (MIP) mode is selected, LFNST may also be disabled and the RST index may not be signaled.

[0093] Considering that larger CUs than 64x64 (or any other predefined size representing the maximum transformation block size) are implicitly partitioned (e.g., TU tiling) by the existing maximum transformation size limit (e.g., 64x64), LFNST index lookup can quadruple data buffering for a given number of decode pipeline stages. Therefore, in some implementations, the maximum size allowed for LFNST may be limited to, for example, 64x64. In some implementations, LFNST may only be enabled using DCT2 as a primary transformation.

[0094] In some other implementations, an intra-secondary transform (IST) for lumen components is provided by defining, for example, 12 sets of quadratic transforms, each with three kernels. An intra-mode dependent index may be used for transform set selection. Kernel selection within a set may be based on signaled syntax elements. IST can be enabled when either DCT2 or ADST is used as both horizontal and vertical primary transforms.

[0095] In some implementations, a 4x4 or 8x8 inseparable transform may be selected depending on the block size. A 4x4 IST may be selected if min(tx_width, tx_height) < 8. For larger blocks, an 8x8 IST may be used. Here, tx_width and tx_height correspond to the transform block width and height, respectively. The input to the IST may be low-frequency linear transform coefficients in a zigzag scan order.

[0096] In some other exemplary implementations, the number of kernel sets may be other than 12, for example, there may be 14 sets of quadratic transform kernels. The mapping between the quadratic kernel sets and the various intra-prediction modes may be predetermined or pre-configured. For example, a particular intra-prediction mode may be mapped to one of 12 or 14 sets. Each kernel set may contain a number (or quantity) of quadratic kernels other than 3. For example, there may be 6 quadratic kernels in each quadratic transform kernel set. The number of kernels in each set may be determined based on a trade-off between the overhead associated with increasing the number of quadratic kernels and the potential additional coding gain. In some implementations, the number of kernels in each kernel set may be between 3 and 6. The coding gain of having 6 arbitrarily chosen quadratic transform kernels in each kernel set may be sufficiently small from a practical and statistical standpoint compared to the increase in coding overhead.

[0097] In some implementations, there are several problems or challenges associated with transform set signaling that can be addressed or improved to increase coding / decoding efficiency. In a non-limiting example, the selection of transform sets for quadratic transforms depends on the intra-predictive mode, which limits the encoder's flexibility in selecting the optimal transform set for the residual block. This disclosure describes various embodiments for improving transform set signaling, where, for any given intra-mode, at least one or more transform sets are available for selection to address at least one of the problems or challenges described above, to improve coding / decoding efficiency, and to advance video codec technology.

[0098] The various embodiments and / or implementations described herein may be performed separately or in any order, and may be applicable to decoding, encoding, or streaming. Furthermore, each of the methods (or embodiments), encoders, and decoders may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). One or more processors execute a program stored on a non-temporary computer-readable medium. In this specification, the term "block" may be interpreted as a prediction block, coding block, or coding unit (CU).

[0099] Figure 14 shows a flowchart 1400 of an exemplary method for improving the signaling of transformation sets, following the principles underlying the implementation described above. The exemplary decoding method flow may include some or all of the following steps, namely, S1410, receiving a coded video bitstream; S1420, extracting a set of transformation coefficients for the current block based on the coded video bitstream; S1430, determining the transformation set index of the current block in intra-predictive mode based on syntax elements explicitly signaled in the coded video bitstream, where the transformation set index indicates a transformation set among multiple transformation sets; S1440, identifying a transformation set according to the transformation set index; S1450, performing an inverse transformation using the set of transformation coefficients and the identified transformation set to obtain a residual block of the current block; and / or S1460, reconstructing the current block based on the residual block. The exemplary method terminates at S1499.

[0100] In this disclosure, the conversion set index can be an integer for indexing one conversion set from a list / group of conversion sets. For example, when there are N conversion sets in the list of conversion sets, the conversion set index can be 0, indicating the second conversion set in the list of conversion sets; 1, indicating the second conversion set in the list of conversion sets; ..., N-1, indicating the Nth conversion set in the list of conversion sets. In some implementations, a conversion set may contain one or more conversion kernels, and when there are two or more conversion kernels in a conversion set, the selection of conversion kernels in the conversion set may be signaled or implicitly derived (for example, based on coded information).

[0101] Other embodiments may include other exemplary methods for improving the signaling of transformation sets. These methods may include some or all of the following steps: receiving a coded video bitstream; extracting a set of transformation coefficients for the current block based on the coded video bitstream; extracting a transformation set index for the current block in any given intra-prediction mode based on the coded video bitstream; identifying a transformation set according to the transformation set index; identifying a transformation kernel in the identified transformation set based on the coded video bitstream; performing an inverse transformation of the set of transformation coefficients based on the identified transformation kernel to obtain a residual block of the current block; and / or reconstructing the current block based on the residual block.

[0102] This disclosure describes other methods for improving the signaling of transformation sets. These methods may include, in whole or in part, the following steps: transforming the residual block of an intra-predicted video block (or current block) using at least one primary transformation kernel to generate a set of primary transformation coefficients; selecting a secondary transformation set from a group of secondary transformation sets for any given intra-prediction mode; selecting a transformation kernel from the secondary transformation set; performing a transformation based on the set of primary transformation coefficients to generate a set of secondary transformation coefficients; and / or encoding the set of secondary transformation coefficients and the transformation set index corresponding to the secondary transformation set into an encoded video bitstream associated with the current block.

[0103] In any part or combination of the above-described implementations, the identified transformation set includes a quadratic transformation set, and / or obtaining the residual block of the current block by performing an inverse transformation of the set of transformation coefficients based on the identified transformation kernel includes performing an inverse quadratic transformation of the set of transformation coefficients based on the identified transformation kernel of the quadratic transformation set to generate a set of linear transformation coefficients of the current block, and / or performing an inverse linear transformation of the set of linear transformation coefficients based on the linear transformation kernel to generate the residual block of the current block.

[0104] In any part or combination of the above-described implementations, extracting a transformation set index for the current block includes extracting a transformation set index corresponding to multiple transformation sets in response to the current block being in intra-predictive mode.

[0105] In any part or combination of the above-described implementations, extracting the transformation set index for the current block based on the coded video bitstream involves entropy decoding the coded video bitstream to obtain the transformation set index based on the context that depends on the coded information.

[0106] In any part or combination of the above-described implementations, the coded information includes at least one of the following: intra-prediction mode, inter-prediction mode, transformation partitioning mode or depth, transformation coefficient value, last non-zero position or end of block (EOB), number of non-zero transformation coefficients, or quantization parameters.

[0107] In any part or combination of the above-described implementations, extracting a transformation set index for the current block based on a coded video bitstream includes obtaining the transformation set index by entropy decoding the coded video bitstream in response to a condition being met, wherein the condition depends on coded information.

[0108] In any part or combination of the above-described implementations, the coded information includes at least one of the following: intra-prediction mode, inter-prediction mode, transformation partitioning mode or depth, transformation coefficient value, last non-zero position or end of block (EOB), number of non-zero transformation coefficients, or quantization parameters.

[0109] In any part or combination of the above-described implementations, the first set of transformations for the first block overlaps with the second set of transformations for the second block, and the first and second blocks have at least one of the following: different intra-prediction modes, different transformation partitioning modes or depths, different transformation coefficient values, different last non-zero positions or EOBs, or different numbers of non-zero transformation coefficients.

[0110] In any part or combination of the above-described implementations, the multiple transformation sets include at least one of all separable transformation sets, all inseparable transformation sets, or a list of separable and inseparable transformation sets.

[0111] In any part or combination of the above-described implementations, the multiple transformation sets include at least one of all secondary transformation sets, all primary transformation sets, or a combination of secondary and primary transformation sets.

[0112] In any part or combination of the above-described implementations, multiple transformation sets include all primary transformation sets, and secondary transformations are implicitly enabled in response to the transformation set index belonging to a first subset, and / or implicitly disabled in response to the transformation set index belonging to a second subset.

[0113] In any part or combination of the above-described implementation forms, identifying a translation kernel in an identified set of translations includes identifying a translation kernel in an identified set of translations, wherein the translation kernels in the identified set of translations are sorted based on coded information.

[0114] In any part or combination of the above-described implementation forms, identifying a conversion set according to a conversion set index includes identifying a conversion set according to a conversion set index, wherein the conversion sets are sorted based on coded information.

[0115] In various embodiments of this disclosure, a transformation set represents a group of multiple transformation kernels / bases and a single transformation kernel / base.

[0116] In various embodiments, when selecting a set of transformations for a residual block in any given intra-mode, multiple sets of transformations may be available, and an index of the selected set of transformations, for example, a transformation set index (tx_set_idx), may be signaled.

[0117] In some implementations, signaling a transformation set index to indicate one of several transformation sets may only be applicable when the current coding block is being intracoded.

[0118] In some implementations, the transformation set index (tx_set_idx) can be entropy coded using a context value that depends on coded information, which may include the intra-prediction mode, inter-prediction mode, transformation partitioning mode / depth, transformation coefficient values, the last non-zero position (or EOB), the number of non-zero transformation coefficients, or some or all of the quantization parameters.

[0119] In some implementations, the transformation set index (tx_set_idx) is conditionally entropy-coded (meaning the syntax is entropy-coded for some conditions and not for others), and the conditions depend on coded information which may include some or all of the intra-prediction mode, transformation partitioning mode / depth, transformation coefficient values, last non-zero position (or EOB), and the number of non-zero transformation coefficients. For example, if the intra-prediction mode of the current block is the first predefined mode, the condition is met and the transformation set index (tx_set_idx) is entropy-coded; if the intra-prediction mode of the current block is the second predefined mode, the condition is not met and the transformation set index (tx_set_idx) is not entropy-coded.

[0120] In some implementations, for different intra-prediction modes, different transformation partitioning modes / depths, and / or different transformation coefficient values, and / or different last non-zero positions (or EOBs), and / or different numbers of non-zero transformation coefficients, the candidate transformation sets may have overlapping transformation sets. For example, if the intra-prediction mode of the current block is the first mode, the candidate transformation sets for the current block include 1, 3, 5, and 6; if the intra-prediction mode of the current block is the second mode, the candidate transformation sets for the current block include 4, 5, 6, and 7, where 5 and 6 are overlapping transformation sets for both the first and second modes.

[0121] In some implementations, this method applies when the transformation set is an inseparable transformation set; or when the transformation set is a separable transformation set; or when the transformation set is a mixture of separable and inseparable transformation sets. In some implementations, the transformation set refers to a set of only quadratic transformation sets, or a set of only primary transformation sets, or a combination of primary and quadratic transformation sets. In some implementations where the transformation set refers to a set of primary transformations, the quadratic transformation is implicitly enabled or disabled for any signaled index belonging to the set.

[0122] In some implementations, the transformation candidates in the transformation set can be sorted for better entropy coding, given coded information. The coded information may include the intra-prediction mode, inter-prediction mode, transformation partitioning mode / depth, transformation coefficient values, the last non-zero position (or EOB), the number of non-zero transformation coefficients, and some or all of the quantization parameters. In some implementations, the transformation set can be sorted for better entropy coding, given coded information. The coded information may include the intra-prediction mode, inter-prediction mode, transformation partitioning mode / depth, transformation coefficient values, the last non-zero position (or EOB), the number of non-zero transformation coefficients, and some or all of the quantization parameters.

[0123] Various embodiments of this disclosure may include methods for encoding a current block into a video bitstream, performed by an encoder, which may include the reverse processing as any part or all of the processing described for a decoder.

[0124] Various embodiments of this disclosure may include methods for encoding a current block for streaming video, performed by one or more electronic devices (e.g., streaming media players), which include any or all of the processing for a decoder and / or any or all of the processing described for an encoder.

[0125] The operations described above may be combined or arranged in any quantity or order as desired. Two or more of the steps and / or operations may be performed in parallel. The embodiments and implementations in this disclosure may be used separately or combined in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders may be implemented by processing circuits (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-temporary computer-readable medium. The embodiments in this disclosure may be applied to rumor blocks or chroma blocks. The term block may be interpreted as a prediction block, coding block, or coding unit, i.e., CU, and the term block may also be used to refer to a transformation block. In the following sections, when referring to block size, it may refer to the width or height of the block, the maximum width and height, the minimum width and height, the area size of the block (width * height), or the aspect ratio (width:height, or height:width).

[0126] The techniques described above can be implemented as computer software using computer-readable instructions and can be physically stored on one or more computer-readable media. For example, Figure 15 shows a computer system (1800) suitable for carrying out a particular embodiment of the disclosed subject matter.

[0127] Computer software may be coded using any suitable machine code or computer language, which may undergo assembly, compilation, linking, or similar mechanisms to create code containing instructions that can be executed directly or via interpretation, microcode execution, or the like by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc.

[0128] Instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, and Internet of Things devices.

[0129] The components shown in Figure 15 for the computer system (1800) are illustrative in nature and are not intended to imply any limitation on the scope of use or functionality of computer software implementing embodiments of the present disclosure. Furthermore, the configuration of the components should not be construed as having any dependencies or requirements with respect to any one or combination of components shown in the exemplary embodiments of the computer system (1800).

[0130] A computer system (1800) may include some human interface input device. The input human interface device may include one or more of the following (only one of each is shown): a keyboard (1801), a mouse (1802), a trackpad (1803), a touchscreen (1810), a data glove (not shown), a joystick (1805), a microphone (1806), a scanner (1807), and a camera (1808).

[0131] The computer system (1800) may also include some human interface output device. Such a human interface output device may stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such a human interface output device may include tactile output devices (e.g., tactile feedback via a touchscreen (1810), data glove (not shown), or joystick (1805), although there may also be tactile feedback devices that do not function as input devices), audio output devices (e.g., speakers (1809), headphones (not shown)), visual output devices (e.g., screens (1810), including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capability, each with or without tactile feedback capability, some of which may be capable of outputting two-dimensional visual output or output beyond three dimensions through means such as stereoscopic output), virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), as well as printers (not shown).

[0132] The computer system (1800) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1820) having media such as CD / DVD (1821), thumb drives (1822), removable hard drives or solid-state drives (1823), legacy magnetic media such as tapes and floppy disks (not shown), and dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0133] Those skilled in the art will also understand that the term “computer-readable medium” as used in relation to the subject matter disclosed herein does not include transmission media, carrier waves, or other transient signals.

[0134] A computer system (1800) may also include an interface (1854) to one or more communication networks (1855). These networks may be, for example, wireless, wired, or optical. Networks may further be local, wide-area, metropolitan, vehicle and industrial, real-time, or latency-tolerant. Examples of networks include cellular networks such as Ethernet®, wireless LAN, GSM®, 3G, 4G, 5G, and LTE; wired or wireless wide-area digital television networks such as cable television, satellite television, and terrestrial television; and local area networks such as vehicle and industrial networks including CAN buses.

[0135] The aforementioned human interface devices, human-accessible memory devices, and network interfaces may be mounted on the core (1840) of the computer system (1800).

[0136] The core (1840) may include one or more central processing units (CPUs) (1841), graphics processing units (GPUs) (1842), dedicated programmable processing units in the form of field-programmable gate areas (FPGAs) (1843), hardware accelerators for specific tasks (1844), graphics adapters (1850), etc. These devices may be connected via a system bus (1848) along with read-only memory (ROM) (1845), random access memory (1846), internal mass storage devices such as internal hard drives and SSDs (1847) that are not accessible to the user. In some computer systems, the system bus (1848) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (1848) or via a peripheral bus (1849). For example, a screen (1810) may be connected to a graphics adapter (1850). Peripheral bus architectures include PCI, USB, and others.

[0137] A computer-readable medium may contain computer code for performing various computer implementation operations. The medium and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may be of a type that is well known and available to those skilled in the computer software technology.

[0138] While this disclosure describes several exemplary embodiments, there are many variations, substitutions, and alternative equivalents that fall within the scope of this disclosure. Therefore, those skilled in the art will recognize that numerous systems and methods not expressly shown or described herein, but embodying the principles of this disclosure and thus falling within the spirit and scope of this disclosure, can be devised.

Claims

1. A method for decoding the current block of the current frame in a coded video bitstream, the method being: A device including a memory storing instructions and a processor communicating with the memory receives the coded video bitstream, The device takes the steps of: extracting a set of conversion coefficients for the current block based on the coded video bitstream; The step of the device determining a transformation set index for the current block in intra-predictive mode based on syntax elements explicitly signaled in the coded video bitstream, wherein the transformation set index indicates a transformation set among a plurality of transformation sets; The device identifies the conversion set according to the conversion set index, The device performs an inverse transform using the set of transformation coefficients and the identified transformation set to obtain a residual block for the current block, The device includes the step of reconstructing the current block based on the residual block, A method that includes this.

2. The identified transformation set includes a secondary transformation set, The step of performing the inverse transform using the set of transformation coefficients and the identified transformation set to obtain the residual block for the current block is: This includes performing an inverse quadratic transformation of the set of transformation coefficients based on the set of quadratic transformations to generate a set of linear transformation coefficients for the current block, The method according to claim 1.

3. The step of determining the conversion set index for the current block is: In response to the current block being in the intra prediction mode, the process includes extracting the transformation set index corresponding to the plurality of transformation sets. The method according to claim 1.

4. The step of determining the conversion set index for the current block is: This includes entropy decoding the coded video bitstream to obtain the transformation set index based on the context that depends on the coded information, The method according to claim 1.

5. The coded information includes at least one of the following: intra-prediction mode, inter-prediction mode, transformation partitioning mode or depth, transformation coefficient value, last non-zero position or end of block (EOB), number of non-zero transformation coefficients, or quantization parameters. The method according to claim 4.

6. The step of determining the conversion set index for the current block is: Entropy decoding the coded video bitstream to obtain the conversion set index in response to the fulfillment of a condition, wherein the condition depends on coded information. The method according to claim 1.

7. The coded information includes at least one of the following: intra-prediction mode, inter-prediction mode, transformation partitioning mode or depth, transformation coefficient value, last non-zero position or end of block (EOB), number of non-zero transformation coefficients, or quantization parameters. The method according to claim 6.

8. A first set of transformations for a first block overlaps with a second set of transformations for a second block, and the first and second blocks have at least one of different intra-prediction modes, different transformation division modes or depths, different transformation coefficient values, different last non-zero positions or EOBs, or different numbers of non-zero transformation coefficients. The method according to claim 1.

9. The plurality of transformation sets include at least one of all separable transformation sets, all non-separable transformation sets, or a list of separable transformation sets and non-separable transformation sets. The method according to claim 1.

10. The aforementioned set of transformations includes at least one of all quadratic transformation sets, all primary transformation sets, or a combination of a set of quadratic and a set of primary transformations. The method according to claim 1.

11. The aforementioned set of transformations includes all primary transformation sets, In response to the conversion set index belonging to the first subset, the secondary conversion is implicitly enabled, or In response to the conversion set index belonging to the second subset, the secondary conversion is implicitly disabled. The method according to claim 1.

12. Identifying a translation kernel in the identified translation set, further comprising sorting the translation kernels in the identified translation set based on coded information, The method according to claim 1.

13. The step of identifying the conversion set according to the conversion set index is: Identifying the conversion set according to the conversion set index, wherein the conversion set is sorted based on coded information, The method according to claim 1.

14. A device for decoding the current block of the current frame in a coded video bitstream, Memory for storing instructions, A processor that communicates with the memory, wherein when the processor executes the instruction, the processor is configured to cause the device to perform the method described in any one of claims 1 to 13, A device that includes this.

15. A computer program for causing a computer to perform the method described in any one of claims 1 to 13.

Images