Method, device, medium, and program for video decoding

Abstract

To provide a method of video decoding at a video decoder.SOLUTION: In the method, first and second high level syntax (HLS) elements are received. The first HLS element indicates whether an explicit multiple transform selection (MTS) is enabled for an intra coded block. The second HLS element indicates whether the explicit MTS is enabled for an inter coded block. The first and second HLS elements control the same set of coding blocks that include the intra coded block and the inter coded block. An implicit MTS is enabled for the intra coded block when the first HLS element indicates that the explicit MTS is disabled for the intra coded block and the second HLS element indicates that the explicit MTS is enabled for the inter coded block.SELECTED DRAWING: Figure 35

Description

[Technical Field]

[0001] Incorporation by Reference This application claims the benefit of U.S. Patent Application No. 16 / 878,390, entitled "Method and Apparatus for Video Coding," filed May 19, 2020, which in turn claims the benefit of U.S. Provisional Application No. 62 / 860,149, entitled "High-level Syntax Control on Enabling Implicit Transform Selection," filed June 11, 2019. The disclosures of the prior applications are incorporated herein by reference in their entireties.

[0002] This disclosure describes embodiments generally related to video coding. [Background technology]

[0003] The background discussion provided herein is intended to generally present the context for the present disclosure. The inventors' work, to the extent that it is described in this background section, and aspects of the description that may not be admitted as prior art at the time of filing, are not admitted expressly or implicitly as prior art to the present disclosure.

[0004] Video coding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a series of images, each with spatial dimensions of, for example, 1920 x 1080 luminance samples and associated chrominance samples. The series of images can have a fixed or variable image rate (also informally called the frame rate), for example, 60 images per second or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video (1920 x 1080 luminance sample resolution at a 60 Hz frame rate) with 8 bits per sample requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 GBytes of storage space.

[0005] One goal of video coding and decoding can be to reduce redundancy in an input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage requirements by two or more orders of magnitude, in some cases. Both lossless and lossy compression, as well as combinations of them, may be used. Lossless compression refers to techniques that allow an exact copy of the original signal to be reconstructed from a compressed version. With lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signal is small enough that the reconstructed signal is useful for the intended application. For video, lossy compression is widely adopted. The amount of acceptable distortion varies depending on the application; for example, users of certain consumer streaming applications can tolerate higher distortion than users of television distribution applications. The achievable compression ratio can reflect that the higher the acceptable / tolerable distortion, the higher the compression ratio.

[0006] Video encoders and decoders can utilize techniques from several broad categories, such as motion compensation, transforms, quantization, and entropy coding.

[0007] Video codec technology can include a technique known as intra-coding. In intra-coding, sample values ​​are represented without reference to other data from the sample or a previously reconstructed reference image. In some video codecs, an image is spatially subdivided into blocks of samples. If all blocks of samples are coded in intra mode, the image may be an intra-image. Intra-images and their derivatives, such as independent decoder refresh images, can be used to reset the decoder state and thus can be used as the first image in a coding video bitstream and video session, or as still images. Samples in intra-blocks may be transformed, and the transform coefficients may be quantized before entropy coding. Intra-prediction can be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value and AC coefficients after the transform, the fewer bits required for a given quantization step size to represent the block after entropy coding.

[0008] Conventional intra-coding, such as that known from MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that rely on, for example, surrounding sample data and / or metadata obtained during encoding / decoding of spatially adjacent and preceding blocks of data in decoding order. Such techniques are hereinafter referred to as "intra-prediction" techniques. Note that, at least in some cases, intra-prediction uses only reference data from the current image being reconstructed, and not from a reference image.

[0009] Intra-prediction can take many different forms. When two or more of such techniques can be used in a given video coding technique, the technique in use can be coded as an intra-prediction mode. In some cases, a mode can have sub-modes and / or parameters, which can be coded separately or included in a mode codeword. The codeword used for a given mode / sub-mode / parameter combination can affect the coding efficiency gain via intra-prediction, and so can the entropy coding technique used to convert the codeword into a bitstream.

[0010] Certain modes of intra prediction were introduced in H.264, refined in H.265, and further improved with new coding techniques such as the Joint Search Model (JEM), Versatile Video Coding (VVC), and Benchmark Set (BMS). Predictor blocks can be formed using neighboring sample values ​​belonging to already available samples. The sample values ​​of neighboring samples are copied into the predictor block according to their direction. A reference to the direction in use can be coded in the bitstream or can itself be predicted.

[0011] Motion compensation can be a lossy compression technique, and can refer to a technique in which blocks of sample data from a previously reconstructed image or portion thereof (reference image) are spatially shifted in a direction indicated by a motion vector (MV) and then used to predict a newly reconstructed image or image portion. In some cases, the reference image may be the same as the image currently being reconstructed. The MV can have two dimensions, X and Y, or three dimensions, with the third dimension being a representation of the reference image in use (the latter may indirectly be the temporal dimension).

[0012] In some video compression techniques, the MV applicable to a particular region of sample data can be predicted from other MVs, for example, from an MV associated with another region of sample data that is spatially adjacent to the region being reconstructed and precedes that MV in decoding order. Doing so can substantially reduce the amount of data required to code the MV, thereby eliminating redundancy and increasing compression. MV prediction can work effectively, for example, when coding an input video signal derived from a camera (known as natural video), because there is a statistical likelihood that regions larger than the region to which a single MV is applicable will move in a similar direction and, therefore, in some cases, can be predicted using similar motion vectors derived from MVs in neighboring regions. This results in the MV found for a given region being similar or identical to the MV predicted from surrounding MVs, and after entropy coding, can be represented using fewer bits than would be used to code the MV directly. In some cases, MV prediction can be an example of lossless compression of a signal (i.e., MV) derived from the original signal (i.e., sample stream). In other cases, MV prediction itself can be lossy, for example, due to rounding errors when computing a predictor from several surrounding MVs.

[0013] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016). Here, we will explain a technique called "spatial merging" among the many MV prediction mechanisms provided by H.265.

[0014] Referring to Figure 1, the current block (101) contains samples found by the encoder during the motion search process to be predictable from a spatially shifted previous block of the same size. Instead of directly coding its MV, the MV can be derived from metadata associated with one or more reference pictures, e.g., the most recent (in decoding order) reference picture, using MVs associated with any one of five surrounding samples, denoted A0, A1, and B0, B1, B2 (102 to 106, respectively). In H.265, MV prediction can use predictors from the same reference picture as neighboring blocks. Summary of the Invention [Means for solving the problem]

[0015] An aspect of the present disclosure provides a first method of video decoding in a video decoder. The method may include receiving a first high-level syntax (HLS) element indicating whether explicit multiple transform selection (MTS) is enabled or disabled for intra-coded blocks, and receiving a second HLS element indicating whether explicit MTS is enabled or disabled for inter-coded blocks. The first and second HLS elements control the same set of coding blocks, including intra-coded blocks and inter-coded blocks. If the first HLS element indicates that explicit MTS is disabled for intra-coded blocks and the second HLS element indicates that explicit MTS is enabled for inter-coded blocks, implicit MTS may be enabled for intra-coded blocks.

[0016] An embodiment of the method may further include applying implicit MTS to intra-coded blocks. A transform type for processing the intra-coded blocks may be determined according to the size of the intra-coded blocks. In various examples, the first or second HLS element may be one of a video parameter set (VPS) syntax element, a sequence parameter set (SPS) syntax element, a picture parameter set (PPS) syntax element, a slice header syntax element, a tile header syntax element, or a tile group header syntax element. In one embodiment, the intra-coded blocks are not coded in intra subpartitioning (ISP) mode. In one embodiment, the method may further include receiving a third HLS element indicating that MTS is enabled for each of the inter-coded blocks and the intra-coded blocks.

[0017] An aspect of the present disclosure provides a second method of video decoding in a video decoder. The second method may include receiving a first HLS element indicating whether MTS is enabled or disabled for an intra-coded block, and receiving a second HLS element indicating whether non-separable quadratic transform (NSST) or matrix-based intra prediction (MIP) is disabled or enabled for the intra-coded block. If the first HLS element indicates that explicit MTS is disabled for the intra-coded block and the second HLS element indicates that NSST or MIP is disabled for the intra-coded block, implicit MTS may be enabled for the intra-coded block.

[0018] This disclosure may also provide a third method of video decoding in a video decoder. The third method may include receiving an intra-coded block associated with a first block-level syntax element indicating whether MTS is applied and a second block-level syntax element indicating whether NSST is applied. If the first block-level syntax element indicates that MTS is not applied and the second block-level syntax element indicates that NSST is not applied, implicit MTS may be enabled for the intra-coded block.

[0019] It should be noted that although this application refers to NSST, the disclosed methods and systems can be applied to variants of NSST, such as reduced size transforms (RSTs) and low frequency non-separable quadratic transforms (LFNSTs). Thus, NSST, RST, and / or LFNST may be used interchangeably throughout this application.

[0020] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a schematic diagram of a current block and its surrounding spatial merge candidates in one example. [Figure 2] FIG. 2 is a simplified block diagram schematic of a communication system (200) according to one embodiment. [Figure 3] FIG. 3 is a simplified block diagram schematic of a communication system (300) according to one embodiment. [Figure 4] FIG. 2 is a schematic diagram of a simplified block diagram of a decoder according to one embodiment. [Figure 5] FIG. 2 is a schematic diagram of a simplified block diagram of an encoder according to one embodiment. [Figure 6] FIG. 4 is a block diagram of an encoder according to another embodiment. [Figure 7] FIG. 10 is a block diagram of a decoder according to another embodiment. [Figure 8A] FIG. 2 illustrates an example transform core matrix for a 4-point DCT-2, according to one embodiment. [Figure 8B] FIG. 2 illustrates an example transform core matrix for an 8-point DCT-2, according to one embodiment. [Figure 8C] FIG. 2 illustrates an exemplary transform core matrix for a 16-point DCT-2, according to one embodiment. [Figure 8D] FIG. 2 illustrates an exemplary transform core matrix for a 32-point DCT-2, according to one embodiment. [Figure 9A] FIG. 1 illustrates sub-block types, sizes, and positions supported in a sub-block transform (SBT) according to one embodiment. [Figure 9B] FIG. 1 illustrates sub-block types, sizes, and positions supported in a sub-block transform (SBT) according to one embodiment. [Figure 9C] FIG. 1 illustrates sub-block types, sizes, and positions supported in a sub-block transform (SBT) according to one embodiment. [Figure 9D] FIG. 1 illustrates sub-block types, sizes, and positions supported in a sub-block transform (SBT) according to one embodiment. [Figure 10] FIG. 1 illustrates the number of sub-partitions depending on block size in intra-sub-partition (ISP) coding mode according to one embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example in which a block is divided into two sub-partitions in ISP coding mode. [Figure 12] FIG. 10 is a diagram illustrating an example in which a block is divided into four sub-partitions in ISP coding mode. [Figure 13A] FIG. 2 illustrates a 64×64 transform core matrix for a 64-point DCT-2 transform according to one embodiment. [Figure 13B] FIG. 2 illustrates a 64×64 transform core matrix for a 64-point DCT-2 transform according to one embodiment. [Figure 13C] FIG. 2 illustrates a 64×64 transform core matrix for a 64-point DCT-2 transform according to one embodiment. [Figure 13D] FIG. 2 illustrates a 64×64 transform core matrix for a 64-point DCT-2 transform according to one embodiment. [Figure 13E] FIG. 2 illustrates a 64×64 transform core matrix for a 64-point DCT-2 transform according to one embodiment. [Figure 14] FIG. 2 illustrates transform basis functions for a DST / DCT transform according to one embodiment. [Figure 15] FIG. 10 illustrates a table showing the mapping relationship between mts_idx values ​​and respective horizontal or vertical transforms according to one embodiment. [Figure 16A] FIG. 1 illustrates a transform core matrix for a DST-7 transform type according to one embodiment. [Figure 16B] FIG. 1 illustrates a transform core matrix for a DST-7 transform type according to one embodiment. [Figure 16C] FIG. 1 illustrates a transform core matrix for a DST-7 transform type according to one embodiment. [Figure 16D] FIG. 1 illustrates a transform core matrix for a DST-7 transform type according to one embodiment. [Figure 17A] FIG. 1 illustrates a transform core matrix for a DCT-8 transform type according to one embodiment. [Figure 17B] FIG. 1 illustrates a transform core matrix for a DCT-8 transform type according to one embodiment. [Figure 17C] FIG. 1 illustrates a transform core matrix for a DCT-8 transform type according to one embodiment. [Figure 17D] FIG. 1 illustrates a transform core matrix for a DCT-8 transform type according to one embodiment. [Figure 18] FIG. 10 illustrates an example of using a sequence parameter set (SPS) syntax element to control the use of multiple transform selection (MTS). [Figure 19] FIG. 10 illustrates a table of mappings between intra-prediction modes and transform sets according to one embodiment. [Figure 20]FIG. 20 illustrates an alternative transform coding process (2000) for RST 8x8 using a 16x64 transform core, according to one embodiment. [Figure 21] FIG. 21 illustrates an alternative transform coding process (2100) for RST 8x8 using a 16x48 transform core, according to one embodiment. [Figure 22] 22 shows an example CU-level syntax table (2200), in which the syntax element lfnst_idx indicating the selection of the low-frequency non-separable quadratic transform (LFNST) kernel is signaled at the end of the CU-level syntax. [Figure 23] FIG. 23 illustrates the process of contractive transformation (2301) and the process of inverse contractive transformation (2302), according to one embodiment. [Figure 24A] FIG. 24 shows the entire top left 8x8 coefficients (shaded sub-block) of the residual block (2410) used as input to compute the secondary transform in RST 8x8. [Figure 24B] FIG. 24 shows the top left three 4x4 sub-block coefficients (shaded sub-blocks) of the residual block (2410) used as input for computing the secondary transform in RST 8x8. [Figure 25] FIG. 10 illustrates a table for transform set selection based on intra-prediction mode according to one embodiment. [Figure 26] FIG. 26 illustrates an example process (2600) for matrix-based intra-prediction (MIP) mode. [Figure 27] 27 illustrates a CU-level syntax table according to one embodiment, in which a flag signaling matrix-based intra-prediction (MIP) mode is shown in a frame (2701). [Figure 28A] FIG. 28 shows a combined text (2800) specifying a transform coding process that performs explicit or implicit transform selection for a current block based on associated syntax elements received from the bitstream. [Figure 28B]FIG. 28 shows a combined text (2800) specifying a transform coding process that performs explicit or implicit transform selection for a current block based on associated syntax elements received from the bitstream. [Figure 29] FIG. 29 illustrates a modification (2900) to the text (2800) corresponding to an implicit transform enable scheme in which implicit transforms for intra residual blocks and explicit transforms for inter residual blocks can coexist. [Figure 30] FIG. 3 illustrates a modification (3000) to text (2800) corresponding to a scenario in which implicit conversion is enabled when non-separable secondary conversion (NSST) is disabled. [Figure 31] FIG. 3 shows a modification (3100) to text (2800) corresponding to a scenario in which implicit conversion is enabled when MIP is disabled. [Figure 32] FIG. 3 shows a modification (3200) to text (2800) corresponding to a scenario in which both MTS and NSST do not apply to the current block. [Figure 33] FIG. 3 shows a modification (3300) to text (2800) corresponding to a scenario in which neither MTS, NSST, nor MIP is applied to the current block. [Figure 34] FIG. 3 illustrates a flowchart of a transform coding process (3400) according to an embodiment of the present disclosure. [Figure 35] FIG. 35 illustrates a flowchart of a transform coding process (3500) according to an embodiment of the present disclosure. [Figure 36] FIG. 36 illustrates a flowchart of a transform coding process (3600) according to an embodiment of the present disclosure. [Figure 37] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] I. Video Coding Encoders and Decoders FIG. 2 illustrates a simplified block diagram of a communication system (200) according to one embodiment of the present disclosure. The communication system (200) includes multiple terminal devices capable of communicating with each other, for example, via a network (250). For example, the communication system (200) includes a first pair of terminal devices (210) and (220) interconnected via the network (250). In the example of FIG. 2, the first pair of terminal devices (210) and (220) perform unidirectional data transmission. For example, the terminal device (210) may code video data (e.g., a stream of video images captured by the terminal device (210)) for transmission to another terminal device (220) via the network (250). The encoded video data may be transmitted in the form of one or more coded video bitstreams. The terminal device (220) may receive the coded video data from the network (250), decode the coded video data to reconstruct the video images, and display the video images according to the reconstructed video data. One-way data transmission may be common, such as in media serving applications.

[0023] In another example, the communication system (200) includes a second pair of terminal devices (230) and (240) that perform bidirectional transmission of coded video data, such as may occur during a video conference. For the bidirectional transmission of data, in one example, each of the terminal devices (230) and (240) can code video data (e.g., a stream of video images captured by the terminal device) for transmission to the other of the terminal devices (230) and (240) over the network (250). Each of the terminal devices (230) and (240) can also receive coded video data transmitted by the other of the terminal devices (230) and (240), decode the coded video data to reconstruct the video images, and display the video images on an accessible display device according to the reconstructed video data.

[0024] In the example of FIG. 2 , terminal devices 210, 220, 230, and 240 may be depicted as a server, a personal computer, and a smartphone, although the principles of the present disclosure need not be so limited. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. Network 250 represents any number of networks that convey coded video data between terminal devices 210, 220, 230, and 240, including, for example, wired (cable) and / or wireless communication networks. Communications network 250 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of network 250 may not be important to the operation of the present disclosure, unless otherwise described herein below.

[0025] 3 shows the arrangement of a video encoder and a video decoder in a streaming environment as an example of an application for the disclosed subject matter. The disclosed subject matter may be equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0026] The streaming system may include a capture subsystem (313), which may include a video source (301), such as a digital camera, generating a stream of uncompressed video images (302). In one example, the stream of video images (302) includes samples captured by the digital camera. The stream of video images (302), depicted as a thick line to emphasize its high data volume compared to the encoded video data (304) (or coded video bitstream), may be processed by an electronic device (320) that includes a video encoder (303) coupled to the video source (301). The video encoder (303) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (304) (or coded video bitstream (304)), depicted as a thin line to emphasize its low data volume compared to the stream of video images (302), may be stored on a streaming server (305) for future use. One or more streaming client subsystems, such as the client subsystems (306) and (308) of Figure 3, can access the streaming server (305) to obtain copies (307) and (309) of the encoded video data (304). The client subsystem (306) can include, for example, a video decoder (310) within an electronic device (330). The video decoder (310) decodes an input copy (307) of the encoded video data and creates an output stream of video images (311) that can be rendered on a display (312) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (304), (307), and (309) (e.g., a video bitstream) can be encoded according to a particular video coding / compression standard. Examples of these standards include ITU-T Recommendation H.265.In one example, a developing video coding standard is informally known as Versatile Video Coding (VVC), and the disclosed subject matter may be used in the context of VVC.

[0027] It should be noted that the electronic devices (320) and (330) may include other components (not shown). For example, the electronic device (320) may include a video decoder (not shown), and the electronic device (330) may also include a video encoder (not shown).

[0028] 4 shows a block diagram of a video decoder (410) according to one embodiment of the present disclosure. The video decoder (410) may be included in an electronic device (430). The electronic device (430) may include a receiver (431) (e.g., a receiving circuit). The video decoder (410) may be used in place of the video decoder (310) in the example of FIG. 3.

[0029] The receiver (431) can receive one or more codec video sequences to be decoded by the decoder (410), in the same or another embodiment, one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences can be received from a channel (401), which can be a hardware / software link to a storage device that stores the encoded video data. The receiver (431) can receive the encoded video data along with other data, such as coded audio data and / or auxiliary data streams, that can be forwarded to respective using entities (not shown). The receiver (431) can separate the coded video sequences from other data. To combat network jitter, a buffer memory (415) can be coupled between the receiver (431) and the entropy decoder / parser (420) (hereinafter "parser (420)"). In certain applications, the buffer memory (415) is part of the video decoder (410). In other cases, it may be external to the video decoder (410) (not shown). In still other cases, there may be a buffer memory (not shown) external to the video decoder (410), for example, to combat network jitter, and another buffer memory (415) internal to the video decoder (410), for example, to handle playback timing. If the receiver (431) is receiving data from a store-and-forward device with sufficient bandwidth and control or from an isochronous network, the buffer memory (415) may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer memory (415) may be needed, may be relatively large, advantageously adaptively sized, and may be implemented at least in part in an operating system or similar element (not shown) external to the video decoder (410).

[0030] The video decoder (410) may include a parser (420) for reconstructing symbols (421) from the coded video sequence. These symbol categories include information used to manage the operation of the video decoder (410) and, potentially, information for controlling a rendering device, such as a rendering device (412) (e.g., a display screen) that is not an integral part of the electronic device (430) but may be coupled to the electronic device (430), as shown in FIG. 4. The rendering device control information may be in the form of a supplemental enhancement information (SEI) message or a video usability information (VUI) parameter set fragment (not shown). The parser (420) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may conform to a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, context-sensitive or non-context-sensitive arithmetic coding, etc. The parser (420) may extract a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), images, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (420) may also extract from coded video sequence information such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0031] The parser (420) can perform entropy decoding / syntax parsing operations on the video sequence received from the buffer memory (415) to create symbols (421).

[0032] The reconstruction of the symbols (421) can include many different units, depending on the type of coded video picture or portion thereof (e.g., inter- and intra-pictures, inter- and intra-blocks, etc.), and other factors. Which units are included and how they are included can be controlled by subgroup control information parsed from the coded video sequence by the parser (420). The flow of such subgroup control information between the parser (420) and the following many units is not shown for clarity.

[0033] In addition to the functional blocks already mentioned, the video decoder (410) may be conceptually subdivided into several functional units, as described below. In an actual implementation operating under commercial constraints, many of these units will interact closely with each other and may be, at least partially, integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

[0034] The first unit is a scalar / inverse transform unit (451), which receives quantized transform coefficients as well as control information from the parser (420) as symbols (421), including the transform used, block size, quantization coefficients, quantization scaling matrix, etc. The scalar / inverse transform unit (451) can output blocks containing sample values ​​that can be input to an aggregator (455).

[0035] In some cases, the output samples of the scaler / inverse transform (451) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed image but can use prediction information from a previously reconstructed portion of the current image. Such prediction information may be provided by an intra-image prediction unit (452). In some cases, the intra-image prediction unit (452) generates blocks of the same size and shape as the block being reconstructed using surrounding already reconstructed information fetched from a current image buffer (458). The current image buffer (458), for example, buffers a partially reconstructed current image and / or a fully reconstructed current image. The aggregator (455) may add, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (452) to the output sample information provided by the scaler / inverse transform unit (451).

[0036] In other cases, the output samples of the scalar / inverse transform unit (451) may relate to an inter-coded, possibly motion-compensated, block. In such cases, the motion-compensated prediction unit (453) may access a reference picture memory (457) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (421) related to the block, these samples may be added by an aggregator (455) to the output of the scalar / inverse transform unit (451) (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (457) from which the motion-compensated prediction unit (453) fetches prediction samples may be controlled by a motion vector and made available to the motion-compensated prediction unit (453) in the form of a symbol (421) that may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (457) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, and the like.

[0037] The output samples of the aggregator (455) may be subjected to various loop filtering techniques in a loop filter unit (456). Video compression techniques may include in-loop filter techniques controlled by parameters contained in the coded video sequence (also called the coded video bitstream) and provided to the loop filter unit (456) as symbols (421) from the parser (420), but may also be responsive to meta-information obtained during decoding of a coded image or previous portion (in decoding order) of the coded video sequence, and to previously reconstructed loop-filtered sample values.

[0038] The output of the loop filter unit (456) may be a sample stream that can be output to a rendering device (412) and stored in a reference image memory (457) for use in future inter-image prediction.

[0039] Once a particular coded picture has been fully reconstructed, it can be used as a reference picture for future predictions. For example, once the coded picture corresponding to the current picture has been fully reconstructed and the coded picture has been identified (e.g., by the parser (420)) as a reference picture, the current picture buffer (458) can become part of the reference picture memory (457), and a new current picture memory can be reallocated before starting reconstruction of the next coded picture.

[0040] The video decoder (410) can perform decoding operations according to a predetermined video compression technique in a standard, such as ITU-T Rec. H.265. The coded video sequence can comply with the syntax specified by the video compression technique or standard being used, in the sense that the coded video sequence conforms to both the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, the profile can select specific tools from all tools available in the video compression technique or standard as the only tools usable under that profile. Compliance also requires that the complexity of the coded video sequence be within a range defined by the level of the video compression technique or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level can, in some cases, be further limited by the specification of a hypothetical reference decoder (HRD) and metadata for HRD buffer management signaled in the coded video sequence.

[0041] In one embodiment, the receiver (431) can receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data can be used by the video decoder (410) to properly decode the data and / or more accurately reconstruct the original video data. The additional data can be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0042] 5 shows a block diagram of a video encoder (503) according to one embodiment of the present disclosure. The video encoder (503) is included in an electronic device (520). The electronic device (520) includes a transmitter (540) (e.g., a transmission circuit). The video encoder (503) can be used in place of the video encoder (303) in the example of FIG. 3.

[0043] The video encoder (503) can receive video samples from a video source (501) (which in the example of FIG. 5 is not part of the electronic device (520)) that can capture video images to be coded by the video encoder (503). In another example, the video source (501) is part of the electronic device (520).

[0044] The video source (501) can provide a source video sequence to be coded by the video encoder (503) in the form of a digital video sample stream, which can be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (501) can be a storage device that stores previously prepared video. In a video conferencing system, the video source (501) can be a camera that captures local image information as a video sequence. The video data can be provided as multiple individual images that, when viewed sequentially, impart motion. The image itself can be organized as a spatial array of pixels, each of which can contain one or more samples, depending on the sampling structure, color space, etc., in use. Those skilled in the art can readily understand the relationship between pixels and samples. The following discussion focuses on samples.

[0045] According to one embodiment, the video encoder (503) can code and compress images of a source video sequence into a coded video sequence (543) in real time or under any other time constraints, as required by the application. Enforcing an appropriate coding rate is one of the functions of the controller (550). In some embodiments, the controller (550) controls and is operatively coupled to other functional units as described below. For clarity, coupling is not depicted. Parameters set by the controller (550) can include parameters related to rate control (e.g., picture skip, quantizer, lambda value for rate-distortion optimization techniques), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (550) can be configured to have other appropriate functions for the video encoder (503) optimized for a particular system design.

[0046] In some embodiments, the video encoder (503) is configured to operate in a coding loop. As an overly simplified explanation, in one example, the coding loop can include a source coder (530) (e.g., responsible for generating symbols, such as a symbol stream, based on an input image to be coded and a reference image) and a (local) decoder (533) embedded in the video encoder (503). The decoder (533) reconstructs the symbols to create sample data in a manner similar to that of the (remote) decoder (because in the video compression techniques contemplated by the disclosed subject matter, any compression between the symbols and the coded video bitstream is lossless). The reconstructed sample stream (sample data) is input to a reference image memory (534). Because decoding of the symbol stream produces bit-exact results regardless of the location (local or remote) of the decoder, the contents of the reference image memory (534) are also bit-exact between the local and remote encoders. In other words, the prediction part of the encoder "sees" as reference picture samples exactly the same sample values ​​that the decoder would "see" if it were to use prediction during decoding. This basic principle of reference picture synchrony (and the drift that occurs when synchrony cannot be maintained, e.g., due to channel errors) is also used in several related technologies.

[0047] The operation of the "local" decoder (533) may be identical to the operation of a "remote" decoder, such as the video decoder (410) already described in detail in connection with Figure 4. However, with brief reference also to Figure 4, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (545) and parser (420) may be lossless, the entropy decoding portion of the video decoder (410), including the buffer memory (415), and the parser (420), may not be fully implemented in the local decoder (533).

[0048] An observation that can be made at this point is that any decoder technology other than syntax analysis / entropy decoding that is present in a decoder must necessarily be present in the corresponding encoder in substantially identical functional form. For this reason, the disclosed subject matter focuses on decoder operation. A description of the encoder technology can be omitted, as it is the reverse of the decoder technology that has been comprehensively described. Only in certain areas is more detailed description necessary, and is provided below.

[0049] In operation, in some examples, the source coder (530) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this manner, the coding engine (532) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as predictive references for the input picture.

[0050] The local video decoder (533) can decode the coded video data of an image that may be designated as a reference image based on the symbols created by the source coder (530). The operation of the coding engine (532) can advantageously be a lossy process. When the coded video data is decoded by a video decoder (not shown in FIG. 5), the reconstructed video sequence may typically be a replica of the source video sequence with some errors. The local video decoder (533) can replicate the decoding process that may be performed by the video decoder on the reference image and store the reconstructed reference image in a reference image cache (534). In this way, the video encoder (503) can locally store copies of reconstructed reference images that have common content as reconstructed reference images (without transmission errors) obtained by the far-end video decoder.

[0051] The predictor (535) can perform the prediction search for the coding engine (532). That is, for a new image to be coded, the predictor (535) can search the reference image memory (534) for sample data (as candidate reference pixel blocks) or specific metadata of the reference image, such as motion vectors, block shapes, etc., which can serve as an appropriate prediction reference for the new image. The predictor (535) can operate on a sample block-by-pixel block basis to find an appropriate prediction reference. In some cases, as determined by the search results obtained by the predictor (535), the input image may have a prediction reference drawn from multiple reference images stored in the reference image memory (534).

[0052] The controller (550) can manage the coding operations of the source coder (530), including, for example, setting the parameters and subgroup parameters used to code the video data.

[0053] The output of all the aforementioned functional units may undergo entropy coding in an entropy coder (545), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, or arithmetic coding.

[0054] The transmitter (540) can buffer the coded video sequence created by the entropy coder (545) and prepare it for transmission over a communication channel (560), which can be a hardware / software link to a storage device that will store the encoded video data. The transmitter (540) can merge the coded video data from the video coder (503) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0055] The controller (550) can manage the operation of the video encoder (503). During coding, the controller (550) can assign a particular coded image type to each coded image, which can affect the coding techniques that can be applied to the respective image. For example, in many cases, an image can be assigned as one of the following image types:

[0056] An intra-picture (I-picture) may be one that can be coded and decoded without using any other picture in a sequence as a source of prediction. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of these variations of I-pictures and their respective uses and characteristics.

[0057] A predicted image (P-image) may be one that can be coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0058] Bidirectionally predicted images (B-pictures) may be those that can be coded and decoded using intra- or inter-prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted images may use more than two reference images and associated metadata to reconstruct a single block.

[0059] A source image is typically spatially divided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks, as determined by the coding assignment applied to the respective image of the block. For example, blocks of an I-image may be non-predictively coded, or they may be predictively coded with reference to previously coded blocks of the same image (spatial prediction or intra-prediction). Pixel blocks of a P-image may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference image. Blocks of a B-image may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference images.

[0060] The video encoder (503) may perform coding operations in accordance with a predetermined video coding technique or standard, such as ITU-T Rec. H.265. In doing so, the video encoder (503) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the coded video data may conform to a syntax specified in the video coding technique or standard being used.

[0061] In one embodiment, the transmitter (540) can transmit additional data along with the encoded video. The source coder (530) can include such data as part of the coded video sequence. The additional data can include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0062] Video may be captured as multiple source images (video images) in time sequence. Intra-image prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given image, while inter-image prediction exploits correlation (temporal or otherwise) between images. In one example, a particular image being coded / decoded, called the current image, is divided into blocks. When a block in the current image is similar to a reference block in a previously coded and still buffered reference image in the video, the block in the current image can be coded by a vector called a motion vector. The motion vector points to a reference block in the reference image and may have a third dimension that identifies the reference image if multiple reference images are used.

[0063] In some embodiments, bi-prediction techniques can be used for inter-picture prediction. According to bi-prediction techniques, two reference images, such as a first reference image and a second reference image, are used, both of which are before the decoding order of the current image in the video (but their display orders may be past and future, respectively). A block in the current image can be coded by a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. A block can be predicted by a combination of the first reference block and the second reference block.

[0064] Furthermore, merge mode techniques can be used for inter-picture prediction to improve coding efficiency.

[0065] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed block-by-block. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into coding tree units (CTUs) for compression, and the CTUs in a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Generally, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree-decomposed into one or more coding units (CUs). For example, a 64x64 pixel CTU can be divided into one CU of 64x64 pixels, four CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In one example, each CU is analyzed to determine the CU's prediction type, such as inter prediction type or intra prediction type. The CU is divided into one or more prediction units (PUs) according to temporal and / or spatial predictability. Generally, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values) of 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0066] 6 shows a diagram of a video encoder (603) according to another embodiment of the present disclosure. The video encoder (603) is configured to receive a processed block of sample values ​​(e.g., a predictive block) in a current video image in a sequence of video images and encode the processed block into a coded image that is part of a coded video sequence. In one example, the video encoder (603) is used in place of the video encoder (303) of the example of FIG. 3.

[0067] In an HEVC example, the video encoder (603) receives a matrix of sample values ​​for a processing block, such as a predictive block of 8x8 samples. The video encoder (603) determines whether the processing block is best coded using intra-mode, inter-mode, or bi-predictive mode, e.g., using rate-distortion optimization. If the processing block is coded in intra-mode, the video encoder (603) can use intra-prediction techniques to encode the processing block into a coded image. If the processing block is to be coded in inter-mode or bi-predictive mode, the video encoder (603) can use inter-prediction or bi-prediction techniques, respectively, to encode the processing block into a coded image. In certain video coding techniques, merge mode may be an inter-picture prediction submode in which motion vectors are derived from one or more motion vector predictors without the benefit of coded motion vector components outside the predictors. In certain other video coding techniques, there may be motion vector components applicable to the current block. In one example, the video encoder (603) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0068] In the example of Figure 6, the video encoder (603) includes an inter-encoder (630), an intra-encoder (622), a residual calculator (623), a switch (626), a residual encoder (624), a general controller (621), and an entropy encoder (625), which are coupled together as shown in Figure 6.

[0069] The inter-encoder (630) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in reference images (e.g., blocks in a previous image and a subsequent image), generate inter-prediction information (e.g., a description of redundant information by an inter-coding technique, a motion vector, merge mode information), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using any suitable technique. In some examples, the reference image is a decoded reference image that is decoded based on the encoded video information.

[0070] The intra encoder (622) is configured to receive samples of a current block (e.g., a processing block), optionally compare the block with previously coded blocks in the same image, generate quantized coefficients after transformation, and optionally also generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). In one example, the intra encoder (622) calculates an intra prediction result (e.g., a predicted block) based on the intra prediction information and a reference block in the same image.

[0071] The general-purpose controller (621) is configured to determine general-purpose control data and control other components of the video encoder (603) based on the general-purpose control data. In one example, the general-purpose controller (621) determines the mode of the block and provides a control signal to the switch (626) based on the mode. For example, if the mode is intra mode, the general-purpose controller (621) controls the switch (626) to select an intra-mode result to be used by the residual calculator (623) and controls the entropy encoder (625) to select intra-prediction information to include in the bitstream. If the mode is inter mode, the general-purpose controller (621) controls the switch (626) to select an inter-prediction result to be used by the residual calculator (623) and controls the entropy encoder (625) to select inter-prediction information to include in the bitstream.

[0072] The residual calculator (623) is configured to calculate the difference (residual data) between the received block and a prediction result selected from the intra-encoder (622) or the inter-encoder (630). The residual encoder (624) is configured to operate on the residual data to encode the residual data to generate transform coefficients. In one example, the residual encoder (624) is configured to transform the residual data from the spatial domain to the frequency domain to generate transform coefficients. The transform coefficients are then subjected to a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (603) also includes a residual decoder (628). The residual decoder (628) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be suitably used in the intra-encoder (622) and the inter-encoder (630). For example, the inter-encoder (630) may generate decoded blocks based on the decoded residual data and inter-prediction information, and the intra-encoder (622) may generate decoded blocks based on the decoded residual data and intra-prediction information. In some examples, the decoded blocks may be appropriately processed to generate a decoded image, which may be buffered in a memory circuit (not shown) and used as a reference image.

[0073] The entropy encoder (625) is configured to format the bitstream to include the encoded blocks. The entropy encoder (625) is configured to include various information in accordance with an appropriate standard, such as the HEVC standard. In one example, the entropy encoder (625) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other appropriate information in the bitstream. Note that, according to the disclosed subject matter, when coding a block in a merged sub-mode of either an inter mode or a bi-prediction mode, residual information is not present.

[0074] 7 shows a diagram of a video decoder (710) according to another embodiment of the present disclosure. The video decoder (710) is configured to receive coded images that are part of a coded video sequence and decode the coded images to generate reconstructed images. In one example, the video decoder (710) is used in place of the video decoder (310) of the example of FIG. 3.

[0075] In the example of Figure 7, the video decoder (710) includes an entropy decoder (771), an inter-decoder (780), a residual decoder (773), a reconstruction module (774), and an intra-decoder (772), which are coupled together as shown in Figure 7.

[0076] The entropy decoder (771) may be configured to reconstruct, from the coded image, specific symbols that represent the syntax elements of which the coded image is composed. Such symbols may include, for example, the mode in which the block is coded (e.g., intra mode, inter mode, bi-prediction mode, etc., the latter two being merged or separate submodes), prediction information (e.g., intra-prediction information or inter-prediction information, etc.) that may identify specific samples or metadata used for prediction by the intra decoder (772) or inter decoder (780), respectively, as well as residual information, for example in the form of quantized transform coefficients. In one example, if the prediction mode is an inter-prediction mode or a bi-prediction mode, the inter-prediction information is provided to the inter decoder (780), and if the prediction type is an intra-prediction type, the intra-prediction information is provided to the intra decoder (772). The residual information may undergo inverse quantization and be provided to the residual decoder (773).

[0077] The inter decoder (780) is configured to receive the inter prediction information and generate an inter prediction result based on the inter prediction information.

[0078] The intra decoder (772) is configured to receive intra prediction information and generate a prediction result based on the intra prediction information.

[0079] The residual decoder (773) is configured to perform inverse quantization to extract dequantized transform coefficients and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (773) may also require certain control information (to include quantizer parameters (QP)), which may be provided by the entropy decoder (771). (The data path not shown may be low-volume control information only.)

[0080] The reconstruction module (774) is configured to combine, in the spatial domain, the residual as output by the residual decoder (773) and the prediction result (possibly as output by an inter- or intra-prediction module) to form a reconstructed block that may be part of a reconstructed image, and the reconstructed block may be part of a reconstructed video. It should be noted that other suitable operations, such as a deblocking operation, may be performed to improve visual quality.

[0081] It should be noted that the video encoders (303), (503), and (603) and the video decoders (310), (410), and (710) may be implemented using any suitable technology. In one embodiment, the video encoders (303), (503), and (603) and the video decoders (310), (410), and (710) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (303), (503), and (503) and the video decoders (310), (410), and (710) may be implemented using one or more processors executing software instructions.

[0082] II. Transform Coding and Related Technologies 1. Example of DCT-2 linear transformation In some embodiments, 4-point, 8-point, 16-point, and 32-point DCT-2 transforms are used as the primary transform. Figures 8A-8D show the transform core matrices for 4-point, 8-point, 16-point, and 32-point DCT-2s, respectively. The elements of these transform core matrices can be represented using 8-bit integers, and therefore these transform core matrices are referred to as 8-bit transform cores. As shown, the transform core matrices of the smaller DCT-2s are subsets of the transform core matrices of the larger DCT-2s.

[0083] The DCT-2 core matrix exhibits symmetry / anti-symmetry properties. Therefore, it can support a so-called "partial butterfly" implementation to reduce the operation count (multiplications, additions / subtractions, shifts). Using a partial butterfly implementation, it is possible to obtain identical results for matrix multiplication compared to not using the partial butterfly implementation.

[0084] 2. Sub-block transform coding example 2.1 Sub-Block Transform (SBT) In some embodiments, a sub-block transform (SBT), also known as a spatially varying transform (SVT), is used. In some embodiments, the SBT is applied to inter-prediction residuals. For example, a coding block can be divided into sub-blocks, and only some of the sub-blocks are processed in the residual block. Zero residual is assumed for the remaining parts of the sub-blocks. Therefore, the residual block is smaller than the coding block, and the transform size in the SBT is smaller than the coding block size. For areas not covered by the residual block, no transform processing is performed.

[0085] 9A-9D show the sub-block types (SVT-H, SVT-V) (e.g., divided vertically or horizontally), sizes, and positions (e.g., left half, left quarter, right half, right quarter, top half, top quarter, bottom half, bottom quarter) supported by SBT. The shaded areas labeled with the letter "A" are residual blocks to be transform coded, and other areas are assumed to be zero residual without transform.

[0086] 2.2. Intra-Subpartition (ISP) Coding Mode In some embodiments, an intra-subpartition (ISP) coding mode is used. In the ISP coding mode, a luma intra-predicted block can be divided vertically or horizontally into two or four subpartitions. The number of subpartitions can depend on the size of the block. Figure 10 shows the number of subpartitions depending on the block size. Figure 11 shows a scenario in which a block is divided into two subpartitions. Figure 12 shows a scenario in which a block is divided into four subpartitions. In one example, all subpartitions satisfy the condition that they have at least 16 samples. In one example, ISP is not applied to the chroma components.

[0087] In one example, for each subpartition divided from a coding block, a residual signal is generated by entropy decoding the respective coefficients transmitted from the encoder, then inverse quantizing and inverse transforming them. Next, a first subpartition of the subpartitions is intra-predicted to generate a prediction signal. The prediction signal is added to each residual signal of the first subpartition to obtain corresponding reconstructed samples. The reconstructed sample values ​​of the first subpartition can then be used to generate a prediction for the second subpartition. This process can be repeated for each subpartition until all subpartitions from the coding block have been reconstructed. In one example, all subpartitions share the same intra-mode.

[0088] In one embodiment, ISP coding modes are tested only with intra modes that are part of the Most Probable Mode (MPM) list. Therefore, if a block uses an ISP, it can be inferred that the MPM flag is 1. Additionally, when an ISP is used for a particular block, the respective MPM list is modified to exclude DC mode and to prioritize horizontal intra mode for horizontal splitting of the ISP and vertical intra mode for vertical splitting.

[0089] In the ISP coding mode, the transformation and reconstruction are performed separately for each subpartition, so each subpartition can be considered as a sub-TU.

[0090] 3. Transform coding using extended DCT-2 transform and multiple transform selection (MTS) In some embodiments, if both the height and width of a coding block are 64 samples or less, the transform size is always the same as the coding block size. If either the height or width of a coding block is greater than 64 samples, when performing transform or intra prediction, the coding block is further divided into multiple sub-blocks, each with a width and height of 64 or less, and a transform operation is performed on each sub-block.

[0091] 3.1 Transform coding using the extended DCT-2 transform In some embodiments, 2-point and 64-point DCT-2 transforms can be used in addition to the 4-point, 8-point, 16-point, and 32-point DCT-2 transforms described above. Figures 13A-13E show the 64x64 transform core matrix for a 64-point DCT-2 transform.

[0092] 3.2 Explicitly Signaled Conversion In some embodiments, in addition to DCT-2 and 4×4 DST-7 transform coding, multiple transform selection (MTS) (also known as extended multiple transform (EMT) or adaptive multiple transform (AMT)) can be used for residual coding of both inter-coded and intra-coded blocks. MTS uses multiple selected transforms from the discrete cosine transform (DCT) / discrete sine transform (DST) family other than the DCT-2 and 4×4 DST-7 transforms. The selection is performed at the encoder and may be explicitly signaled from the encoder to the decoder. For example, the selected transform may include a DST-7 or DCT-8 transform. Figure 14 shows the transform basis functions of the DST / DCT transform. In some embodiments, the DST / DCT transform core matrix used in MTS is represented in an 8-bit representation.

[0093] In some embodiments, MTS can be applied to CUs with both width and height of 32 samples or less. Whether MTS is applied can be controlled by a flag represented by mts_flag. For example, if mts_flag is equal to 0, only DCT-2 is applied to coding of the residual block. If mts_flag is equal to 1, indicating that MTS is applied, a selected transform can be used. For example, the index indicated by mts_idx can be further signaled using two bins to specify the horizontal and vertical transforms used.

[0094] Figure 15 shows a table (1500) that shows the mapping relationship between mts_idx values ​​and the respective horizontal or vertical transform. Rows (1301) with an mts_idx value of -1 correspond to scenarios where mts_flag is 0 (indicating that MTS is not applied) and a DCT-2 transform is used. Rows (1302)-(1305) with mts_idx values ​​of 0, 1, 2, or 3 correspond to scenarios where mts_flag is equal to 1 (indicating that MTS is applied). In the two right-hand columns of table (1500), 0 represents the DCT-2 transform type, 1 represents the DCT-7 transform type, and 2 represents the DCT-8 transform type.

[0095] Figures 16A to 16D show transform core matrices for a DCT-7 transform type, and Figures 17A to 17D show transform core matrices for a DCT-8 transform type.

[0096] In some embodiments, MTS can be enabled or disabled using high-level syntax (HLS) elements. Each HLS element may be a video parameter set (VPS) syntax element, a sequence parameter set (SPS) syntax element, a picture parameter set (PPS) syntax element, a slice header syntax element, a tile header syntax element, or a tile group header syntax element, etc. Figure 18 shows an example of using SPS syntax elements to control MTS usage. As shown, the SPS syntax element sps_mts_enabled_flag may be signaled to indicate whether MTS is enabled for a video sequence. Once MTS is enabled, two syntax elements, sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag, may be signaled to indicate whether MTS is enabled for coding inter-predicted or intra-predicted blocks, respectively.

[0097] In one embodiment, implicit MTS is applied when the above signaling-based MTS (called explicit MTS) is not used. Implicit MTS can select transforms according to block width and height instead of based on signaling. For example, implicit MTS can be used to select DST-7 transforms for the short side of a transform block and DCT-2 transforms for the long side of the transform block.

[0098] 3.3 Scenarios in which implicit conversion selection applies In various embodiments, in certain scenarios, DST-7 and / or DCT-8 can be used without being explicitly signaled. For example, DST-7 and / or DCT-8 can be used implicitly based on information available to both the encoder and the corresponding decoder. These scenarios include:

[0099] 3.3.1 Intra-Subpartitioning (ISP) For residual blocks coded in ISP mode, as long as the block width of the residual block is between 4 and 16, inclusive, the horizontal transform is selected as DST-7, and as long as the block height of the residual block is between 4 and 16, inclusive, the vertical transform is selected as DST-7.

[0100] 3.3.2 Sub-Block Transform (SBT) In SBT mode, for sub-TUs located in the left half (or quarter) or right half (or quarter) of the current CU, the horizontal transform can be DCT-8 or DST-7, respectively. Otherwise (the sub-TU has the same width as the current CU), DCT-2 can be used. For sub-TUs located in the top half (or quarter) or bottom half (or quarter) of the current CU, the vertical transform can be DCT-8 or DST-7, respectively. Otherwise (the sub-TU has the same height as the current CU), DCT-2 can be used.

[0101] 3.3.3 MTS Disabled by HLS Elements For example, if sps_mts_enabled_flag is signaled as true, but both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false, then for intra prediction residuals, horizontal transform can be selected as DST-7 if the respective block width is between 4 and 16, and vertical transform can be selected as DST-7 if the respective block height is between 4 and 16.

[0102] 4. Non-separable quadratic transformation (NSST) 4.1 Initial design of NSST In some embodiments, a mode-dependent non-separable quadratic transform (NSST) may be applied between the forward core transform and quantization (in the encoder) and between the inverse quantization and inverse core transform (in the corresponding decoder). To maintain low complexity, in some embodiments, the NSST may be applied only to low-frequency coefficients after the linear transform. If both the width (W) and height (H) of a transform coefficient block are equal to or greater than 8, an 8x8 non-separable quadratic transform may be applied to the top-left 8x8 region of the transform coefficient block. Otherwise, if either W or H of the transform coefficient block is equal to 4, a 4x4 non-separable quadratic transform may be applied, and the 4x4 non-separable quadratic transform is performed on the top-left min(8,W) x min(8,H) region of the transform coefficient block. The above transform selection rules apply to both luma and chroma components.

[0103] The implementation of matrix multiplication for non-separable transforms is explained as follows using a 4x4 input block as an example: To apply a non-separable transform, a 4x4 input block X

number

number

number

[0104] The non-separable transform is computed as follows:

number

number

number

[0105] In one example of NSST, there can be a total of 35x3 non-separable secondary transforms for both 4x4 and 8x8 block sizes, where 35 is the number of transform sets corresponding to the intra prediction modes, and 3 is the number of NSST candidates for each intra prediction mode. The mapping from intra prediction modes to transform sets is defined in table (1900) shown in Figure 19. For example, the transform set applied to luma / chroma transform coefficients can be specified by the corresponding luma / chroma intra prediction mode according to table (1900). For intra prediction modes greater than 34 (e.g., diagonal prediction direction), the transform coefficient blocks are transposed before / after the secondary transform at the encoder / decoder.

[0106] For each transform set, the selected non-separable secondary transform candidate is further specified by an explicitly signaled CU-level NSST index. The index is signaled in the bitstream once per intra CU after transform coefficients and truncated unary binarization are used. The truncation value is 2 for planar or DC modes and 3 for angular intra prediction modes. This NSST index is signaled only if the CU has multiple non-zero coefficients. If not signaled, the default value is 0. A zero value for this syntax element indicates that no secondary transform is applied to the current CU, while values ​​1 through 3 indicate which secondary transform from the set should be applied.

[0107] NSST may not be applied to blocks coded in transform skip mode. When an NSST index is signaled for a CU and is not equal to zero, NSST is not used for component blocks coded in transform skip mode within the CU. If a CU with all component blocks is coded in transform skip mode or the number of non-zero coefficients in a non-transform skip mode CB is less than two, an NSST index is not signaled for the CU.

[0108] 4.2 Reduced Size Transformation (RST) In some embodiments, a variant of NSST called reduced size transform (RST) or low frequency non-separable quadratic transform (LFNST) is used. RST uses a transform zero-out method. Whether the intra prediction mode is planar or DC is checked for entropy coding the transform index of NSST.

[0109] In one example, four transform sets are applied, each transform set including three RST transform cores. The RST transform cores can have a size of 16x48 (or 16x64) (applied to transform coefficient blocks whose height and width are both 8 or greater) or 16x16 (applied to transform coefficient blocks whose height or width is either 4). For convenience of notation, a 16x48 (or 16x64) transform is denoted as RST 8x8, and a 16x16 transform is denoted as RST 4x4.

[0110] For RST 8x8, two alternative transform coding processes (2000) and (2100) using 16x64 and 16x48 transform cores are shown in Figures 20 and 21, respectively. VVC Draft 5 adopts the use of 16x48 transform cores.

[0111] In the example process (2000) of FIG. 20, the encoder may first perform a forward primary transform (2010) on the residual block, followed by a forward secondary transform (2012) on the coefficients generated from the forward primary transform (2010). In the forward secondary transform (2012), the 64 coefficients of the 4×4 sub-blocks A, B, C, and D in the upper left corner of the coefficient block (2013) are represented as a 64-length vector, which can be multiplied by a 16×64 transform matrix according to equation (3) to obtain a 16-length vector. The elements in the 16-length vector are backfilled into the 4×4 sub-block A in the upper left corner of the coefficient block (2013). The coefficients in sub-blocks B, C, and D can take on a value of zero. The coefficients obtained after the forward secondary transform (2012) are then quantized in step (2014) and entropy coded to generate coded bits in the bitstream (2016).

[0112] The coded bits are received at the decoder side and, after entropy decoding, can be inversely quantized (2024) to generate a coefficient block (2023). An inverse secondary transform (2022) can be performed across the 16 coefficients in the top-left 4x4 sub-block E to obtain 64 coefficients that are backfilled into the 4x4 sub-blocks E / F / G / H. The coefficients in the inverse secondary transform (2022) block (2023) can then be processed with an inverse primary transform (2020) to obtain a reconstructed residual block.

[0113] The example process (2100) of Figure 21 is similar to the process (2000), except that fewer coefficients (48) are processed during the forward quadratic transform (2012). Specifically, the 48 coefficients in sub-blocks A / B / C are processed with a smaller transform matrix of size 16x48. Using a smaller transform matrix can reduce the memory size for storing the transform matrix and the complexity of the respective calculations.

[0114] FIG. 22 shows an exemplary CU-level syntax table (2200), and at the end of the CU-level syntax, a syntax element lfnst_idx indicating the selection of the LFNST kernel is signaled.

[0115] 4.3 Example of RST Calculation The main concept of the reduction transform (RT) is to map an N-dimensional vector to an R-dimensional vector in a different space, where R / N (R < N) is the reduction factor.

[0116] The RST matrix is an R×N matrix as follows.

Equation

[0117] FIG. 23 shows the process of reduction transform (2301) and the process of reduction inverse transform (2302). T represents an RST transform matrix having dimensions of RxN, and T T represents the transpose matrix of T having dimensions of NxR.

[0118] In RST 8×8, a reduction ratio of 4 (1 / 4 size) is achieved. For example, instead of a 64×64 which is the size of a conventional 8×8 non-separable transform matrix, a 16×64 direct matrix is used. The 64×16 inverse RST matrix is used on the decoder side to generate core (primary) transform coefficients in the upper left region of the 8×8. The forward RST 8×8 uses 16×64 (or 8×64 in the case of an 8×8 block) matrices to generate non-zero coefficients only in the upper left 4×4 region within a given 8×8 region. In other words, when RST is applied, the 8×8 region excluding the upper left 4×4 region will have only zero coefficients. In the case of RST 4×4, 16×16 (or 8×16 in the case of a 4×4 block) direct matrix multiplication can be applied.

[0119] Additionally, for RST 8x8, to further reduce the transform matrix size, instead of using the entire top-left 8x8 coefficient of the residual block (2410) (shaded sub-block in Figure 24A) as the input for computing the secondary transform, the top-left three 4x4 sub-block coefficients of the residual block (2410) (shaded sub-block in Figure 24B) are used as the input for computing the secondary transform.

[0120] In one example, the inverse RST is conditionally applied if the following two conditions are met: (i) the respective block size is greater than or equal to a given threshold (W>=4&&H>=4), and (ii) the transform skip mode flag is equal to 0. For example, if both the width (W) and height (H) of a transform coefficient block are greater than 4, then the RST 8x8 is applied to the top-left 8x8 region of the transform coefficient block. Otherwise, the RST 4x4 is applied to the top-left min(8,W)×min(8,H) region of the transform coefficient block.

[0121] In one example, when the RST index is equal to 0, the RST is not applied. Otherwise, the RST is applied and the kernel is selected using the RST index. In one example, the RST is applied to intra CUs in both intra and inter slices, and to both luma and chroma. If dual tree is enabled, the RST indexes for luma and chroma are signaled separately. For inter slices (dual tree is disabled), a single RST index is signaled and used for both luma and chroma. When ISP mode is selected, the RST is disabled and the RST index is not signaled.

[0122] 4.4 Example of RST transformation matrix selection In one example, an RST matrix can be selected from four transform sets, each consisting of two transforms. Which transform set to apply can be determined based on the applied intra-prediction mode as follows: When one of three cross-component linear model (CCLM) modes is indicated, transform set 0 can be selected. Otherwise, transform set selection can be performed according to table (2500) shown in FIG. 25. An index for accessing table (2500), denoted IntraPredMode, can be in the range of [-14, 83], which is, for example, the post-transform mode index used for wide-angle intra-prediction.

[0123] 5. Matrix-based Intra Prediction (MIP) Mode In some embodiments, a matrix-based intra-prediction (MIP) mode is used. Figure 26 shows an example process (2600) for the MIP mode. To predict samples for a rectangular block (2610) of width W and height H, MIP takes as input one line of H reconstructed adjacent boundary samples to the left of the block (2610) and one line of W reconstructed adjacent boundary samples above the block (2610). If reconstructed samples are not available, they can be generated in a manner similar to conventional intra-prediction.

[0124] The generation of the predicted signal can be based on the following three steps (2601) to (2603): In step (2601), among the boundary samples, four samples are extracted by averaging if W=H=4, and eight samples otherwise.

[0125] In step (2602), matrix vector multiplication A k bdry red , followed by offset b k The sum of the averaged samples bdry red The result is a reduced prediction signal on the set of subsampled samples (2621) in the original block.k and offset b k can be selected based on the MIP mode index k.

[0126] In step (2603), prediction signals for the remaining positions (2622) are generated from the prediction signals on the subsampled set (2621) by linear interpolation, which is a single-step linear interpolation in each direction.

[0127] Matrix A required to generate the predicted signal k and the offset vector b k can be obtained from three sets of matrices S0, S1, S2. Set S0 contains 18 matrices, each with 16 rows and 4 columns.

number

number

number

number

number

number

[0128] As shown, given an 8x8 block (2610), MIP takes four averages along each axis of the boundary. The resulting eight input samples enter a matrix-vector multiplication. The matrix is ​​taken from set S1. This results in 16 samples (2621) in the odd positions of the prediction block. Therefore, a total of (816) / (88) = 2 multiplications per sample are performed. After adding the offset, these samples are vertically interpolated using the scaled-down top boundary samples. Horizontal interpolation continues by using the original left boundary samples. In this case, the interpolation process does not require multiplications.

[0129] 5.1 MIP Mode Signaling In some embodiments, for each coding unit (CU) in intra mode, a flag may be transmitted in the bitstream indicating whether MIP mode is applied to the corresponding prediction unit (PU). If MIP mode is applied, the MIP mode index predmode is signaled using an MPM list containing three MPMs.

[0130] The MPM can be derived using the above and left PU intra modes as follows: Three fixed mapping tables map_angular_to_mip idx , idx∈{0,1,2}, and each table represents each conventional intra prediction mode predmode as described in the following equation: Angular Associates a specific MIP mode. predmode MIP =map_angular_to_mip[predmode Angular ] (Formula 5) Here, map_angular_to_mip is a fixed lookup table. The index of the mapping table is determined based on the width W and height H of the PU, and a total of three indexes are available, as explained below. idx(PU)=idx(W,H)∈{0,1,2} (Equation 6) This indicates from which of the three sets the MIP parameters should be obtained above.

[0131] In some examples, to generate an MPM list for a current block coded by MIP mode, the above MIP mode, i.e.

number

number

[0132]

number

number

number

number

number

number

[0133]

number

number

[0134] Finally, the derived

number

number

number

number

[0135] As an example, Figure 27 shows a CU level syntax table in which a flag signaling MIP mode is shown in frame (2701).

[0136] 5.2 MPM List Derivation for Conventional Intra Prediction Modes In some embodiments, MIP modes are harmonized with MPM-based coding of conventional intra-prediction modes as follows: The luma and chroma MPM list derivation process for conventional intra-prediction modes is MIP A separate fixed table, map_mip_to_angular, that maps the idx , idx∈{0,1,2}, predmode Angular =map_mip_to_angular[predmode MIP ] (Formula 11) where map_mip_to_angular is a fixed lookup table. For luma MPM list derivation, if the adjacent luma blocks are in MIP mode predmode MIP Whenever this block is coded by the conventional intra prediction mode predmode Angular For chroma MPM list derivation, whenever the current luma block uses a MIP mode, the same mapping is used to convert the MIP mode to a conventional intra prediction mode.

[0137] III. Enabling Implicit Conversion Selection Based on High-Level or Block-Level Syntax Elements In some embodiments, two transform coding schemes may be used: implicit transforms (also called implicit transform selection) and explicit transforms (also called explicit transform selection).

[0138] In an implicit transform, a group of non-DCT 2 transforms (e.g., DST-1, DCT-5, DST-7, DCT-8, DST-4, DCT-4) can be selected without transform index signaling. For example, the group of non-DCT 2 transforms can be selected using pre-coded information available to both the encoder and the corresponding decoder. The pre-coded information can include, but is not limited to, an intra-prediction mode (e.g., planar mode, DC mode, angular mode), a block size, a block width, a block height, a block aspect ratio, a block area size, an intra-coding mode (e.g., whether multiple reference lines (MRL), ISP, or MIP is used), a position of a selected spatial merge candidate (e.g., top merge candidate, left merge candidate), and an inter-prediction mode (e.g., position-dependent predictive combining (PDPC-inter) mode, combined inter-intra prediction (CIIP) mode).

[0139] In contrast, an explicit transform allows for the selection of one transform from a group of candidate transform types (e.g., DCT-2, DST-1, DCT-5, DST-7, DCT-8, DST-4, DCT-4) with an index signaled to indicate which transform type is selected.

[0140] 1. Enabling implicit conversion when explicit MTS is disabled In some embodiments, for intra-predicted residual blocks that are not coded by ISP, implicit transform selection may be enabled when explicit MTS is disabled for both intra-predicted and inter-predicted residual blocks, as indicated by the high-level syntax (HLS) element, as described in Section II.3.2. For example, if the SPS syntax elements sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both 0, the decoder may determine to enable implicit transform selection for the picture or block associated with the SPS syntax element.

[0141] As an example, Figures 28A-28B show a combination of text (2800) specifying a transform coding process that performs explicit or implicit transform selection for a current block based on associated syntax elements received from the bitstream. The text (2800) can be used as part of a video coding standard. Figure 28A shows two sections (2801) and (2802), while Figure 28B shows one section (2803) and two tables (2804) and (2805).

[0142] Section (2801) describes the input and output of the transform coding process: the position, size, color components, and dequantized transform coefficients of the current block are the inputs, and the residual samples after inverse transform processing using the selected transform are the output.

[0143] Section (2802) describes the derivation of the variable represented by implicitMtsEnabled. The variable indicates whether implicit selection is enabled. As described, if the SPS syntax element sps_mts_enabled_flag is equal to 1 (indicating that MTS is enabled for the picture or block associated with this SPS syntax element), implicitMtsEnabled is equal to 1 if one of the following three conditions is true: (i) ISP is used to code the current block, (ii) SBT is enabled and both sides of the current block are less than or equal to 32, indicating that SBT is used, or (iii) sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both equal to 0 (indicating that explicit MTS is disabled for both intra-coded and inter-coded blocks), and the current block is intra-coded. Otherwise, implicitMtsEnabled is set equal to 0, which indicates that implicit conversion selection is disabled for the current block.

[0144] As described in section (2802), condition (i) allows for enabling implicit conversion for coding an intra-predicted block coded in ISP mode. Condition (iii) allows for enabling implicit conversion for coding an intra-predicted block not coded in ISP mode when explicit MTS is disabled for both the intra-predicted block and the inter-predicted block.

[0145] In section (2803), the vertical and horizontal transforms are determined according to the variable implicitMtsEnabled, the input to the process, and the associated syntax elements. For example, if both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are equal to 0, the current block (for luma component and intra prediction) may be given a horizontal transform kernel (denoted by trTypeHor) and a vertical transform kernel (denoted by trTypeBer) according to equations (3-1) and (3-2), respectively.

[0146] In tables (2804) and (2805), the numbers 1 and 2 for indicating horizontal or vertical transform kernel types indicate DST-7 and DCT-8 transforms, respectively, and the number 0 indicates a DCT-2 transform. In some examples, a DST-4 transform can be used instead of a DST-7 transform.

[0147] 28A-28B, for blocks controlled by the HLS elements sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag, explicit MTS for inter-coded blocks and implicit transforms for intra-coded blocks (non-ISP coding) cannot coexist. However, implicit transform selection for intra-predicted residual blocks does not necessarily depend on the HLS element controlling the MTS that enables inter-predicted residuals (e.g., sps_explicit_mts_inter_enabled_flag). Thus, in some embodiments, for intra-coded blocks not predicted by ISP mode, whether implicit transforms can be applied depends on whether MTS can be applied to intra-predicted residuals (e.g., the value of sps_explicit_mts_intra_enabled_flag), but not on whether MTS can be applied to inter-predicted residuals (the value of sps_explicit_mts_inter_enabled_flag). Such a control mechanism allows for simultaneously enabling both inter MTS (explicit MTS applied to inter-predicted residual blocks) and implicit transforms (for intra-predicted residuals).

[0148] For example, a first and second HLS element may be received at a decoder. The first and second HLS elements control the same set of images or regions (e.g., a video sequence, an image, a slice, a tile, etc.) that may contain intra-coded and inter-coded residual blocks. The first HLS element indicates whether explicit MTS is enabled or disabled for each intra-coded residual block, and the second HLS element indicates whether explicit MTS is enabled or disabled for each inter-coded residual block. With respect to the term inter MTS, the explicit MTS applied to an intra-coded residual block may be referred to as an intra MTS.

[0149] If the first HLS element indicates that explicit MTS is disabled for an intra-coded residual block, the decoder can accordingly decide to enable implicit transforms (or implicit MTS) for the intra-coded block without considering the value of the second HLS element. For example, the second HLS element can be either 0 or 1, which does not affect the enabling of implicit transforms for the intra-coded residual block.

[0150] As an example, Figure 29 shows a modification (2900) to text (2800) corresponding to an implicit conversion enable scheme in which implicit conversion of intra residual blocks and explicit conversion of inter residual blocks can coexist. In the modification (2900), removed text is marked with a strikethrough and added text is marked with an underline. As shown, the condition for sps_explicit_mts_inter_enabled_flag being 0 has been removed.

[0151] 2. Disabling implicit conversions when other coding tools are enabled In some embodiments, whether implicit transforms can be applied to intra-coded blocks (not predicted by the ISP mode) depends on whether a particular coding tool is enabled as indicated by the HLS element.

[0152] 2.1 When NSST is enabled In one embodiment, when an HLS element indicates that NSST is enabled, implicit transforms can be disabled for intra-coded blocks (not predicted by ISP mode). For example, as described in Section II.4.1, a transform matrix T is used in (Equation 3) for secondary transform processing. The transform matrix T can include constant elements and can be designed assuming certain statistics of target coefficient blocks, such as coefficient blocks resulting from explicit transform selection. Therefore, coefficient blocks resulting from implicit transform selection may not match NSST in terms of the statistics of their respective coefficients. For example, applying NSST to the results of implicit transforms does not improve coding performance. For the above reasons, it may be desirable to disable implicit transforms when NSST is enabled.

[0153] In this disclosure, the term NSST may be used to refer to a family of non-separable quadratic transform coding schemes, such as initial designs of NSST, RST, LFNST, etc.

[0154] For example, a decoder may receive two HLS elements, where one HLS element (e.g., sps_explicit_mts_intra_enabled_flag) indicates that explicit MTS is disabled and the other HLS element (e.g., sps_lfnst_enabled_flag) indicates that NSST is enabled. Thus, the decoder may decide not to enable implicit conversion for intra-coded blocks controlled by these two HLS elements. In contrast, if one HLS element indicates that explicit MTS is disabled and the other HLS element also indicates that NSST is disabled, the decoder may decide to enable implicit conversion for the respective intra-coded blocks.

[0155] As an example, Figure 30 shows a modification (3000) to text (2800) corresponding to a scenario in which implicit conversion is enabled when NSST is disabled. As shown, an additional restriction is added to condition (iii) for deriving the variable implicitNtsEnabled: "sps_lfnst_enabled_flag is equal to 0."

[0156] 2.2 When MIP is enabled Similar to the scenario in which NSST is enabled, in some embodiments, implicit transforms are disabled for intra-coded blocks (not predicted by ISP mode) when the HLS element indicates that MIP is enabled. For example, applying an MIP intra-coding mode to a block results in a residual block with different statistics than a residual block coded in a regular intra-coding mode. Thus, a MIP-coded residual block may not match an implicit transform that can assume residual statistics resulting from a regular intra-mode. Therefore, it may be desirable to disable implicit transforms when MIP is enabled.

[0157] For example, a decoder may receive two HLS elements. One HLS element (e.g., sps_explicit_mts_intra_enabled_flag) indicates that explicit MTS is disabled, and the other HLS element (e.g., sps_mip_enabled_flag) indicates that MIP is enabled. Thus, the decoder may decide not to enable implicit transforms for intra-coded blocks controlled by these two HLS elements. In contrast, if both explicit MTS and MIP are disabled, as indicated by the two HLS elements, the decoder may decide to enable implicit transforms for the respective intra-coded blocks.

[0158] As an example, Figure 31 shows a modification (3100) to text (2800) corresponding to a scenario in which implicit conversion is enabled when MIP is disabled. As shown, an additional restriction is added to condition (iii) for deriving the variable implicitNtsEnabled: "sps_mip_enabled_flag is equal to 0."

[0159] 3. Enabling implicit conversion based on block-level syntax element directives In some embodiments, whether an implicit transform (not predicted by the ISP mode) can be applied to an intra-coded block depends on whether MTS, NSST, or MIP is applied to the intra-coded block, as indicated by the block-level syntax element.

[0160] 3.1 When MTS and NSST are not applied In one embodiment, whether an implicit transform can be applied to an intra-coded block depends on whether both MTS and NSST are not applied to the intra-coded block, as indicated by the block-level syntax element.

[0161] For example, the decoder may receive first and second CU-level (or block-level) syntax elements associated with a current block that is intra-coded and not predicted by the ISP mode. The current block may be located at coordinates [x0][y0] within the image.

[0162] The first CU-level syntax element (e.g., tu_mts_idx[x0][y0]) may indicate that a DCT-2 transform may be used for the current block instead of a transform used in MTS (e.g., DST-7, DCT-8, DCT-4, etc.). As an example, in table (2804), when tu_mts_idx[x0][y0] has a value of 0, the trTypeHor and trTypeVer variables both have values ​​of 0, which indicates that the horizontal and vertical transforms are DCT-2 transforms and that a DST-7 or DCT-8 transform is not applied. The second CU-level syntax element (e.g., lfnst_idx[x0][y0]) may indicate that NSST is not applied to the current block. Based on the above first and second CU-level syntax elements, the decoder may determine to enable implicit MTS for the current block.

[0163] In contrast, if the first CU level syntax element and the second CU level syntax element indicate that MTS is applied (tu_mts_idx[x0][y0] has a value of 1, 2, 3, or 4) or that NSST is applied (lfnst_idx[x0][y0] has a non-zero value), the decoder can decide to disable implicit MTS.

[0164] As an example, Figure 32 shows a modification (3200) to text (2800) corresponding to a scenario in which neither MTS nor NSST applies to the current block. As shown, the original condition (iii) in text (2800) is replaced with the condition that the two CU-level syntax elements "tu_mts_idx[x0][y0] and lfnst_idx[x0][y0] are both equal to 0."

[0165] 3.2 When MTS, NSST, and MIP are not applied In one embodiment, whether an implicit transform can be applied to an intra-coded block depends on whether MTS, NSST, and MIP are not applied to the intra-coded block, as indicated by the block-level syntax elements. Compared to Section III.3.1, another coding tool, MIP, is further considered.

[0166] For example, the decoder may receive first, second, and third CU-level (or block-level) syntax elements associated with a current block that is intra-coded and not predicted by the ISP mode. The current block may be located at coordinates [x0][y0] within the image.

[0167] The first CU-level syntax element (e.g., tu_mts_idx[x0][y0]) may indicate that a DCT-2 transform can be used for the current block instead of the transform used in MTS. The second CU-level syntax element (e.g., lfnst_idx[x0][y0]) may indicate that NSST is not applied to the current block. The third CU-level syntax element (e.g., tu_mip_flag) may indicate that MIP is not applied to the current block. Based on the above three CU-level syntax elements, the decoder can decide to enable implicit MTS for the current block.

[0168] In contrast, if the above three CU-level syntax elements indicate that MTS is applied (tu_mts_idx[x0][y0] has a value of 1, 2, 3, or 4), NSST is applied (lfnst_idx[x0][y0] has a non-zero value), or MIP is applied (tu_mip_flag has a value of 1), the decoder can decide to disable implicit MTS.

[0169] As an example, Figure 33 shows a modification (3300) to text (2800) corresponding to a scenario in which neither MTS, NSST, nor MIP applies to the current block. As shown, the original condition (iii) in text (2800) is replaced with the condition that the three CU-level syntax elements "tu_mts_idx[x0][y0], intra_mip_flag[x0][y0], and lfnst_idx[x0][y0] are all equal to 0."

[0170] 4. Example of the transformation coding process 34-36 show flowcharts of transform coding processes (3400), (3500), and (3600) according to some embodiments of the present disclosure. The processes (3400), (3500), and (3600) can be used in an inverse transform process at a decoder to generate residual blocks for blocks being reconstructed. In various embodiments, the processes (3400), (3500), and (3600) can be performed by processing circuitry, such as processing circuitry of terminal devices (210), (220), (230), and (240), processing circuitry performing the functions of a video decoder (310), or processing circuitry performing the functions of a video decoder (410). In some embodiments, the processes (3400), (3500), and (3600) can be implemented by software instructions, such that the processing circuitry performs the processes (3400), (3500), and (3600) when the processing circuitry executes the software instructions.

[0171] 4.1 Process (3400): Enabling implicit conversion when explicit MTS is disabled The process (3400) starts from (S3401) and proceeds to (S3410).

[0172] At (S3410), a first HLS element is received at the decoder. The first HLS element may indicate whether explicit MTS is disabled for an intra-coded block. For example, the intra-coded block may be referred to as the current block because it is being reconstructed. The first HLS element may be associated with a set of coding blocks that includes the current block.

[0173] At (S3420), a second HLS element is received at the decoder. The second HLS element may indicate whether explicit MTS is enabled for inter-coded blocks. For example, the second HLS element may be associated with the same set of coding blocks as the first HLS element. Both the current block and the inter-coded block are included in the set of coding blocks.

[0174] At (S3430), the decoder may determine to enable implicit MTS for the current block if the first HLS element indicates that explicit MTS is disabled for intra-coded blocks and the second HLS element indicates that explicit MTS is enabled for inter-coded blocks. The process (3400) may proceed to (S3499), where it ends.

[0175] 4.2 Process (3500): Disable implicit conversions when other coding tools are enabled The process (3500) starts from (S3501) and proceeds to (S3510).

[0176] At (S3510), a first HLS element is received at a decoder. The first HLS element may indicate whether explicit MTS is enabled for an intra-coded block. The intra-coded block may be a block being reconstructed. The first HLS element controls a set of coding blocks that includes the intra-coded block.

[0177] At (S3520), a second HLS element is received at the decoder. The second HLS element can indicate whether NSST or MIP is disabled for intra-coded blocks. For example, the second HLS element can control the same set of coding blocks as the first HLS element.

[0178] At (S3530), the decoder may determine whether to enable implicit MTS for processing intra-coded blocks based on the first HLS element and the second HLS element. For example, the decoder may determine to enable implicit MTS for intra-coded blocks if the first HLS element indicates that explicit MTS is disabled for the intra-coded blocks and the second HLS element indicates that NSST or MIP is disabled for the intra-coded blocks. Alternatively, the decoder may determine to disable implicit MTS for intra-coded blocks if the first HLS element indicates that explicit MTS is disabled for the intra-coded blocks but the second HLS element indicates that NSST or MIP is enabled for the intra-coded blocks. The process (3500) may proceed to (S3599) and end at (S3599).

[0179] 4.3 Process (3600): Enabling implicit conversion based on block-level syntax element indication The process (3600) starts from (S3601) and proceeds to (S3610).

[0180] At S (3610), an intra-coded block is received at a decoder. The intra-coded block may be associated with a first block-level syntax element indicating whether MTS applies and a second block-level syntax element indicating whether NSST applies. For example, the intra-coded block belongs to a CU that includes a CU-level syntax element that includes the first and second block-level syntax elements.

[0181] At (S3620), the decoder may determine whether to enable implicit MTS for the intra-coded block based on the first block level syntax element and the second block level syntax element. For example, the decoder may determine to enable implicit MTS for the intra-coded block if the first block level syntax element indicates that MTS is not applied and the second block level syntax element indicates that NSST is not applied. Alternatively, the decoder may determine to disable implicit MTS for the intra-coded block if the first block level syntax element indicates that MTS is not applied but the second block level syntax element indicates that NSST is applied. Process (3600) may proceed to (S3699) and end at (S3699).

[0182] IV. Computer Systems The techniques described above can be implemented using computer-readable instructions and as computer software physically stored on one or more computer-readable media. For example, Figure 37 illustrates a computer system (3700) suitable for implementing certain embodiments of the disclosed subject matter.

[0183] Computer software can be coded using any suitable machine code or computer language that can be subjected to assembly, compilation, linking, or similar mechanisms to produce code containing instructions that can be executed directly or through translation, microcode execution, etc. by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc.

[0184] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0185] 37 for computer system (3700) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure, nor should the arrangement of components be interpreted as having a dependency or requirement related to any one or combination of components illustrated in the exemplary embodiment of computer system (3700).

[0186] The computer system (3700) may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, through tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). Human interface devices may also be used to capture certain media that do not necessarily involve direct conscious human input, such as sound (e.g., speech, music, environmental sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and video (e.g., two-dimensional video, three-dimensional video, including stereoscopic video).

[0187] The input human interface devices may include one or more (only one of each shown) of a keyboard (3701), a mouse (3702), a trackpad (3703), a touchscreen (3710), a data glove (not shown), a joystick (3705), a microphone (3706), a scanner (3707), and a camera (3708).

[0188] The computer system (3700) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (3710), data gloves (not shown), or joystick (3705), although some haptic feedback devices may not function as input devices), audio output devices (such as speakers (3709), headphones (not shown)), visual output devices (such as screens (3710), including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capability and with or without haptic feedback capability—some of which may output two-dimensional visual output or output in more than three dimensions via means such as stereographic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0189] The computer system (3700) may also include human-accessible storage and associated media such as optical media including CD / DVD ROM / RW (3720) with CD / DVD or similar media (3721), thumb drives (3722), removable hard drives or solid state drives (3723), conventional magnetic media such as tape and floppy disks (not shown), and dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0190] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not encompass transmission media, carrier waves, or other transitory signals.

[0191] The computer system (3700) may also include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, or optical. Furthermore, the networks may be local, wide-area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, and the like. Examples of networks include local area networks such as Ethernet and wireless LAN; cellular networks including GSM, 3G, 4G, 5G, LTE, and the like; television wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV; and vehicular and industrial networks including CAN Bus. Certain networks generally require an external network interface adapter connected to a particular general-purpose data port or peripheral bus (3749) (e.g., a USB port on the computer system (3700)). Other networks are generally integrated into the core of the computer system (3700) by attachment to a system bus, as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (3700) can communicate with other entities. Such communication may be unidirectional, receive only (e.g., broadcast TV), unidirectional transmit only (e.g., from a CANbus to a particular CANbus device), or bidirectional, e.g., communication to other computer systems using local-area or wide-area digital networks. As noted above, specific protocols and protocol stacks may be used with each of these networks and network interfaces.

[0192] The aforementioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core (3740) of the computer system (3700).

[0193] The cores (3740) may include one or more central processing units (CPUs) (3741), graphics processing units (GPUs) (3742), dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) (3743), hardware accelerators for specific tasks (3744), etc. These devices, along with internal mass storage devices such as read-only memory (ROM) (3745), random access memory (3746), and non-user-accessible internal hard drives, SSDs (3747), may be connected via a system bus (3748). In some computer systems, the system bus (3748) is accessible in the form of one or more physical plugs, allowing expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (3748) or via a peripheral bus (3749). Peripheral bus architectures include PCI, USB, etc.

[0194] The CPU (3741), GPU (3742), FPGA (3743), and accelerator (3744) can execute specific instructions that, in combination, can constitute the above-mentioned computer code. That computer code can be stored in ROM (3745) or RAM (3746). Transient data can also be stored in RAM (3746), while persistent data can be stored, for example, in internal mass storage (3747). Rapid storage and retrieval from any of the memory devices can be enabled through the use of cache memory, which can be closely associated with one or more of the CPU (3741), GPU (3742), mass storage (3747), ROM (3745), RAM (3746), etc.

[0195] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The media and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0196] By way of example and not limitation, the architecture (3700), and specifically a computer system having a core (3740), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be user-accessible mass storage devices, as introduced above, as well as media associated with the core's (3740) specific storage device of a non-transitory nature, such as the core's internal mass storage device (3747) or ROM (3745). Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (3740). The computer-readable media can include one or more memory devices or chips according to particular needs. The software can cause the core (3740) and the processor therein (including a CPU, GPU, FPGA, etc.) to perform particular processes or particular portions of particular processes described herein, such as defining data structures stored in RAM (3746) and modifying such data structures according to software-defined processes. Additionally or alternatively, a computer system may provide functionality as a result of logic embedded in hardware or embedded in circuitry (e.g., accelerator (3744)), which can operate in place of or together with software to perform particular processes or portions of particular processes described herein. References to software can include logic, and vice versa, where applicable. References to computer-readable media can encompass software for execution, circuitry embodying logic for execution, or circuitry (such as an integrated circuit (IC)) storing both, where applicable. The present disclosure encompasses any suitable combination of hardware and software. Appendix A: Abbreviations AMT: Adaptive Multiplexing ASIC: Application Specific Integrated Circuit BMS: Benchmark Set CANBus: Controller Area Network Bus CCLM: Cross-component linear model CD: Compact Disc COT: Composite Orthonormal Transform CPU: Central Processing Unit CRT: cathode ray tube CTB: coding tree block CTU: Coding Tree Unit CU: Coding Unit DVD: Digital Video Disc EMT: Extended Multiplex Transform FPGA: Field Programmable Gate Area GOP: Group of Pictures GPU: Graphics Processing Unit GSM: Global System for Mobile Communications HDR: High Dynamic Range HEVC: High Efficiency Video Coding HLS: High Level Syntax HRD: Hypothetical Reference Decoder IBC: Intra-block copy IC: Integrated Circuit IDT: Identity Transformation ISP: Intra-subpartitioning JEM: Joint Exploration Model JVET: Joint Video Exploration Team KLT: Karhunen-Loeve transformation LAN: Local Area Network LCD: Liquid crystal display LFNST: Low-Frequency Nonseparable Quadratic Transform LTE: Long Term Evolution MIP: Matrix-based intra prediction mode MRL (or MRLP): Multiple Reference Line Prediction MTS: Multiple conversion selection MV: Motion Vector NSST: Nonseparable Quadratic Transformation OLED: Organic Light Emitting Diode PB: Predicted Block PCI: Peripheral Component Interconnect PLD: Programmable Logic Device PPS: Image Parameter Set PU: Prediction Unit RAM: Random Access Memory ROM: Read-only memory RST: Reduced size conversion SBT: Sub-Block Transform SDR: Standard Dynamic Range SEI: Supplemental Extended Information SNR: Signal to Noise Ratio SPS: Sequence Parameter Set SSD: Solid State Drive SVT: Spatial Variation Transform TSM:Conversion Skip Mode TU: conversion unit, USB: Universal Serial Bus VPS: Video Parameter Set VUI: Video Usability Information VVC: Versatile Video Coding

[0197] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope. [Explanation of symbols]

[0198] 101 Current Block 200 Communication Systems 210 Terminal Equipment 220 Terminal Equipment 230 Terminal Equipment 240 Terminal Equipment 250 Network 300 Communication Systems 301 Video Sources 302 Video Images 303 Video Encoder 304 Encoded Video Data 305 Streaming Server 306 Client Subsystem 307 Copy 308 Client Subsystem 309 copies 310 Video Decoder 311 Video Images 312 Display 313 Capture Subsystem 320 Electronic Devices 330 Electronic Devices 401 Channel 410 Video Decoder 412 Rendering Devices 415 Buffer Memory 420 Parser 421 Symbol 430 Electronic Devices 431 Receiver 451 Scaler / Descaler Unit 452 Intra-Image Prediction Unit 453 Motion Compensation Prediction Unit 455 Aggregator 456 Loop Filter Unit 457 Reference Image Memory 458 Current Image Buffer 501 Video Sources 503 Video Encoder 520 Electronic Devices 530 Source Coder 532 Coding Engine 533 Local Video Decoder 534 Reference Image Memory 535 Predictor 540 Transmitter 543 coded video sequences 545 Entropy Coder 550 Controller 560 Communication Channels 603 Video Encoder 621 General-purpose controller 622 Intra Encoder 623 Residual Calculator 624 Residual Encoder 625 Entropy Encoder 626 Switch 628 Residual Decoder 630 Interencoder 710 Video Decoder 771 Entropy Decoder 772 Intra Decoder 773 Residual Decoder 774 Reconstruction Module 780 Interdecoder Line 1301 Line 1302 Line 1303 Line 1304 Line 1305 1500 tables 1900 table 2000 Alternative Transformation Coding Process 2010 Forward Linear Transform 2012 Forward Quadratic Transformation 2013 Coefficient Block 2016 Bitstream 2020 Inverse Linear Transform 2022 Inverse Quadratic Transformation 2023 coefficient block 2024 Inverse quantization 2100 Alternative Transformation Coding Process 2200 CU Level Syntax Table 2301 The process of reduction transformation 2302 The process of inverse reduction 2410 Residual Block 2500 tables 2600 MIP mode example process 2610 Rectangular Block 2621 subsampled sets 2622 remaining positions 2701 frames 2800 Text Section 2801 2802 Section 2803 Section 2804 table 2805 table 2900 Fix 3000 Modification 3100 Fix 3200 Modification 3300 Modification 3400 Conversion Coding Process 3500 Conversion Coding Process 3600 Conversion Coding Process 3700 Computer Systems 3701 Keyboard 3702 Mouse 3703 Trackpad 3705 Joystick 3706 Microphone 3707 Scanner 3708 Camera 3709 Speaker 3710 Touchscreen 3720 CD / DVD ROM / RW 3721 Medium 3722 thumb drive 3723 Solid State Drive 3740 cores 3741 Central Processing Unit (CPU) 3742 Graphics Processing Unit (GPU) 3743 Field Programmable Gate Area (FPGA) 3744 Hardware Accelerator 3745 Read-Only Memory (ROM) 3746 Random Access Memory 3747 Internal Mass Storage 3748 System Bus 3749 Peripheral bus

Claims

1. 1. A method of video decoding in a video decoder, comprising: receiving a first high level syntax (HLS) element indicating whether multiple transform selection (MTS) is enabled or disabled for a video sequence; receiving a transform unit corresponding to an intra-coded block, the intra-coded block being associated with a second HLS element indicating whether explicit Multiple Transform Selection (MTS) is enabled or disabled for the intra-coded block, a variable CuPredMode[xTbY][yTbY] indicating whether the current block is intra-coded, and a third HLS element indicating whether Non-Separable Secondary Transform (NSST) is disabled or enabled for the intra-coded block; If the first HLS element indicates that MTS is enabled, The following conditions: the second HLS element indicating that the explicit MTS is disabled; the variable CuPredMode[xTbY][yTbY] is MODE_INTRA, indicating that the current block is intra-coded; and the third HLS element indicating that the NSST does not apply; enabling implicit MTS for the intra-coded block if all of the following are satisfied: A method comprising:

2. applying the implicit MTS to the intra-coded block. The method of claim 1 further comprising:

3. The method of claim 2 , wherein in the step of applying the implicit MTS to the intra-coded block, a transform type for processing the intra-coded block is determined according to a size of the intra-coded block.

4. The method of claim 1 , wherein the intra-coded block is not coded in an intra-subpartitioning (ISP) mode.

5. Apparatus configured to carry out the method of any one of claims 1 to 4.

6. A computer program comprising instructions which, when executed by at least one processor, cause said at least one processor to perform the method of any one of claims 1 to 4.

7. 1. A method of video encoding in a video encoder, comprising: determining a first high level syntax (HLS) element indicating whether multiple transform selection (MTS) is enabled or disabled for the video sequence; determining a transform unit corresponding to a current block, the current block being associated with a second HLS element indicating whether explicit multiple transform selection (MTS) is enabled or disabled for the current block, a variable CuPredMode[xTbY][yTbY] indicating whether the current block is intra-coded, and a third HLS element indicating whether non-separable secondary transform (NSST) is disabled or enabled for the current block; If the first HLS element indicates that MTS is enabled, The following conditions: the second HLS element indicating that the explicit MTS is disabled; The variable CuPredMode[xTbY][yTbY] is MODE_INTRA, indicating that the current block is intra-coded; and the third HLS element indicating that the NSST does not apply; If all of the following are satisfied, implicit MTS is enabled for the current block; determining a transform unit corresponding to the current block; A method comprising:

8. An apparatus configured to perform the method of claim 7.

9. A computer program comprising instructions that, when executed by at least one processor, cause the at least one processor to perform the method of claim 7.

10. 1. A method for transmitting a video bitstream, performed by a video encoder, comprising: generating a video bitstream, generating the video bitstream comprises: determining a first high level syntax (HLS) element indicating whether multiple transform selection (MTS) is enabled or disabled for the video sequence; determining a transform unit corresponding to a current block, the current block being associated with a second HLS element indicating whether explicit multiple transform selection (MTS) is enabled or disabled for the current block, a variable CuPredMode[xTbY][yTbY] indicating whether the current block is intra-coded, and a third HLS element indicating whether non-separable secondary transform (NSST) is disabled or enabled for the current block; If the first HLS element indicates that MTS is enabled, The following conditions: the second HLS element indicating that the explicit MTS is disabled; The variable CuPredMode[xTbY][yTbY] is MODE_INTRA, indicating that the current block is intra-coded; and the third HLS element indicating that the NSST does not apply; If all of the following are satisfied, implicit MTS is enabled for the current block; determining a transform unit corresponding to the current block; generating a video bitstream, transmitting the video bitstream; A method comprising:

11. An apparatus configured to perform the method of claim 10.

12. 11. A computer program comprising instructions that, when executed by at least one processor, cause the at least one processor to perform the method of claim 10.

Images