INTERACTION BETWEEN SECONDARY TRANSFORMATION AND SUB-BLOCK TRANSFORMATION

Abstract

The present invention relates to a device (e.g., a decoder for video decoding) that can determine that a sub-block transform (SBT) is enabled for a video block. The device can determine whether a secondary transform (e.g., a low-frequency non-separable transform (LFNST) or a non-separable primary transform (NSPT)) is enabled for the video block. The device can determine a transform function to decode the video block based on determining whether the secondary transform is enabled for the video block. For example, a first type of transform function (e.g., discrete cosine transform (DCT) II) can be used to decode the video block if the LFNST or NSPT is enabled for the video block. A second type of transform function can be used to decode the video block if the secondary transform is disabled for the video block. The device can decode the video block using the transform function.