Image encoding / decoding apparatus and apparatus for transmitting data

Abstract

Image encoding / decoding apparatus and apparatus for transmitting data. An image decoding method according to the present document includes the steps of determining whether to parse an MTS index for applying MTS to a current block, deriving residual samples of the current block by applying MTS to the current block based on the MTS index, and generating a reconstructed picture based on the residual samples, wherein the step of determining whether to parse the MTS index includes determining a tree type of the current block, a partition type of the current block, and whether a zero-out for MTS has been performed on the current block.

Description

[0001] This application is a divisional application of the original invention patent application No. 202080083271.1 (International Application No.: PCT / KR2020 / 013492, Application Date: October 5, 2020, Invention Title: Transformation-Based Image Coding Method and Apparatus). Technical Field

[0002] This disclosure relates to an image coding technique, and more specifically, to a method and apparatus for encoding images based on transformations in an image coding system. Background Technology

[0003] Today, the demand for high-resolution and high-quality images / videos, such as 4K, 8K, or even higher Ultra High Definition (UHD) images / videos, is constantly growing across various fields. As image / video data becomes higher resolution and higher quality, the amount of information or bits transmitted increases compared to traditional image data. Therefore, transmission and storage costs increase when using media such as traditional wired / wireless broadband lines to transmit image data or when using existing storage media to store image / video data.

[0004] In addition, there is increasing interest and demand for immersive media such as virtual reality (VR) and artificial reality (AR) content or holograms, and broadcasting of images / videos with image characteristics that differ from real images such as game images is on the rise.

[0005] Therefore, there is a need for efficient image / video compression techniques to effectively compress, transmit, store, and reproduce information with high resolution and high quality images / videos that have the various characteristics described above. Summary of the Invention

[0006] Technical issues

[0007] One aspect of this disclosure is to provide a method and apparatus for increasing image coding efficiency.

[0008] Another technical aspect of this disclosure is to provide a method and apparatus for increasing the efficiency of transform index coding.

[0009] Another technical aspect of this disclosure is to provide an image encoding method and apparatus using MTS.

[0010] Another technical aspect of this disclosure is to provide an image encoding method and apparatus using an MTS index.

[0011] Technical solution

[0012] According to embodiments of this disclosure, an image decoding method performed by a decoding device is provided. The method includes: determining whether to parse an MTS index used to apply an MTS to a current block; deriving a residual sample of the current block by applying the MTS to the current block based on the MTS index; and generating a reconstructed image based on the residual sample, wherein determining whether to parse the MTS index determines the tree type of the current block, the partition type of the current block, and whether to perform a zeroing operation on the current block for the MTS.

[0013] The MTS index can be resolved when the tree type of the current block is not dual-tree chromatic and the LFNST index indicating the LFNST kernel applied to the current block is 0.

[0014] The MTS index can be resolved if the larger of the width and height of the current block is less than or equal to 32.

[0015] The MTS index can be resolved when the current block has not been divided into multiple sub-blocks and the sub-block transformation performed by dividing the coding unit has not been applied to the current block.

[0016] Determine whether to perform zeroing for the MTS. Determine whether there is a valid coefficient in the second region of the current block, excluding the first region in the upper left of the current block where valid coefficients can exist. If there is no valid coefficient in the second region, the MTS index can be resolved.

[0017] The LFNST index and the MTS index can be signaled at the coding unit level, and the MTS index can be signaled immediately after the signaling of the LFNST index.

[0018] According to another embodiment of this disclosure, an image encoding method performed by an encoding device is provided. The method includes: deriving transform coefficients of a current block based on the MTS for residual samples; and encoding residual information derived through the quantization of the transform coefficients and an MTS index indicating the MTS kernel, wherein the MTS index is encoded based on the tree type of the current block, the partition type of the current block, and whether a zeroing operation for the MTS is performed on the current block.

[0019] According to another embodiment of the present disclosure, a digital storage medium may be provided that stores image data including a bitstream and encoded image information generated according to an image encoding method performed by an encoding device.

[0020] According to another embodiment of the present disclosure, a digital storage medium can be provided that stores image data including encoded image information and bitstreams to enable a decoding device to perform an image decoding method.

[0021] Technical effect

[0022] According to this disclosure, the overall image / video compression efficiency can be increased.

[0023] According to this disclosure, the efficiency of transform index encoding can be increased.

[0024] Another technical aspect of this disclosure provides an image encoding method and apparatus using MTS.

[0025] Another technical aspect of this disclosure provides an image encoding method and apparatus using MTS.

[0026] The effects achievable through the specific examples of this disclosure are not limited to those listed above. For example, various technical effects may exist that can be understood or deduced by a person skilled in the art based on this disclosure. Therefore, the specific effects of this disclosure are not limited to those expressly described herein, but may include various effects that can be understood or deduced based on the technical features of this disclosure. Attached Figure Description

[0027] Figure 1 Examples of video / image coding systems to which this disclosure can be applied are illustrated schematically.

[0028] Figure 2 This is a diagram that schematically illustrates the configuration of a video / image encoding device to which this disclosure can be applied.

[0029] Figure 3 This is a diagram that schematically illustrates the configuration of a video / image decoding device to which this disclosure can be applied.

[0030] Figure 4 The structure of a content streaming system applying this disclosure is illustrated.

[0031] Figure 5 Multiple transformation techniques according to embodiments of the present disclosure are illustrated schematically.

[0032] Figure 6 The intra-frame orientation patterns for 65 predicted directions are schematically shown.

[0033] Figure 7 This is a diagram used to illustrate an embodiment of the RST according to the present disclosure.

[0034] Figure 8 This is a diagram illustrating the order in which the output data of a forward first transformation is arranged into a one-dimensional vector, based on the example.

[0035] Figure 9 This is a diagram illustrating the order in which the output data of the forward quadratic transform is arranged into two-dimensional blocks, based on the example.

[0036] Figure 10This is a diagram illustrating a wide-angle intra-frame prediction mode according to an implementation method described in this document.

[0037] Figure 11 This is a diagram illustrating the block shape to which LFNST is applied.

[0038] Figure 12 This is a diagram illustrating the arrangement of the output data of the positive LFNST according to the example.

[0039] Figure 13 This is a diagram illustrating that the amount of output data for the positive LFNST, as shown in the example, is limited to a maximum of 16.

[0040] Figure 14 This is a diagram illustrating the zeroing process in a block of 4×4LFNST, based on the example.

[0041] Figure 15 This is a diagram illustrating the zeroing process in a block of 8×8 LFNST, based on the example.

[0042] Figure 16 This is a diagram illustrating the zeroing of a block in an 8×8 LFNST application, based on another example.

[0043] Figure 17 This is a diagram illustrating an example of how a coded block is divided into sub-blocks.

[0044] Figure 18 This is a diagram illustrating another example of how a coded block is divided into sub-blocks.

[0045] Figure 19 This is a diagram illustrating the symmetry between the M×2 (M×1) block and the 2×M (1×M) block according to the example.

[0046] Figure 20 This is a diagram illustrating an example of transposing a 2×M block according to the example.

[0047] Figure 21 The scanning order of the 8×2 or 2×8 region is shown according to the example.

[0048] Figure 22 This is a flowchart illustrating the operation of a video decoding device according to an embodiment of the present disclosure.

[0049] Figure 23 This is a flowchart illustrating the operation of a video encoding device according to an embodiment of the present disclosure. Detailed Implementation

[0050] While this disclosure may be readily modified and includes various embodiments, specific embodiments thereof have been illustrated by way of example in the accompanying drawings and will now be described in detail. However, this is not intended to limit this disclosure to the specific embodiments disclosed herein. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the technical concept of this disclosure. The singular form may include the plural form unless the context clearly indicates otherwise. Terms such as “comprising” and “having” are intended to indicate the presence of the features, numbers, steps, operations, elements, components, or combinations thereof used in the following description, and should therefore not be construed as pre-excluding the possibility of the presence or addition of one or more different features, numbers, steps, operations, elements, components, or combinations thereof.

[0051] Furthermore, for ease of description of their different features and functions, the components in the accompanying drawings described herein are illustrated independently; however, this does not imply that each component is implemented by a separate piece of hardware or software. For example, any two or more of these components may be combined to form a single component, and any single component may be divided into multiple components. Embodiments in which components are combined and / or divided will fall within the scope of this disclosure, provided they do not depart from the spirit of this disclosure.

[0052] In the following description, preferred embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Furthermore, in the drawings, the same reference numerals are used for the same components, and repeated descriptions of the same components will be omitted.

[0053] This document relates to video / image coding. For example, the methods / examples disclosed in this document may relate to the VVC (Video Coding Universal) standard (ITU-T Rec.H.266), next-generation video / image coding standards after VVC, or other video coding-related standards (e.g., HEVC (High Efficiency Video Coding) standard (ITU-T Rec.H.265), EVC (Essential Video Coding) standard, AVS2 standard, etc.).

[0054] This document provides various implementations related to video / image encoding, and these implementations may be combined and performed in combination with each other unless otherwise specified.

[0055] In this document, video can refer to a collection of images over a period of time. Typically, an image is a unit representing a specific time region, while a strip / patch is a unit that constitutes a part of an image. A strip / patch can include one or more coding tree units (CTUs). An image can consist of one or more strips / patches. An image can consist of one or more patch groups. A patch group can include one or more patches.

[0056] A pixel or primitive (pel) can refer to the smallest unit that makes up a picture (or image). Alternatively, "sample" can be used as the term corresponding to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component. Alternatively, a sample can refer to a pixel value in the spatial domain, or, when the pixel value is transformed to the frequency domain, it can refer to the transform coefficients in the frequency domain.

[0057] A unit can represent the basic unit of image processing. A unit may include a specific region and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. Depending on the context, units and terms such as blocks and regions may be used interchangeably. Typically, an M×N block may include a set (or array) of samples or transform coefficients consisting of M columns and N rows.

[0058] In this document, the terms " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" can mean "A and / or B". Additionally, "A, B" can mean "A and / or B". Furthermore, "A / B / C" can mean "at least one of A, B, and / or C". Additionally, "A / B / C" can mean "at least one of A, B, and / or C".

[0059] Additionally, in this document, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" could include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term "or" in this document should be interpreted as indicating "additionally or alternatively".

[0060] In this disclosure, "at least one of A and B" can mean "only A", "only B" or "both A and B". Furthermore, in this disclosure, the expression "at least one of A or B" or "at least one of A and / or B" can be interpreted as "at least one of A and B".

[0061] Furthermore, in this disclosure, "at least one of A, B, and C" may mean "A only", "B only", "C only" or "any combination of A, B, and C". Additionally, "at least one of A, B, or C" or "at least one of A, B, and / or C" may mean "at least one of A, B, and C".

[0062] Additionally, the parentheses used in this disclosure can indicate "for example". Specifically, when indicated as "prediction (intra-frame prediction)", it can mean that "intra-frame prediction" is proposed as an example of "prediction". In other words, "prediction" in this disclosure is not limited to "intra-frame prediction", and "intra-frame prediction" is proposed as an example of "prediction". Furthermore, when indicated as "prediction (i.e., intra-frame prediction)", this can also mean that "intra-frame prediction" is proposed as an example of "prediction".

[0063] The technical features described individually in one of the accompanying drawings of this disclosure may be implemented individually or simultaneously.

[0064] Figure 1 Examples of video / image coding systems to which this disclosure can be applied are illustrated schematically.

[0065] Reference Figure 1 A video / image encoding system may include a first device (source device) and a second device (receiving device). The source device may transmit encoded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or network.

[0066] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0067] Video sources can be obtained through processes that capture, synthesize, or generate video / images. Video sources may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, video / image archives including previously captured video / images, etc. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated by computers, etc. In this case, the video / image capture process can be replaced by a process that generates related data.

[0068] Encoding devices can encode input video / images. They can perform a series of processes such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0069] A transmitter can send encoded video / image information or data, output in bitstream form, to a receiver in a receiving device via a digital storage medium or network, either as a file or a stream. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiver can receive / extract the bitstream and send the received / extracted bitstream to a decoding device.

[0070] Decoding devices can decode video / images by performing a series of processes such as dequantization, inverse transform, and prediction, which correspond to the operations of encoding devices.

[0071] The renderer can render decoded video / images. The rendered video / images can then be displayed on a monitor.

[0072] Figure 2 This diagram schematically illustrates the configuration of a video / image encoding apparatus to which this disclosure may be applied. In the following, the term "video encoding apparatus" may include an image encoding apparatus.

[0073] Reference Figure 2 The encoding device 200 may include an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 described above may be constituted by one or more hardware components (e.g., an encoder chipset or processor). Furthermore, the memory 270 may include a decoded picture buffer (DPB) and may be constituted by a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0074] Image partitioner 210 can divide an input image (or picture or frame) input to encoding device 200 into one or more processing units. As an example, a processing unit may be referred to as a coding unit (CU). In this case, starting from a coding tree unit (CTU) or a maximum coding unit (LCU), the coding units can be recursively partitioned according to a quadtree-binary-tritree (QTBTTT) structure. For example, based on a quadtree structure, a binary tree structure, and / or a ternary tree structure, a coding unit can be partitioned into multiple coding units of varying depths. In this case, for example, a quadtree structure can be applied first, and a binary tree structure and / or a ternary tree structure can be applied later. Alternatively, a binary tree structure can be applied first. The encoding process according to this disclosure can be performed based on the final coding units without further partitioning. In this case, the maximum coding unit can be directly used as the final coding unit based on the encoding efficiency according to the image characteristics. Alternatively, the coding units can be recursively partitioned into deeper coding units as needed, thereby allowing the optimally sized coding unit to be used as the final coding unit. Here, the encoding process may include processes such as prediction, transformation, and reconstruction, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be separate from or distinct from the final encoding unit described above. The prediction unit may be a unit for predicting samples, and the transformation unit may be a unit for deriving the transform coefficients and / or a unit for deriving the residual signal from the transform coefficients.

[0075] Depending on the context, units and terms such as blocks and regions can be used to represent each other. Typically, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. Samples can typically represent pixels or pixel values, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component. Samples can be used as a term corresponding to pixels or primitives (pellets) in a picture (or image).

[0076] Subtractor 231 subtracts the predicted signal (predicted block, predicted sample array) output from predictor 220 from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is sent to converter 232. Predictor 220 can perform prediction on the processing target block (hereinafter referred to as "current block") and can generate a prediction block that includes the prediction samples of the current block. Predictor 220 can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. As discussed later in the description of each prediction mode, the predictor can generate various prediction-related information such as prediction mode information and send the generated information to entropy encoder 240. The prediction information can be encoded in entropy encoder 240 and output as a bitstream.

[0077] Intra-predictor 222 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the reference samples can be located near or separate from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Depending on the level of detail in the prediction direction, the directional modes can include, for example, 33 or 65 directional prediction modes. However, this is just an example, and more or fewer directional prediction modes can be used depending on the settings. Intra-predictor 222 can determine the prediction mode to be applied to the current block by using the prediction modes applied to neighboring blocks.

[0078] Inter-frame predictor 221 can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference image. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted based on blocks, sub-blocks, or samples, according to the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally neighboring block may be the same as or different from each other. The temporally neighboring block may be referred to as a juxtaposed reference block, a juxtaposed CU (colCU), etc., and the reference image including the temporally neighboring block may be referred to as a juxtaposed image (colPic). For example, inter-frame predictor 221 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference image index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in jump mode and merge mode, the inter-frame predictor 221 can use motion information of neighboring blocks as motion information of the current block. In jump mode, unlike merge mode, residual signals cannot be sent. In motion information prediction (motion vector prediction, MVP) mode, motion vectors of neighboring blocks can be used as motion vector predictors, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0079] Predictor 220 can generate prediction signals based on various prediction methods. For example, the predictor can apply intra-frame prediction or inter-frame prediction to the prediction of a block, and can also apply intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as combined intra-frame and inter-frame prediction (CIIP). Additionally, the predictor can perform prediction on a block based on an intra-block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or palette mode can be used for content image / video encoding such as games, etc. Although IBC essentially performs prediction within the current block, its execution is similar to inter-frame prediction in that it derives a reference block within the current block. That is, IBC can use at least one of the inter-frame prediction techniques described in this disclosure.

[0080] The predicted signals generated by the inter-frame predictor 221 and / or the intra-frame predictor 222 can be used to generate the reconstructed signal or the residual signal. The transformer 232 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques can include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented as a graph. CNT refers to a transform obtained based on the predicted signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size, or to blocks of variable size that are not square.

[0081] Quantizer 233 quantizes the transform coefficients and sends them to entropy encoder 240, which encodes the quantized signal (information about the quantized transform coefficients) and outputs the encoded signal in a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 233 can rearrange the block-type quantized transform coefficients into a one-dimensional vector based on the coefficient scan order and generate information about the quantized transform coefficients based on this one-dimensional vector form. Entropy encoder 240 can perform various encoding methods such as exponential Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). Entropy encoder 240 can encode information required for video / image reconstruction, other than the quantized transform coefficients (e.g., values ​​of syntax elements), either together or separately. The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form on a unit-by-unit basis in the Network Abstraction Layer (NAL). The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS). Additionally, the video / image information may include general constraint information. In this disclosure, information and / or syntax elements sent from the encoding device to / signaled to the decoding device may be included in the video / image information. The video / image information can be encoded using the encoding process described above and included in the bitstream. The bitstream can be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcast networks, communication networks, and / or the like, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that sends the signal output from the entropy encoder 240 or a memory (not shown) that stores it may be configured as an internal / external element of the encoding device 200, or the transmitter may be included in the entropy encoder 240.

[0082] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, by applying dequantization and inverse transform using vectorized transform coefficients via dequantizer 234 and inverse transformer 235, the residual signal (residual block or residual sample) can be reconstructed. Adder 155 adds the reconstructed residual signal to the prediction signal output from inter-frame predictor 221 or intra-frame predictor 222, thereby generating a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). When there is no residual for the processing target block, as in the case of applying a jump mode, the prediction block can be used as the reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the target image, and, as described later, for inter-frame prediction of the next image by filtering.

[0083] In addition, luminance mapping with chroma scaling (LMCS) can be applied in image encoding and / or reconstruction processing.

[0084] Filter 260 can improve subjective / objective video quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be stored in memory 270, specifically in the DPB of memory 270. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive ring filter, bilateral filter, etc. As discussed later in the description of each filtering method, filter 260 can generate various filtering-related information and send the generated information to entropy encoder 240. The filtering information can be encoded in entropy encoder 240 and output as a bitstream.

[0085] The modified reconstructed image sent to memory 270 can be used as a reference image in inter-frame predictor 221. Accordingly, the encoding device can avoid prediction mismatch between the encoding device 100 and the decoding device when applying inter-frame prediction, and can also improve encoding efficiency.

[0086] Memory 270DPB can store modified reconstructed images for use as reference images in inter-frame predictor 221. Memory 270 can store motion information of blocks in the current image from which motion information has been derived (or encoded) and / or motion information of blocks in reconstructed images. The stored motion information can be sent to inter-frame predictor 221 to be used as motion information of neighboring blocks or temporally neighboring blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current image and send them to intra-frame predictor 222.

[0087] Figure 3This is a diagram that schematically illustrates the configuration of a video / image decoding device to which this disclosure can be applied.

[0088] Reference Figure 3 The video decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to embodiments, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 described above may be constituted by one or more hardware components (e.g., a decoder chipset or processor). Additionally, the memory 360 may include a decoded picture buffer (DPB) and may be constituted by a digital storage medium. The hardware components may also include the memory 360 as an internal / external component.

[0089] When the input includes a bitstream containing video / image information, the decoding device 300 can interact with data already prepared therein. Figure 2 The processing of video / image information in the encoding device correspondingly reconstructs the image. For example, the decoding device 300 can deduce units / blocks based on information related to block segmentation obtained from the bitstream. The decoding device 300 can perform decoding by using processing units applied in the encoding device. Therefore, the decoding processing unit can be, for example, an encoding unit, which can be segmented along a quadtree structure, binary tree structure, and / or ternary tree structure using encoding tree units or maximum encoding units. One or more transform units can be derived using encoding units. And, the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproducer.

[0090] Decoding device 300 can receive data from... in the form of a bitstream. Figure 2The signal output by the encoding device can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), Video Parameter Set (VPS), etc. In addition, the video / image information may also include general constraint information. The decoding device can further decode the picture based on the information about the parameter sets and / or general constraint information. In this disclosure, the signaling / receiving information and / or syntax elements, which will be described subsequently, can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 310 can decode the information in the bitstream based on encoding methods such as Exponential Golomb coding, CAVLC, CABAC, etc., and can output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine a context model using information about the target syntax element and the decoding information of neighboring and target blocks, or information about symbols / bins decoded in previous steps, predict the bin generation probability based on the determined context model, and perform arithmetic decoding on the bins to generate symbols corresponding to each syntax element value. Here, the CABAC entropy decoding method can update the context model after determining it using information about symbols / bins decoded for the context model of the next symbol / bin. Prediction information from the information decoded in the entropy decoder 310 can be provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values ​​(i.e., quantization transform coefficients) and associated parameter information that have undergone entropy decoding in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). Additionally, filtering information from the information decoded in the entropy decoder 310 can be provided to the filter 350. Furthermore, a receiver (not shown) that receives the signal output from the encoding device can also configure the decoding device 300 as an internal / external component, and the receiver can be a component of the entropy decoder 310. Additionally, the decoding device according to this disclosure can be referred to as a video / image / picture encoding device, and the decoding device can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0091] The dequantizer 321 can output transform coefficients by dequantizing the quantized transform coefficients. The dequantizer 321 can rearrange the quantized transform coefficients into two-dimensional blocks. In this case, the rearrangement can be performed based on the order of coefficient scans already performed in the encoding device. The dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0092] The inverse converter 322 obtains the residual signal (residual block, residual sample array) by performing an inverse transformation on the transformation coefficients.

[0093] The predictor can perform predictions on the current block and generate a prediction block that includes prediction samples for the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on information about the prediction output from the entropy decoder 310, and specifically, can determine the intra-frame / inter-frame prediction mode.

[0094] The predictor can generate a predicted signal based on various prediction methods. For example, the predictor can apply intra-frame prediction or inter-frame prediction to the prediction of a block, and can also apply intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as combined intra-frame and inter-frame prediction (CIIP). Additionally, the predictor can perform intra-block copying (IBC) for the prediction of a block. Intra-block copying can be used for content image / video encoding such as in games with screen content coding (SCC). Although IBC essentially performs prediction within the current block, its execution is similar to inter-frame prediction in that it derives a reference block within the current block. That is, IBC can use at least one of the inter-frame prediction techniques described in this disclosure.

[0095] The intra-predictor 331 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the reference samples can be located near or separate from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. The intra-predictor 331 can determine the prediction mode applied to the current block by using the prediction modes applied to neighboring blocks.

[0096] Inter-frame predictor 332 can deduce the predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference image. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted based on the correlation between motion information of neighboring blocks and the current block, on a block, sub-block, or sample basis. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference image index of the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the mode of inter-frame prediction for the current block.

[0097] Adder 340 can generate a reconstruction signal (reconstructed image, reconstruction block, reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from predictor 330. When there is no residual for processing the target block, as in the case of applying the jump mode, the prediction block can be used as the reconstruction block.

[0098] Adder 340 can be referred to as a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current block, and as described later, it can be output by filtering or used for inter-frame prediction of the next image.

[0099] In addition, luminance mapping with chroma scaling (LMCS) can be applied in image decoding processing.

[0100] Filter 350 can improve subjective / objective video quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be sent to memory 360, specifically to the DPB of memory 360. Various filtering methods can include, for example, deblocking filtering, adaptive sample shifting, adaptive ring filtering, bilateral filtering, etc.

[0101] The (modified) reconstructed image stored in the DPB of memory 360 can be used as a reference image in inter-frame predictor 332. Memory 360 can store motion information of blocks in the current image from which motion information has been derived (or decoded) and / or motion information of blocks in a reconstructed image. The stored motion information can be sent to inter-frame predictor 332 to be used as motion information of neighboring blocks or temporally neighboring blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current image and send them to intra-frame predictor 331.

[0102] The examples described in this specification in the predictor 330, dequantizer 321, inverse transformer 322 and filter 350 of the decoding device 300 can be similarly or correspondingly applied to the predictor 220, dequantizer 234, inverse transformer 235 and filter 260 of the encoding device 200, respectively.

[0103] As described above, prediction is performed to improve compression efficiency during video encoding. Accordingly, a prediction block can be generated that includes prediction samples for the current block, which is the target block for encoding. Here, the prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block can be derived identically in both the encoding and decoding devices, and the encoding device can improve image encoding efficiency by signaling to the decoding device information about the residual between the original block and the prediction block (residual information), not the original sample values ​​of the original block itself. The decoding device can derive a residual block including residual samples based on the residual information, generate a reconstructed block including reconstructed samples by adding the residual block to the prediction block, and generate a reconstructed image including the reconstructed block.

[0104] Residual information can be generated through transformation and quantization processes. For example, an encoding device can derive a residual block between the original block and the prediction block, derive transform coefficients by performing a transform process on the residual samples (residual sample array) included in the residual block, and derive quantized transform coefficients by performing a quantization process on the transform coefficients. This allows it to signal the associated residual information to the decoding device (via a bitstream). Here, the residual information can include the value information, position information, transform technique, transform kernel, quantization parameters, etc., of the quantized transform coefficients. The decoding device can perform quantization / dequantization processes based on the residual information and derive residual samples (or residual sample blocks). The decoding device can generate a reconstructed block based on the prediction block and the residual block. The encoding device can derive the residual block by performing dequantization / inverse transform on the quantized transform coefficients to serve as a reference for inter-frame prediction of the next image, and can generate a reconstructed image based on this.

[0105] Figure 4 The structure of a content streaming system applying this disclosure is illustrated.

[0106] Furthermore, the content streaming system using this disclosure can generally include an encoding server, a streaming server, a web server, a media storage device, a user device, and a multimedia input device.

[0107] An encoding server is used to compress content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and then sends it to a streaming server. As another example, in cases where the multimedia input device, such as a smartphone, camera, or camcorder, directly generates the bitstream, the encoding server can be omitted. The bitstream can be generated by applying the encoding method or bitstream generation method disclosed herein. Furthermore, the streaming server can temporarily store the bitstream during the sending or receiving process.

[0108] The streaming server sends multimedia data to the user's device via a web server based on the user's request. The web server acts as a tool to notify the user of available services. When a user requests a desired service, the web server transmits the request to the streaming server, and the streaming server sends the multimedia data to the user. In this context, the content streaming system may include a separate control server, which in this case controls the commands / responses between the corresponding devices within the content streaming system.

[0109] A streaming server can receive content from media storage devices and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a predetermined period of time.

[0110] For example, user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, board-type PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc. The servers in the content streaming system can operate as distributed servers, and in this case, data received by each server can be processed in a distributed manner.

[0111] Figure 5 Multiple transformation techniques according to embodiments of the present disclosure are illustrated schematically.

[0112] Reference Figure 5 The converter can correspond to the aforementioned Figure 2 The converter in the encoding device, and the inverse converter can correspond to the aforementioned Figure 2 Inverse converter in encoding devices, or Figure 3 The inverse converter in the decoding device.

[0113] The transformer can derive (first) transform coefficients (S510) by performing a first transform based on residual samples (residual sample array) in the residual block. This first transform can be referred to as the core transform. In this paper, the first transform can be based on multiple transform selection (MTS), and when multiple transforms are used as a first transform, it can be referred to as a multi-core transform.

[0114] Multi-core transform can represent a method of performing transforms by additionally using Discrete Cosine Transform (DCT) Type 2 and Discrete Sine Transform (DST) Type 7, DCT Type 8, and / or DST Type 1. In other words, multi-core transform can represent a method of transforming a spatial domain residual signal (or residual block) into frequency domain transform coefficients (or primary transform coefficients) based on multiple transform kernels selected from DCT Type 2, DST Type 7, DCT Type 8, and DST Type 1. In this paper, from the perspective of the transformer, primary transform coefficients can be referred to as temporary transform coefficients.

[0115] In other words, when applying conventional transform methods, transform coefficients can be generated by applying a spatial-to-frequency domain transform to the residual signal (or residual block) based on DCT type 2. In contrast, when applying multi-core transforms, transform coefficients (or single-stage transform coefficients) can be generated by applying a spatial-to-frequency domain transform to the residual signal (or residual block) based on DCT type 2, DST type 7, DCT type 8, and / or DST type 1. In this paper, DCT type 2, DST type 7, DCT type 8, and DST type 1 can be referred to as transform types, transform kernels, or transform cores. These DCT / DST transform types can be defined based on basis functions.

[0116] When performing a multi-core transform, a vertical transform kernel and a horizontal transform kernel can be selected from the transform kernels for the target block. A vertical transform can be performed on the target block based on the vertical transform kernel, and a horizontal transform can be performed on the target block based on the horizontal transform kernel. Here, the horizontal transform can indicate the transform of the horizontal components of the target block, and the vertical transform can indicate the transform of the vertical components of the target block. The vertical transform kernel / horizontal transform kernel can be adaptively determined based on the prediction mode and / or transform index of the target (CU or sub-block), including the residual block.

[0117] Furthermore, according to the example, if a transformation is performed by applying an MTS, the mapping relationship of the transformation kernels can be set by setting specific basis functions to predetermined values ​​and combining the basis functions to be applied in the vertical or horizontal transformation. For example, when the horizontal transformation kernel is denoted as trTypeHor and the vertical transformation kernel is denoted as trTypeVer, a value of 0 for trTypeHor or trTypeVer can be set to DCT2, a value of 1 for trTypeHor or trTypeVer can be set to DST7, and a value of 2 for trTypeHor or trTypeVer can be set to DCT8.

[0118] In this scenario, the MTS index information can be encoded and signaled to the decoding device to indicate any one of the multiple transform cores. For example, MTS index 0 can indicate that both trTypeHor and trTypeVer values ​​are 0, MTS index 1 can indicate that both trTypeHor and trTypeVer values ​​are 1, MTS index 2 can indicate that trTypeHor is 2 and trTypeVer is 1, MTS index 3 can indicate that trTypeHor is 1 and trTypeVer is 2, and MTS index 4 can indicate that both trTypeHor and trTypeVer values ​​are 2.

[0119] In one example, the transformation kernel set based on MTS index information is shown in the table below.

[0120] [Table 1]

[0121] tu_mts_idx[x0][y0] 0 1 2 3 4 trTypeHor 0 1 2 1 2 trTypeVer 0 1 1 2 2

[0122] The transformer can perform a quadratic transformation based on the (first) transform coefficients to derive modified (second) transform coefficients (S520). A first transform is a transformation from the spatial domain to the frequency domain, while a quadratic transform refers to transforming to a more compact representation using the correlations existing between the (first) transform coefficients. The quadratic transform can include an inseparable transform. In this case, the quadratic transform can be called an inseparable quadratic transform (NSST) or a mode-dependent inseparable quadratic transform (MDNSST). NSST can represent a transform based on an inseparable transform matrix, performing a quadratic transform on the (first) transform coefficients derived from the first transform to generate modified transform coefficients (or quadratic transform coefficients) for the residual signal. Here, based on the inseparable transform matrix, the transform can be applied first to the (first) transform coefficients without separating the vertical and horizontal transforms (or applying the horizontal / vertical transforms independently). In other words, NSST is not applied solely to (first-order) transform coefficients in the vertical and horizontal directions, but can also represent a transform method that, for example, rearranges a two-dimensional signal (transform coefficients) into a one-dimensional signal through a specific predetermined direction (e.g., row-first or column-first direction) and then generates modified transform coefficients (or second-order transform coefficients) based on an inseparable transform matrix. For example, row-first order is for M×N blocks arranged in the order of first row, second row, ..., and Nth row, while column-first order is for M×N blocks arranged in the order of first column, second column, ..., and Mth column. NSST can be applied to the upper left region of a block containing (first-order) transform coefficients (hereinafter referred to as a transform coefficient block). For example, when both the width W and height H of the transform coefficient block are 8 or greater, an 8×8 NSST can be applied to the upper left 8×8 region of the transform coefficient block. Furthermore, when both the width (W) and height (H) of the transform coefficient block are 4 or greater, and the width (W) or height (H) of the transform coefficient block is less than 8, a 4×4 NSST can be applied to the upper left min(8,W)×min(8,H) region of the transform coefficient block. However, the implementation is not limited to this. For example, even if only the condition that the width W or height H of the transform coefficient block is 4 or greater is met, a 4×4 NSST can be applied to the upper left min(8,W)×min(8,H) region of the transform coefficient block.

[0123] Specifically, for example, if a 4×4 input block is used, the inseparable quadratic transformation can be performed as follows.

[0124] A 4×4 input block X can be represented as follows.

[0125] [Formula 1]

[0126]

[0127] If X is represented as a vector, then the vector It can be represented as follows.

[0128] [Equation 2]

[0129]

[0130] In Equation 2, the vector It is a one-dimensional vector obtained by rearranging the two-dimensional block X of Equation 1 according to the row priority order.

[0131] In this case, the inseparable quadratic transformation can be calculated as follows.

[0132] [Formula 3]

[0133]

[0134] In this formula, represents the transformation coefficient vector, while T represents the 16×16 (inseparable) transformation matrix.

[0135] Using Equation 3 above, the 16×1 transformation coefficient vector can be derived. Furthermore, the vector can be scanned in sequence (horizontal, vertical, and diagonal, etc.). Reorganize into 4×4 blocks. However, the above calculation is an example, and the hypercube-Givens transform (HyGT) and similar methods can also be used to calculate inseparable quadratic transformations in order to reduce the computational complexity of inseparable quadratic transformations.

[0136] Furthermore, in inseparable quadratic transforms, the transform kernel (or transform type) can be selected as mode-dependent. In this case, the mode can include intra-frame prediction mode and / or inter-frame prediction mode.

[0137] As described above, an inseparable quadratic transformation can be performed based on an 8×8 transformation or a 4×4 transformation determined by the width (W) and height (H) of the transform coefficient block. An 8×8 transformation is a transformation applicable to an 8×8 region contained within the transform coefficient block when both W and H are equal to or greater than 8, and this 8×8 region can be the top-left 8×8 region within the transform coefficient block. Similarly, a 4×4 transformation is a transformation applicable to a 4×4 region contained within the transform coefficient block when both W and H are equal to or greater than 4, and this 4×4 region can be the top-left 4×4 region within the transform coefficient block. For example, the 8×8 transform kernel matrix can be a 64×64 / 16×64 matrix, while the 4×4 transform kernel matrix can be a 16×16 / 8×16 matrix.

[0138] Here, to select mode-dependent transform kernels, two inseparable quadratic transform kernels can be configured for each transform set of inseparable quadratic transforms for both 8×8 and 4×4 transforms, and there can be four transform sets. That is, four transform sets can be configured for 8×8 transforms, and four transform sets can be configured for 4×4 transforms. In this case, each transform set in the four transform sets for 8×8 transforms can include two 8×8 transform kernels, and each transform set in the four transform sets for 4×4 transforms can include two 4×4 transform kernels.

[0139] However, as the size of the transformation (i.e., the size of the region to which the transformation is applied) can be, for example, a size other than 8×8 or 4×4, the number of sets can be n, and the number of transformation kernels in each set can be k.

[0140] The transform set can be referred to as the NSST set or the LFNST set. A specific set within the transform set can be selected, for example, based on the intra-prediction mode of the current block (CU or sub-block). The Low-Frequency Inseparable Transform (LFNST) can be an example of a reduced inseparable transform, which will be described later, and represents an inseparable transform for low-frequency components.

[0141] For reference, for example, intra-prediction modes may include two non-directional (or non-angular) intra-prediction modes and 65 directional (or angular) intra-prediction modes. Non-directional intra-prediction modes may include planar intra-prediction mode number 0 and DC intra-prediction mode number 1, and directional intra-prediction modes may include 65 intra-prediction modes numbered 2 through 66. However, this is an example, and this document can be applied even if the number of intra-prediction modes differs. Furthermore, in some cases, intra-prediction mode number 67 may be used, and intra-prediction mode number 67 may represent a linear model (LM) mode.

[0142] Figure 6 The intra-frame orientation patterns for 65 predicted directions are schematically shown.

[0143] Reference Figure 6 Based on the intra-prediction mode 34 with a left-top diagonal prediction direction, intra-prediction modes can be divided into intra-prediction modes with horizontal directionality and intra-prediction modes with vertical directionality. Figure 6In the diagram, H and V denote horizontal and vertical orientation, respectively, and the numbers -32 to 32 indicate a displacement of 1 / 32 unit at the sample grid position. These numbers can represent the offset for the mode index value. Intra-prediction modes 2 to 33 are horizontally oriented, and intra-prediction modes 34 to 66 are vertically oriented. Strictly speaking, intra-prediction mode 34 can be considered neither horizontal nor vertical, but it can be classified as horizontally oriented when determining the transform set of the quadratic transform. This is because the input data is transposed for a vertical orientation mode symmetric to intra-prediction mode 34, and the input data alignment method for the horizontal mode is used for intra-prediction mode 34. Transposing the input data means switching the rows and columns of the two-dimensional M×N block data to N×M data. Intra-prediction modes 18 and 50 can represent the horizontal and vertical intra-prediction modes, respectively, and intra-prediction mode 2 can be called the upper-right diagonal intra-prediction mode because it has a left reference pixel and performs prediction in the upper-right direction. Similarly, intra-prediction mode 34 can be referred to as the bottom-right diagonal intra-prediction mode, while intra-prediction mode 66 can be referred to as the bottom-left diagonal intra-prediction mode.

[0144] Based on the example, four transform sets can be mapped according to the intra-frame prediction mode, as shown in the table below.

[0145] [Table 2]

[0146] lfnstPredModeIntra lfnstTrSetIdx lfnstPredModeIntra<0 1 0<=lfnstPredModeIntra<=1 0 2<=lfnstPredModeIntra<=12 1 13<=lfnstPredModeIntra<=23 2 24<=lfnstPredModeIntra<=44 3 45<=lfnstPredModeIntra<=55 2 56<=lfnstPredModeIntra<=80 1 81<=lfnstPredModeIntra<=83 0

[0147] As shown in Table 2, any one of the four transform sets, i.e., lfnstTrSetIdx, can be mapped to any one of the four indices (i.e., 0 to 3) according to the intra-prediction mode.

[0148] When a specific set is determined to be used for an inseparable quadratic transform, one of the k transform kernels in that set can be selected using the inseparable quadratic transform index. The encoding device can derive the inseparable quadratic transform index indicating the specific transform kernel based on rate-distortion (RD) check and can signal the inseparable quadratic transform index to the decoding device. The decoding device can select one of the k transform kernels in the specific set based on the inseparable quadratic transform index. For example, lfnst index value 0 can refer to the first inseparable quadratic transform kernel, lfnst index value 1 can refer to the second inseparable quadratic transform kernel, and lfnst index value 2 can refer to the third inseparable quadratic transform kernel. Alternatively, lfnst index value 0 can indicate that the first inseparable quadratic transform is not applied to the target block, and lfnst index values ​​1 to 3 can indicate three transform kernels.

[0149] The converter can perform an inseparable quadratic transform based on the selected transform core and obtain modified (quadratic) transform coefficients. As mentioned above, the modified transform coefficients can be derived as transform coefficients quantized by a quantizer and can be encoded and signaled to the decoding device, and transmitted to the dequantizer / inverse converter in the encoding device.

[0150] Furthermore, as mentioned above, if the second transformation is omitted, the (first) transformation coefficients, which are the output of the first (separable) transformation, can be derived as the transformation coefficients quantized by the quantizer as described above, and can be encoded and signaled to the decoding device, and transmitted to the dequantizer / inverse transformer in the encoding device.

[0151] The inverse transformer can perform a series of processes in the reverse order of those already performed in the aforementioned transformers. The inverse transformer can receive (dequantized) transform coefficients and derive (first) transform coefficients by performing a second (inverse) transform (S550), and obtain residual blocks (residual samples) by performing a first (inverse) transform on the (first) transform coefficients (S560). In this respect, from the perspective of the inverse transformer, the first transform coefficients can be referred to as modified transform coefficients. As described above, the encoding and decoding devices can generate reconstructed blocks based on the residual blocks and the prediction blocks, and can generate reconstructed images based on the reconstructed blocks.

[0152] The decoding device may also include a second-order inverse transform application determiner (or a component for determining whether to apply the second-order inverse transform) and a second-order inverse transform determiner (or a component for determining the second-order inverse transform). The second-order inverse transform application determiner can determine whether to apply the second-order inverse transform. For example, the second-order inverse transform can be NSST, RST, or LFNST, and the second-order inverse transform application determiner can determine whether to apply the second-order inverse transform based on a second-order transform flag obtained by parsing the bitstream. In another example, the second-order inverse transform application determiner can determine whether to apply the second-order inverse transform based on the transform coefficients of the residual block.

[0153] A second-order inverse transform determiner can determine the second-order inverse transform. In this case, the second-order inverse transform determiner can determine the second-order inverse transform applied to the current block based on the LFNST (NSST or RST) transform set specified according to the intra-prediction mode. In an implementation, the second-order transform determination method can be determined depending on the first-order transform determination method. Various combinations of the first and second-order transforms can be determined based on the intra-prediction mode. Furthermore, in the example, the second-order inverse transform determiner can determine the region where the second-order inverse transform is applied based on the size of the current block.

[0154] Furthermore, as mentioned above, if the second (inverse) transform is omitted, the (dequantized) transform coefficients can be received, a first (separable) inverse transform can be performed, and a residual block (residual sample) can be obtained. As mentioned above, the encoding and decoding devices can generate a reconstructed block based on the residual block and the prediction block, and can generate a reconstructed image based on the reconstructed block.

[0155] Furthermore, in this disclosure, a reduced quadratic transformation (RST) in which the size of the transformation matrix (kernel) is reduced can be applied to the concept of NSST in order to reduce the computational and storage requirements of the inseparable quadratic transformation.

[0156] Furthermore, the transform kernel, transform matrix, and coefficients constituting the transform kernel matrix described in this disclosure, i.e., kernel coefficients or matrix coefficients, can be represented in 8 bits. This is feasible for implementation in decoding and encoding devices, and compared to existing 9-bit or 10-bit representations, it reduces the amount of storage required to store the transform kernel and can reasonably accommodate performance degradation. Additionally, representing the kernel matrix in 8 bits allows for the use of smaller multipliers and is more suitable for Single Instruction Multiple Data (SIMD) instructions for optimal software implementation.

[0157] In this specification, the term "RST" can refer to a transformation performed on the residual samples of a target block based on a transformation matrix whose size is reduced according to a reduction factor. When performing a reduction transformation, the computational cost required for the transformation can be reduced due to the smaller size of the transformation matrix. In other words, RST can be used to address computational complexity issues that arise when transforming large blocks or when transforming indivisible blocks.

[0158] RST can be referred to by various terms such as reduced transform, reduced quadratic transform, reduced transform, simplified transform, and simple transform, and the names that RST can be called are not limited to the examples listed. Alternatively, since RST is performed primarily in the low-frequency region of the transform block that includes non-zero coefficients, it can be called low-frequency inseparable transform (LFNST). The transform index can be called the LFNST index.

[0159] Furthermore, when performing a second inverse transform based on RST, the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include: an inverse reduced second transformer that derives modified transform coefficients based on the inverse RST of the transform coefficients; and an inverse first transformer that derives the residual samples of the target block based on the inverse first transform of the modified transform coefficients. An inverse first transform refers to the inverse transform of a first transform applied to the residuals. In this disclosure, deriving transform coefficients based on a transform can mean deriving the transform coefficients by applying a transform.

[0160] Figure 7This is a diagram illustrating an embodiment of the RST according to the present disclosure.

[0161] In this disclosure, "target block" may refer to the current block, residual block, or transform block to be encoded.

[0162] In the example RST, an N-dimensional vector can be mapped to an R-dimensional vector in another space, thus determining the reduced transformation matrix, where R is less than N. N can refer to the square of the length of the side of the block to which the transformation is applied, or the total number of transformation coefficients corresponding to the block to which the transformation is applied, and the reduction factor can refer to the R / N value. The reduction factor can be called a reduction factor, shrinkage factor, simplification factor, or other various terms. Furthermore, R can be called a reduction coefficient, but depending on the situation, the reduction factor can refer to R. Additionally, depending on the situation, the reduction factor can refer to the N / R value.

[0163] In this example, the reduction factor or reduction coefficient can be signaled via a bitstream, but the example is not limited to this. For instance, a predetermined value for the reduction factor or reduction coefficient can be stored in each of the encoding device 200 and the decoding device 300, and in this case, the reduction factor or reduction coefficient does not need to be signaled separately.

[0164] The size of the reduced transformation matrix, as shown in the example, can be less than N×N (the size of the regular transformation matrix) and can be R×N, as defined in Equation 4 below.

[0165] [Formula 4]

[0166]

[0167] Figure 7 The matrix T in the reduced transformation block shown in (a) can refer to the matrix T in Equation 4. R×N .like Figure 7 As shown in (a), when the reduced transformation matrix T R×N By multiplying by the residual sample of the target block, the transformation coefficients of the current block can be derived.

[0168] In the example, if the size of the block to which the transformation is applied is 8×8 and R = 16 (i.e., R / N = 16 / 64 = 1 / 4), then according to Figure 7 The RST of (a) can be represented as the matrix operation shown in Equation 5. In this case, the storage and multiplication computations can be reduced to approximately 1 / 4 by a reduction factor.

[0169] In this disclosure, matrix operations can be understood as operations on column vectors obtained by multiplying a column vector by a matrix placed to the left of the column vector.

[0170] [Formula 5]

[0171]

[0172] In Equation 5, r1 to r 64 The residual samples of the target block can be represented, and specifically, they can be the transformation coefficients generated by applying a single transformation. As a result of the calculation in Equation 5, the transformation coefficients c of the target block can be derived. i And derive c i The process can be shown in Equation 6.

[0173] [Formula 6]

[0174]

[0175] As a result of Equation 6, the transformation coefficients c1 to c of the target block can be derived. R In other words, when R = 16, the transformation coefficients c1 to c of the target block can be derived. 16 If a conventional transform is applied instead of an RST, and a 64×64 (N×N) transform matrix is ​​multiplied by a 64×1 (N×1) residual sample, only 16(R) transform coefficients are derived for the target block because of the application of the RST, even though 64(N) transform coefficients are derived for the target block. Since the total number of transform coefficients used for the target block is reduced from N to R, the amount of data sent from the encoding device 200 to the decoding device 300 is reduced, thus improving the transmission efficiency between the encoding device 200 and the decoding device 300.

[0176] When considering the size of the transformation matrix, the size of a regular transformation matrix is ​​64×64 (N×N), but the size of a reduced transformation matrix is ​​reduced to 16×64 (R×N). Therefore, compared to performing a regular transformation, the storage utilization rate of performing an RST can be reduced by the R / N ratio. Furthermore, compared to the number of multiplications (N×N) when using a regular transformation matrix, using a reduced transformation matrix can reduce the number of multiplications (R×N) by the R / N ratio.

[0177] In the example, the transformer 232 of the encoding device 200 can derive the transform coefficients of the target block by performing a first transform and an RST-based second transform on the residual samples of the target block. These transform coefficients can be passed to the inverse transformer of the decoding device 300, and the inverse transformer 322 of the decoding device 300 can derive the modified transform coefficients based on the inverse reduced second transform (RST) for the transform coefficients, and can derive the residual samples of the target block based on the inverse first transform for the modified transform coefficients.

[0178] Based on the example inverse RST matrix T N×RIts size is N×R, which is larger than the size of the conventional inverse transformation matrix N×N, and is the same as the reduced transformation matrix T shown in Equation 4. R×N It has a transpose relationship.

[0179] Figure 7 The matrix T in the reduced inverse transform block shown in (b) t It can refer to the inverse RST matrix T N×R T (The superscript T indicates transpose). For example... Figure 7 As shown in (b), when the inverse RST matrix T N×R T Multiplying by the transform coefficients of the target block allows for the derivation of the modified transform coefficients of the target block or the residual samples of the target block. The inverse RST matrix T R×N T It can be represented as (T) R×N ) T N×R .

[0180] More specifically, when the inverse RST is used as a second inverse transformation, when the inverse RST matrix T N×R T When multiplied by the transform coefficients of the target block, the modified transform coefficients of the target block can be derived. Furthermore, the inverse RST can be used as the inverse first-order transform, and in this case, when the inverse RST matrix T... N×R T When multiplied by the transformation coefficients of the target block, the residual sample of the target block can be derived.

[0181] In the example, if the size of the block to which the inverse transform is applied is 8×8 and R = 16 (i.e., R / N = 16 / 64 = 1 / 4), then according to Figure 7 The RST of (b) can be represented as the matrix operation shown in Equation 7.

[0182] [Formula 7]

[0183]

[0184] In Equation 7, c1 to c 16 This can represent the transformation coefficients of the target block. As a result of the calculation in Equation 7, the transformation coefficients representing the modifications to the target block or the r of the residual samples of the target block can be derived. j And derive r j The process can be shown in Equation 8.

[0185] [Formula 8]

[0186]

[0187] As a result of Equation 8, the transformation coefficients representing the modification of the target block or the residual samples of the target block, r1 to r2, can be derived. N From the perspective of the size of the inverse transformation matrix, the size of the regular inverse transformation matrix is ​​64×64 (N×N), but the size of the inverse reduced transformation matrix is ​​reduced to 64×16 (R×N). Therefore, compared with performing the regular inverse transformation, the storage utilization rate of performing the inverse RST can be reduced by the R / N ratio. In addition, when comparing the number of multiplications N×N when using the regular inverse transformation matrix, using the inverse reduced transformation matrix can reduce the number of multiplications (N×R) by the R / N ratio.

[0188] The transform set configuration shown in Table 2 can also be applied to 8×8 RST. That is, 8×8 RST can be applied based on the transform sets in Table 2. Since a transform set includes two or three transforms (kernels) depending on the intra-prediction mode, it can be configured to select one of up to four transforms, including those without applying a secondary transform. In the transforms without applying a secondary transform, the application of an identity matrix can be considered. Assuming indices 0, 1, 2, and 3 are assigned to the four transforms respectively (for example, index 0 can be assigned to the case where the identity matrix is ​​applied, i.e., without applying a secondary transform), the transform index or lfnst index, which is used as a syntax element, can be signaled for each transform coefficient block, thereby specifying the transform to be applied. That is, for the top-left 8×8 block, 8×8 NSST in the RST configuration can be specified via the transform index, or 8×8 lfnst can be specified when applying LFNST. 8×8lfnst and 8×8RST refer to transformations of 8×8 regions within a transform coefficient block when both W and H of the target block are equal to or greater than 8, and the 8×8 region can be the top-left 8×8 region within the transform coefficient block. Similarly, 4×4lfnst and 4×4RST refer to transformations of 4×4 regions within a transform coefficient block when both W and H of the target block are equal to or greater than 4, and the 4×4 region can be the top-left 4×4 region within the transform coefficient block.

[0189] According to embodiments of this disclosure, for the transformation during the encoding process, only 48 data points can be selected, and a maximum 16×48 transformation kernel matrix can be applied to them, instead of applying a 16×64 transformation kernel matrix to the 64 data points forming an 8×8 region. Here, "maximum" means that m has a maximum value of 16 in the m×48 transformation kernel matrix to generate m coefficients. That is, when performing RST by applying an m×48 transformation kernel matrix (m≤16) to an 8×8 region, 48 data points are input, and m coefficients are generated. When m is 16, 48 data points are input, and 16 coefficients are generated. That is, assuming 48 data points form a 48×1 vector, the 16×48 matrix and the 48×1 vector are multiplied sequentially, thereby generating a 16×1 vector. Here, the 48 data points forming the 8×8 region can be appropriately arranged to form a 48×1 vector. For example, a 48×1 vector can be constructed based on 48 data points constituting the region other than the lower right 4×4 region within the 8×8 region. Here, when matrix operations are performed by applying a maximum 16×48 transformation kernel matrix, 16 modified transformation coefficients are generated. These 16 modified transformation coefficients can be arranged in the upper left 4×4 region according to the scan order, and the upper right 4×4 region and the lower left 4×4 region can be filled with zeros.

[0190] For the inverse transform in the decoding process, the transpose of the aforementioned transform kernel matrix can be used. That is, when performing inverse RST or LFNST during the inverse transform performed by the decoding device, the input coefficient data for applying inverse RST is arranged in a one-dimensional vector according to a predetermined arrangement order, and the modified coefficient vector obtained by multiplying the one-dimensional vector with the corresponding inverse RST matrix to the left of the one-dimensional vector is arranged in a two-dimensional block according to a predetermined arrangement order.

[0191] In summary, during the transformation process, when RST or LFNST is applied to an 8×8 region, matrix operations are performed on the 48 transformation coefficients in the upper left, upper right, and lower left regions of the 8×8 region (excluding the lower right region) with a 16×48 transformation kernel matrix. For matrix operations, the 48 transformation coefficients are input as a one-dimensional array. When performing matrix operations, 16 modified transformation coefficients are derived, and these modified coefficients can be arranged in the upper left region of the 8×8 region.

[0192] Conversely, in the inverse transform process, when the inverse RST or LFNST is applied to an 8×8 region, the 16 transform coefficients corresponding to the upper left region of the 8×8 region can be input as a one-dimensional array according to the scan order, and matrix operations can be performed with a 48×16 transform kernel matrix. That is, the matrix operation can be expressed as (48×16 matrix) * (16×1 transform coefficient vector) = (48×1 modified transform coefficient vector). Here, an n×1 vector can be interpreted as having the same meaning as an n×1 matrix, and therefore can be represented as an n×1 column vector. Furthermore, * denotes matrix multiplication. When performing matrix operations, 48 ​​modified transform coefficients can be derived, and these 48 modified transform coefficients can be arranged in the upper left, upper right, and lower left regions of the 8×8 region, excluding the lower right region.

[0193] When the inverse quadratic transform is based on the Regression-Simplified Transform (RST), the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include an inverse reduced quadratic transformer for deriving modified transform coefficients based on the inverse RST of the transform coefficients, and an inverse first-order transformer for deriving residual samples of the target block based on the inverse first-order transform of the modified transform coefficients. The inverse first-order transform refers to the inverse transform applied to the first-order transform of the residuals. In this disclosure, deriving transform coefficients based on a transform may refer to deriving transform coefficients by applying a transform.

[0194] The non-separate transform (LFNST) described above will be described in detail below. LFNST may include a forward transform performed by the encoding device and an inverse transform performed by the decoding device.

[0195] The encoding device receives the result (or part of the result) derived after applying a first (core) transform as input and applies a forward second transform (second transform).

[0196] [Formula 9]

[0197] y = G T x

[0198] In Equation 9, x and y are the input and output of the quadratic transformation, respectively, and G is the matrix representing the quadratic transformation, with the transformation basis vectors consisting of column vectors. In the case of inverse LFNST, when the dimension of the transformation matrix G is expressed as [number of rows × number of columns], in the case of forward LFNST, the transpose of matrix G becomes G... T Dimensions.

[0199] For the inverse LFNST, the dimensions of matrix G are [48×16], [48×8], [16×16], [16×8], and the [48×8] matrix and the [16×8] matrix are partial matrices of the eight transformed basis vectors sampled from the left side of the [48×16] matrix and the [16×16] matrix, respectively.

[0200] On the other hand, for a positive LFNST, matrix G T The dimensions are [16×48], [8×48], [16×16], and [8×16], and the [8×48] matrix and the [8×16] matrix are partial matrices obtained by sampling 8 transformation basis vectors from the upper part of the [16×48] matrix and the [16×16] matrix, respectively.

[0201] Therefore, in the case of forward LFNST, a [48×1] vector or a [16×1] vector can be used as input x, and a [16×1] vector or an [8×1] vector can be used as output y. In video encoding and decoding, the output of the forward first transform is two-dimensional (2D) data. Therefore, in order to construct a [48×1] vector or a [16×1] vector as input x, it is necessary to construct a one-dimensional vector by properly arranging the 2D data as the output of the forward transform.

[0202] Figure 8 This is a diagram illustrating the order in which the output data of a forward first transformation is arranged into a one-dimensional vector, based on the example. Figure 8 The left figures of (a) and (b) show the order used to construct the [48×1] vector, and Figure 8 The right figures (a) and (b) illustrate the order used to construct the [16×1] vector. In the case of LFNST, this can be achieved by combining 2D data with... Figure 8 Arrange the same order in (a) and (b) to obtain a one-dimensional vector x.

[0203] The orientation of the output data for the forward first transform can be determined based on the intra-prediction mode of the current block. For example, when the intra-prediction mode of the current block is horizontal relative to the diagonal direction, the orientation can be determined by... Figure 8 The output data of the forward first transform are arranged in the order of (a), and when the intra-prediction mode of the current block is perpendicular to the diagonal direction, it can be arranged according to... Figure 8 The output data of the first forward transformation are arranged in the order of (b).

[0204] Based on the example, different methods can be applied. Figure 8 The arrangement order of (a) and (b), and for derivation and application Figure 8The arrangement order of (a) and (b) results in the same outcome (y vector), and the column vectors of matrix G can be rearranged according to the arrangement order. That is, the column vectors of G can be rearranged such that each element constituting the x vector is always multiplied by the same transformation basis vector.

[0205] Since the output y derived by Equation 9 is a one-dimensional vector, when two-dimensional data is required as input data in the process of using the result of the forward quadratic transform as input (e.g., in the process of performing quantization or residual coding), the output y vector of Equation 9 needs to be properly arranged as 2D data again.

[0206] Figure 9 This is a diagram illustrating the order in which the output data of the forward quadratic transform is arranged into two-dimensional blocks, based on the example.

[0207] In the case of LFNST, the output values ​​can be arranged in 2D blocks according to a predetermined scan order. Figure 9 (a) shows how the output values ​​are arranged at 16 positions in a 2D block according to the diagonal scan order when the output y is a [16×1] vector. Figure 9 (b) shows that when the output y is an [8×1] vector, the output values ​​are arranged in 8 positions of the 2D block according to the diagonal scan order, and the remaining 8 positions are filled with zeros. Figure 9 In (b), X indicates that it is filled with zeros.

[0208] According to another example, since the order in which the output vector y is processed during quantization or residual coding can be preset, the output vector y does not need to be arranged as shown in the example. Figure 9 In the 2D block shown. However, in the case of residual coding, data encoding can be performed in 2D block (e.g., 4×4) cells (e.g., CG (coefficient group)), and in this case, according to as Figure 9 The data is arranged in a specific order within the diagonal scanning sequence.

[0209] Furthermore, the decoding device can configure the one-dimensional input vector y by arranging the two-dimensional data output from the dequantization process according to a preset scan order used for the inverse transform. The input vector y can be output as the output vector x using the following formula.

[0210] [Formula 10]

[0211] XGy

[0212] In the case of inverse LFNST, the output vector x can be derived by multiplying the input vector y, which is a [16×1] vector or an [8×1] vector, by the G matrix. For inverse LFNST, the output vector x can be a [48×1] vector or a [16×1] vector.

[0213] The output vector x is based on Figure 8 The sequence shown is arranged in a two-dimensional block and is arranged as two-dimensional data, which becomes the input data (or part of the input data) for the inverse first transformation.

[0214] Therefore, the inverse quadratic transform is the opposite of the forward quadratic transform process in general, and in the case of the inverse transform, unlike in the forward direction, the inverse quadratic transform is applied first, followed by the inverse first transform.

[0215] In the inverse LFNST, one of eight [48×16] matrices and eight [16×16] matrices can be chosen as the transformation matrix G. Whether to apply the [48×16] matrix or the [16×16] matrix depends on the size and shape of the block.

[0216] Additionally, eight matrices can be derived from the four transform sets shown in Table 2 above, and each transform set can consist of two matrices. The choice of which of the four transform sets to use is determined based on the intra-prediction mode, and more specifically, based on the values ​​of the intra-prediction mode extended by taking into account wide-angle intra-prediction (WAIP). The selection of which matrix to use from the two matrices constituting the chosen transform set is derived via index signaling. More specifically, 0, 1, and 2 can be used as index values ​​for transmission; 0 can indicate that LFNST is not applied, and 1 and 2 can indicate either of the two transform matrices constituting the transform set selected based on the intra-prediction mode values.

[0217] Figure 10 This is a diagram illustrating a wide-angle intra-frame prediction mode according to an implementation method described in this document.

[0218] Typical intra-prediction mode values ​​can have values ​​from 0 to 66 and from 81 to 83, and intra-prediction mode values ​​extended due to WAIP can have values ​​from -14 to 83 as shown. Values ​​from 81 to 83 indicate CCLM (Cross-Component Linear Model) mode, and values ​​from -14 to -1 and from 67 to 80 indicate intra-prediction mode extended due to WAIP application.

[0219] When the width of the current prediction block is greater than its height, the top reference pixel is typically closer to the interior of the block to be predicted. Therefore, prediction in the lower left direction is more accurate than prediction in the upper right direction. Conversely, when the height of the block is greater than its width, the left reference pixel is typically closer to the interior of the block to be predicted. Therefore, prediction in the upper right direction is more accurate than prediction in the lower left direction. Thus, applying remapping (i.e., mode index modification) to the index of the wide-angle intra-frame prediction mode can be advantageous.

[0220] When wide-angle intra-prediction is applied, information about existing intra-prediction patterns can be signaled, and after the information is parsed, it can be remapped to the index of the wide-angle intra-prediction pattern. Therefore, the total number of intra-prediction patterns used for a specific block (e.g., a non-square block of a specific size) can remain unchanged; that is, the total number of intra-prediction patterns is 67, and the encoding of the intra-prediction patterns used for a specific block can remain unchanged.

[0221] Table 3 below illustrates the process of deriving the modified intra-frame mode by remapping the intra-frame prediction mode to the wide-angle intra-frame prediction mode.

[0222] [Table 3]

[0223]

[0224] In Table 3, the extended intra-prediction mode values ​​are ultimately stored in the `predModeIntra` variable, and `ISP_NO_SPLIT` indicates that the CU block is not divided into sub-partitions using the intra-segmentation (ISP) technique currently used in the VVC standard. The `cIdx` variable values ​​of 0, 1, and 2 indicate the cases for the luma, Cb, and Cr components, respectively. The `log2` function shown in Table 3 returns a log value with a base of 2, and the `Abs` function returns the absolute value.

[0225] The variable `predModeIntra`, which indicates the intra-prediction mode, along with the height and width of the transform block, are used as input values ​​for the wide-angle intra-prediction mode mapping process, and the output value is the modified intra-prediction mode `predModeIntra`. The height and width of the transform block or coded block can be the height and width of the current block used for intra-prediction mode remapping. In this case, the variable `whRatio`, which reflects the width-to-width ratio, can be set to `Abs(Log2(nW / nH))`.

[0226] For non-square blocks, the intra-prediction mode can be divided into two cases and modified accordingly.

[0227] First, if all conditions (1) to (3) are met, (1) the width of the current block is greater than its height, (2) the intra-prediction mode before modification is equal to or greater than 2, and (3) the intra-prediction mode is less than the value derived as (8+2*whRatio) when the variable whRatio is greater than 1 and less than 8 when the variable whRatio is less than or equal to 1 (predModeIntra is less than (whRatio>1)?(8+2*whRatio):8), then the intra-prediction mode is set to a value 65 greater than predModeIntra [predModeIntra is set to be equal to (predModeIntra+65)].

[0228] If the above is different, that is, if conditions (1) to (3) are satisfied, (1) the height of the current block is greater than the width, (2) the intra-prediction mode before modification is less than or equal to 66, and (3) the intra-prediction mode is greater than the value derived as (60-2*whRatio) when whRatio is greater than 1 and greater than 60 when whRatio is less than or equal to 1 (predModeIntra is greater than (whRatio>1)?(60-2*whRatio):60), then the intra-prediction mode is set to a value 67 smaller than predModeIntra [predModeIntra is set to be equal to (predModeIntra-67)].

[0229] Table 2 above illustrates how to select the transform set in LFNST based on intra-prediction mode values ​​extended by WAIP. For example... Figure 10 As shown, modes 14 to 33 and modes 35 to 80 are symmetrical about the prediction directions around mode 34. For example, modes 14 and 54 are symmetrical about the direction corresponding to mode 34. Therefore, the same set of transformations is applied to modes located in mutually symmetrical directions, and this symmetry is also reflected in Table 2.

[0230] Furthermore, it is assumed that the positive LFNST input data of mode 54 is symmetrical to the positive LFNST input data of mode 14. For example, for modes 14 and 54, according to Figure 8 (a) and Figure 8 The arrangement shown in (b) rearranges the two-dimensional data into one-dimensional data. Furthermore, it can be seen that... Figure 8 (a) and Figure 8 The pattern in the sequence shown in (b) is symmetrical about the direction indicated by pattern 34 (diagonal direction).

[0231] Furthermore, as mentioned above, the size and shape of the target block determine which transformation matrix, either the [48×16] matrix or the [16×16] matrix, will be applied to the LFNST.

[0232] Figure 11 This is a diagram illustrating the block shape to which LFNST is applied. Figure 11 (a) shows a 4×4 block. Figure 11 (b) shows 4×8 blocks and 8×4 blocks. Figure 11 (c) shows a 4×N block or an N×4 block, where N is 16 or greater. Figure 11 (d) shows an 8×8 block. Figure 11 (e) shows an M×N block, where M≥8, N≥8 and N>8 or M>8.

[0233] exist Figure 11In the diagram, blocks with thick boundaries indicate the area where LFNST is applied. For Figure 11 For blocks (a) and (b), LFNST is applied to the top-left 4×4 region, and for Figure 11 Block (c) is individually applied to two consecutively arranged top-left 4×4 regions. Figure 11 In (a), (b), and (c), since the LFNST is applied in units of 4×4 regions, this LFNST will be referred to as "4×4 LFNST" in the following text. Based on the matrix dimension of G, a [16×16] or [16×8] matrix can be applied.

[0234] More specifically, a [16×8] matrix is ​​applied to Figure 11 (a) consists of 4×4 blocks (4×4TU or 4×4CU), and a [16×16] matrix is ​​applied to it. Figure 11 The blocks in (b) and (c) are used to adjust the worst-case computational complexity to 8 multiplications per sample.

[0235] about Figure 11 In (d) and (e), LFNST is applied to the top-left 8×8 region, and this LFNST is referred to as "8×8 LFNST" below. As the corresponding transformation matrix, a [48×16] matrix or a [48×8] matrix can be applied. In the case of the forward LFNST, since the [48×1] vector (the X vector in Equation 9) is input as input data, not all sample values ​​from the top-left 8×8 region are used as input values ​​for the forward LFNST. That is, if... Figure 8 The left-hand order of (a) or Figure 8 As can be seen from the left-hand order of (b), the [48×1] vector can be constructed based on the samples belonging to the other three 4×4 blocks while leaving the bottom right 4×4 block as is.

[0236] A [48×8] matrix can be applied to Figure 11 The 8×8 blocks (8×8TU or 8×8CU) in (d) and the [48×16] matrix can be applied Figure 11 The 8×8 blocks in (e). This is also to adjust the worst-case computational complexity to 8 multiplications per sample.

[0237] Depending on the block shape, when the corresponding forward LFNST (4×4 or 8×8 LFNST) is applied, 8 or 16 output data (the Y vector in Equation 9, [8×1] or [16×1] vectors) are generated. In the forward LFNST, due to matrix G... T Due to its characteristic, the amount of output data is equal to or less than the amount of input data.

[0238] Figure 12 This is a diagram illustrating the arrangement of the output data of the forward LFNST according to an example, and showing the blocks in which the output data of the forward LFNST is arranged according to the block shape.

[0239] exist Figure 12 The shaded area in the upper left corner of the block shown corresponds to the region where the output data of the forward LFNST is located. The positions marked with 0 indicate samples filled with a value of 0, and the remaining areas represent regions that were not altered by the forward LFNST. In regions not altered by LFNST, the output data of the first forward transform remains unchanged.

[0240] As mentioned above, since the size of the applied transformation matrix varies depending on the shape of the block, the amount of output data also varies. Figure 12 The output data of a forward LFNST may not completely fill the top-left 4×4 block. Figure 12 In cases (a) and (d), the [16×8] matrix and the A[48×8] matrix are applied to the block indicated by the thick line or a portion of the area inside the block, respectively, and an [8×1] vector is generated as the output of the positive LFNST. That is, according to Figure 9 The scan order shown in (b) can fill only 8 output data, such as Figure 12 As shown in (a) and (d), zeros can be filled in the remaining 8 positions. Figure 11 In the case of (d) of the LFNST application block, such as Figure 12 As shown in (d), the two 4×4 blocks adjacent to the top-left 4×4 block, the top-right and bottom-left blocks, are also filled with the value 0.

[0241] As described above, essentially, by signaling the LFNST index, it is specified whether to apply LFNST and the transformation matrix to be applied. Figure 12 As shown, when LFNST is applied, since the number of output data of the positive LFNST can be equal to or less than the number of input data, the following area filled with zero values ​​appears.

[0242] 1) such as Figure 12 As shown in (a), the samples are from the eighth position and the subsequent positions in the scanning order of the top left 4×4 block, that is, from the ninth to the sixteenth position.

[0243] 2) such as Figure 12 As shown in (d) and (e), when applying a [48×16] matrix or a [48×8] matrix, the two 4×4 blocks adjacent to the top left 4×4 block or the second and third 4×4 blocks in the scan order.

[0244] Therefore, if non-zero data is found in regions 1) and 2), it is determined that LFNST has not been applied, allowing the signaling of the corresponding LFNST index to be omitted.

[0245] Based on the example, such as in the case of LFNST used in the VVC standard, since the signaling for the LFNST index is executed after residual coding, the encoding device can know from the residual coding whether non-zero data (valid coefficients) exists at all locations within the TU or CU block. Therefore, the encoding device can determine whether to execute signaling regarding the LFNST index based on the presence of non-zero data, and the decoding device can determine whether to parse the LFNST index. The signaling for the LFNST index is executed when non-zero data does not exist in the areas specified in 1) and 2) above.

[0246] Because truncated unary codes are used as the binarization method for the LFNST index, the LFNST index consists of up to two bins, and 0, 10, and 11 are assigned as binary codes for possible LFNST index values ​​0, 1, and 2, respectively. In the current case of LFNST used for VVC, context-based CABAC encoding is applied to the first bin (regular encoding), and bypass encoding is applied to the second bin. The total number of contexts in the first bin is 2. When (DCT-2, DCT-2) is applied as a single transform pair for the horizontal and vertical directions, and the luma and chroma components are encoded in a dual-tree type, one context is assigned, and the other context is applied for the rest. The encoding of the LFNST index is shown in the table below.

[0247] [Table 4]

[0248]

[0249] In addition, the following simplification method can be applied to the LFNST used.

[0250] (i) As shown in the example, the number of output data for a positive LFNST can be limited to a maximum of 16.

[0251] exist Figure 11 In case (c), 4×4 LFNST can be applied to two adjacent 4×4 regions to the upper left, and in this case, a maximum of 32 LFNST output data can be generated. When the number of output data for the forward LFNST is limited to a maximum of 16, in the case of 4×N / N×4 (N≥16) blocks (TU or CU), 4×4 LFNST is applied only to one 4×4 region to the upper left, and LFNST can be applied only to... Figure 11 All blocks are processed at once. This simplifies the implementation of image encoding.

[0252] Figure 13 The example shows that the amount of output data for a positive LFNST is limited to a maximum of 16. Figure 13 When LFNST is applied to the top left 4×4 region of a 4×N or N×4 block (where N is 16 or greater), the output data of the forward LFNST becomes 16.

[0253] (ii) As in the example, zeroing can be additionally applied to regions where LFNST has not been applied. In this document, zeroing can mean filling all positions belonging to a specific region with a value of 0. That is, zeroing can be applied to regions that have not changed due to LFNST and maintain the result of a positive first transformation. As mentioned above, since LFNST is divided into 4×4 LFNST and 8×8 LFNST, zeroing can be divided into two types as follows ((ii)-(A) and (ii)-(B)).

[0254] (ii)-(A) When 4×4LFNST is applied, the area where 4×4LFNST is not applied can be zeroed. Figure 14 This is a diagram illustrating the zeroing process in a block of 4×4LFNST, based on the example.

[0255] like Figure 14 As shown, regarding the block that applied 4×4LFNST, that is, for Figure 12 All blocks in (a), (b) and (c) where LFNST is not applied can be filled with zeros.

[0256] on the other hand, Figure 14 (d) shows that when the maximum number of output data for the positive LFNST is limited to 16 (e.g. Figure 13 When (as shown), zero out the remaining blocks that have not applied 4×4LFNST.

[0257] (ii)-(B) When 8×8LFNST is applied, areas where 8×8LFNST is not applied can be zeroed. Figure 15 This is a diagram illustrating the zeroing process in a block of 8×8 LFNST, based on the example.

[0258] like Figure 15 As shown, regarding the application of 8×8LFNST, that is, for Figure 12 In all blocks in (d) and (e), the entire region where LFNST is not applied can be filled with zeros.

[0259] (iii) Due to the zeroing presented in (ii) above, the zero-filled area may not be the same as when LFNST was applied. Therefore, it can be determined by comparison. Figure 12In the case of LFNST, a wider area is used to perform zeroing as proposed in (ii) to check for the presence of non-zero data.

[0260] For example, when (ii)-(B) is applied, in the examination Figure 12 After checking whether there is non-zero data in the zero-filled regions in (d) and (e), additional checks are performed. Figure 15 The presence of non-zero data in the region filled with 0s can be checked, and signaling for the LFNST index can be executed only if no non-zero data exists.

[0261] Of course, even with the zeroing proposed in application (ii), the existence of non-zero data can be checked in the same way as existing LFNST index signaling. That is, when checking... Figure 12 After confirming the presence of non-zero data within the zero-padded block, LFNST index signaling can be applied. In this case, the encoding device only performs zeroing and the decoding device does not assume zeroing; that is, it only checks whether non-zero data exists within the zero-padded block. Figure 12 In regions explicitly marked as 0, LFNST index resolution can be performed.

[0262] Alternatively, according to another example, the following can be performed: Figure 16 The reset is shown. Figure 16 This is a diagram illustrating the zeroing of a block in an 8×8 LFNST application, based on another example.

[0263] like Figure 14 and Figure 15 As shown, zeroing can be applied to all areas except the area where LFNST is applied, or it can be applied only to a local area, such as... Figure 16 As shown. Zeroing only applies to items other than... Figure 16 Zeroing the area outside the top-left 8x8 area does not apply to the bottom-right 4x4 block within the top-left x8 area.

[0264] Various implementations of the simplified methods for applying LFNST (combinations of (i), (ii)-(A), (ii)-(B), (iii)) can be derived. Of course, the combinations of the above simplified methods are not limited to the following implementations, and any combination can be applied to LFNST.

[0265] Implementation

[0266] - Limit the number of output data for the positive LFNST to a maximum of 16 → (i)

[0267] - When 4×4LFNST is applied, all areas where 4×4LFNST is not applied are zeroed →(ii)-(A)

[0268] - When 8×8 LFNST is applied, all areas where 8×8 LFNST is not applied are zeroed → (ii)-(B)

[0269] - After checking whether non-zero data also exists in existing areas filled with zero values ​​and areas filled with zero due to additional clearing ((ii)-(A), (ii)-(B)), signal the LFNST index → ​​(iii) only if no non-zero data exists.

[0270] In the implementation scenario, when LFNST is applied, the region containing non-zero output data is limited to the upper left 4×4 region. More specifically, in Figure 14 (a) and Figure 15 In case (a), the eighth position in the scan order is the last position where non-zero data can exist. Figure 14 (b) and (c) and Figure 15 In case (b), the sixteenth position in the scan order (i.e., the position at the bottom right edge of the top-left 4×4 block) is the last position in which data other than 0 can exist.

[0271] Therefore, after applying LFNST, and after checking whether non-zero data exists at a position that is not allowed in the residual encoding process (at a position beyond the last position), it can be determined whether to signal the LFNST index.

[0272] In the case of the zeroing method proposed in (ii), the computational cost required to perform the entire transformation process can be reduced because of the amount of data ultimately generated when both the first transformation and LFNST are applied. That is, when LFNST is applied, since zeroing is applied to regions where the output data of the forward first transformation exists but LFNST is not applied, it is not necessary to generate data for regions that are zeroed during the forward first transformation. Therefore, the computational cost required to generate the corresponding data can be reduced. The additional effects of the zeroing method proposed in (ii) are summarized below.

[0273] First, as mentioned above, reduce the amount of computation required to perform the entire transformation process.

[0274] In particular, when (ii)-(B) is applied, the worst-case computational cost is reduced, making the transformation process lighter. In other words, generally, a large amount of computation is required to perform a single transformation of a large size. By applying (ii)-(B), the amount of data derived as a result of performing a forward LFNST can be reduced to 16 or less. Furthermore, as the size of the entire block (TU or CU) increases, the effect of reducing the number of transformation operations further increases.

[0275] Secondly, it can reduce the amount of computation required for the entire transformation process, thereby reducing the power consumption required to perform the transformation.

[0276] Third, it reduces the delay involved in the transformation process.

[0277] Secondary transforms, such as LFNST, add computational complexity to existing primary transforms, thus increasing the overall latency involved in performing the transform. Specifically, in the case of intra-frame prediction, the increased latency due to secondary transforms during encoding leads to an increase in latency until reconstruction because reconstructed data from adjacent blocks is used during prediction. This can result in an increase in the overall latency of intra-frame predictive coding.

[0278] However, if the zeroing proposed in application (ii) is applied, the delay time for performing a single transformation can be greatly reduced when LFNST is applied, maintaining or reducing the delay time of the entire transformation, making it easier to implement the encoding device.

[0279] In traditional intra-frame prediction, the block to be encoded is treated as a single coding unit and encoding is performed without segmentation. However, Intra-Frame Sub-Partition (ISP) coding means performing intra-frame prediction coding by dividing the block to be encoded horizontally or vertically. In this case, reconstructed blocks can be generated by performing encoding / decoding on a block-by-block basis, and the reconstructed blocks can be used as reference blocks for the next block. According to implementations, in ISP coding, a coding block can be divided into two or four sub-blocks and encoded, and in ISP, within a sub-block, intra-frame prediction is performed with reference to the reconstructed pixel values ​​of the adjacent left or upper sub-block. Hereinafter, "encoding" can be used as a concept encompassing both encoding performed by an encoding device and decoding performed by a decoding device.

[0280] Table 5 shows the number of sub-blocks divided according to the block size when applying ISP, and the sub-partitions divided according to ISP can be called transform blocks (TU).

[0281] [Table 5]

[0282] Block size (CU) Number of divisions 4×4 Unavailable 4×8、8×4 2 All other cases 4

[0283] ISP divides blocks within a predicted luma frame into two or four sub-partitions in the vertical or horizontal direction, based on the block size. For example, the minimum block size that can be applied to ISP is 4×8 or 8×4. When the block size is larger than 4×8 or 8×4, the block is divided into 4 sub-partitions.

[0284] Figure 17 and Figure 18 An example of how a coded block is divided into sub-blocks is given, and more specifically, Figure 17Examples of coded block (width (W) × height (H)) divisions of 4×8 blocks or 8×4 blocks are shown, and Figure 18 Examples of partitioning are shown for cases where the coded block is not a 4×8 block, 8×4 block, or 4×4 block.

[0285] When applying ISP, sub-blocks are encoded sequentially from left to right or top to bottom (e.g., horizontally or vertically) according to the partitioning type. After reconstruction processing via inverse transform and intra-prediction for one sub-block, the encoding of the next sub-block can be performed. For the leftmost or topmost sub-block, reconstructed pixels of already encoded blocks are referenced, as in conventional intra-prediction methods. Furthermore, when each side of a subsequent internal sub-block is not adjacent to the previous sub-block, reconstructed pixels of already encoded adjacent blocks are referenced to derive reference pixels adjacent to the corresponding side, as in conventional intra-prediction methods.

[0286] In ISP coding mode, all sub-blocks can be encoded using the same intra-prediction mode, and signals can be sent indicating whether ISP coding is used and whether the sub-blocks are divided in the direction (horizontal or vertical). For example... Figure 17 and Figure 18 As shown, the number of sub-blocks can be adjusted to 2 or 4 depending on the shape of the block. When the size (width × height) of a sub-block is less than 16, it can be restricted so that it is not allowed to be divided into corresponding sub-blocks or the ISP encoding itself is not applied.

[0287] In ISP prediction mode, a coding unit is divided into two or four partition blocks (i.e., sub-blocks) and prediction is performed, and the same intra-prediction mode is applied to the two or four partition blocks.

[0288] As described above, in terms of partitioning direction, both the horizontal direction (when M×N coding units with horizontal and vertical lengths of M and N are partitioned horizontally, if an M×N coding unit is divided into two, then the M×N coding unit is divided into M×(N / 2) blocks, and if an M×N coding unit is divided into four blocks, then the M×N coding unit is divided into M×(N / 4) blocks) and the vertical direction (when M×N coding units are partitioned vertically, if an M×N coding unit is divided into two, then the M×N coding unit is divided into (M / 2)×N blocks, and if an M×N coding unit is divided into four, then the M×N coding unit is divided into (M / 4)×N blocks) are possible. When M×N coding units are partitioned horizontally, the partition blocks are encoded in a top-to-bottom order, and when M×N coding units are partitioned vertically, the partition blocks are encoded in a left-to-right order. In the case of horizontal (vertical) division, the reconstructed pixel values ​​of the upper (left) partition can be referenced to predict the current encoded partition.

[0289] Transforms can be applied to residual signals generated in blocks using the ISP prediction method. Multiple transform selection (MTS) techniques based on the DST-7 / DCT-8 combination and the existing DCT-2 can be applied to a forward-based single transform (core transform), and a forward low-frequency non-separable transform (LFNST) can be applied to the transform coefficients generated from the single transform to generate the final modified transform coefficients.

[0290] In other words, LFNST can be applied to partitions divided by applying the ISP prediction mode, and the same intra-prediction mode is applied to the partitioned partitions, as described above. Therefore, when selecting an LFNST set derived based on the intra-prediction mode, the derived LFNST set can be applied to all partitions. That is, because the same intra-prediction mode is applied to all partitions, the same LFNST set can be applied to all partitions.

[0291] According to the implementation, LFNST can be applied only to transform blocks with both horizontal and vertical lengths of 4 or greater. Therefore, when the horizontal or vertical length of a partition block divided according to the ISP prediction method is less than 4, LFNST is not applied and no LFNST index is signaled. Furthermore, when applying LFNST to each partition block, the corresponding partition block can be considered as a transform block. When the ISP prediction method is not applied, LFNST can be applied to the coded block.

[0292] The method of applying LFNST to each partition block will be described in detail.

[0293] According to the implementation method, after applying the forward LFNST to each partition block, only a maximum of 16 (8 or 16) coefficients are left in the upper left 4×4 region in the order of scanning the transform coefficients, and then zeroing can be applied, in which the remaining positions and regions are all filled with 0.

[0294] Alternatively, according to the implementation, when the length of one side of the partition block is 4, LFNST is applied only to the upper left 4×4 region, and when the length (i.e., width and height) of all sides of the partition block is 8 or greater, LFNST can be applied to the remaining 48 coefficients in the upper left 8×8 region, excluding the lower right 4×4 region.

[0295] Alternatively, according to the implementation method, in order to adjust the worst-case computational complexity to 8 multiplications per sample, when each partition is 4×4 or 8×8, only 8 transformation coefficients can be output after applying the forward LFNST. That is, when the partition is 4×4, an 8×16 matrix can be used as the transformation matrix, and when the partition is 8×8, an 8×48 matrix can be used as the transformation matrix.

[0296] In the current VVC standard, LFNST index signaling is executed on a unit-by-unit basis. Therefore, in ISP prediction mode, and when LFNST is applied to all partition blocks, the same LFNST index value can be applied to the corresponding partition block. That is, when an LFNST index value is sent once at the unit-level, the corresponding LFNST index can be applied to all partition blocks within that unit. As mentioned above, the LFNST index value can have values ​​of 0, 1, and 2, where 0 indicates no LFNST application, and 1 and 2 represent two transform matrices existing in a set of LFNSTs when LFNST is applied.

[0297] As mentioned above, the LFNST set is determined by the intra-prediction mode, and in the case of ISP prediction mode, since all partition blocks in the coding unit are predicted in the same intra-prediction mode, the partition blocks can refer to the same LFNST set.

[0298] As another example, LFNST index signaling is still performed on a unit-by-unit basis. However, in ISP predictive mode, it is uncertain whether LFNST is applied uniformly to all blocks. For each block, the application of the LFNST index value signaled at the unit-by-unit level and the application of LFNST can be determined by separate conditions. Here, separate conditions can be signaled via a bitstream in the form of flags for each block. When the flag value is 1, the LFNST index value signaled at the unit-by-unit level is applied, and when the flag value is 0, LFNST may not be applied.

[0299] In an encoding unit that uses ISP mode, an example of applying LFNST when the length of one side of the partition block is less than 4 is described below.

[0300] First, when the size of the partition block is N×2 (2×N), LFNST can be applied to the upper left M×2 (2×M) region (where M≤N). For example, when M=8, the upper left region becomes 8×2 (2×8), so the region with 16 residual signals can be the input of the forward LFNST, and an R×16 (R≤16) forward transformation matrix can be applied.

[0301] Here, the forward LFNST matrix can be a separate, additional matrix besides those included in the current VVC standard. Furthermore, for worst-case complexity control, an 8×16 matrix, where only the top 8 rows of the 16×16 matrix are sampled, can be used for the transformation. The complexity control method will be described in detail later.

[0302] Secondly, when the size of the partition block is N×1 (1×N), LFNST can be applied to the upper left M×1 (1×M) region (where M≤N). For example, when M=16, the upper left region becomes 16×1 (1×16), so the region with 16 residual signals can be the input of the forward LFNST, and an R×16 (R≤16) forward transformation matrix can be applied.

[0303] Here, the corresponding forward LFNST matrix can be a separate additional matrix besides those included in the current VVC standard. Furthermore, to control worst-case complexity, an 8×16 matrix, where only the top 8 rows of the 16×16 matrix are sampled, can be used for the transformation. The complexity control method will be described in detail later.

[0304] The first and second embodiments can be applied simultaneously, or either one of the two embodiments can be applied. Specifically, in the case of the second embodiment, because a transformation is considered in the LFNST, experiments have shown that the compression performance improvement obtainable in the existing LFNST is relatively small compared to the LFNST index signaling cost. However, in the case of the first embodiment, a compression performance improvement similar to that obtainable from the conventional LFNST is observed. That is, in the case of ISP, the contribution of applying 2×N and N×2 LFNSTs to the actual compression performance can be observed experimentally.

[0305] In the current VVC's LFNST, symmetry is applied between intra-prediction modes. The same set of LFNSTs is applied to two directional modes set around mode 34 (prediction in the 45-degree diagonal direction at the bottom right), for example, the same set of LFNSTs is applied to mode 18 (horizontal prediction mode) and mode 50 (vertical prediction mode). However, in modes 35 through 66, when a forward LFNST is applied, the input data is transposed before the LFNST is applied.

[0306] VVC supports Wide-Angle Intra-Prediction (WAIP) mode. Considering WAIP mode, the LFNST set is derived based on the modified intra-prediction mode. For modes extended from WAIP, the LFNST set is determined using symmetry, just as in general intra-prediction directional modes. For example, because mode-1 is symmetric to mode 67, the same LFNST set is applied, and because mode-14 is symmetric to mode 80, the same LFNST set is applied. Modes 67 through 80 apply the LFNST transform after transposing the input data before applying the forward LFNST.

[0307] When applying LFNST to the top-left M×2 (M×1) block, symmetry with respect to LFNST cannot be applied because the block to which LFNST is applied is not square. Therefore, instead of applying symmetry based on intra-prediction mode, as shown in Table 2 for LFNST, symmetry between M×2 (M×1) and 2×M (1×M) blocks can be applied.

[0308] Figure 19 This is a diagram illustrating the symmetry between M×2 (M×1) blocks and 2×M (1×M) blocks according to the implementation method.

[0309] like Figure 19 As shown, since pattern 2 in the M×2 (M×1) block can be considered symmetric to pattern 66 in the 2×M (1×M) block, the same LFNST set can be applied to both the 2×M (1×M) block and the M×2 (M×1) block.

[0310] In this case, in order to apply the LFNST set applied to the M×2 (M×1) block to the 2×M (1×M) block, the LFNST set is selected based on mode 2 instead of mode 66. That is, the LFNST can be applied after transposing the input data of the 2×M (1×M) block, before applying the forward LFNST.

[0311] Figure 20 This is a diagram illustrating an example of a transposed 2×M block according to an embodiment.

[0312] Figure 20 (a) is a diagram illustrating how LFNST can be applied by reading 2×M blocks of input data in column-major order. Figure 20 (b) is a diagram illustrating how LFNST can be applied by reading the input data of an M×2 (M×1) block in row-major order. The method for applying LFNST to the top-left M×2 (M×1) or 2×M (M×1) block is described below.

[0313] 1. First, as Figure 20 As shown in (a) and (b), the input data is arranged into an input vector that constitutes a positive LFNST. For example, refer to Figure 19 For an M×2 block predicted using mode 2, follow Figure 20 In the order of (b), for a 2×M block predicted in pattern 66, the input data is in the following order: Figure 20 The sequential arrangement of (a) can then be applied to LFNST set for mode 2.

[0314] 2. For an M×2 (M×1) block, considering WAIP, the LFNST set is determined based on the modified intra-prediction mode. As mentioned above, a preset mapping relationship is established between the intra-prediction mode and the LFNST set, which can be represented by the mapping table shown in Table 2.

[0315] For a 2×M (1×M) block, taking into account WAIP, a symmetric mode around the prediction mode (mode 34 in the case of the VVC standard) can be obtained from the modified intra-prediction mode, moving downwards along a 45-degree diagonal. The LFNST set is then determined based on the corresponding symmetric mode and the mapping table. The symmetric mode (y) around mode 34 can be derived using the following formula. The mapping table will be described in more detail below.

[0316] [Equation 11]

[0317] If 2 ≤ x ≤ 66, then y = 68 - x.

[0318] Otherwise (x≤-1 or x≥67), y=66-x

[0319] 3. When applying forward LFNST, the transform coefficients can be derived by multiplying the input data prepared in process 1 by the LFNST kernel. The LFNST kernel can be selected based on the LFNST set determined in process 2 and the predetermined LFNST index.

[0320] For example, when M=8 and a 16×16 matrix is ​​used as the LFNST kernel, 16 transform coefficients can be generated by multiplying the matrix by 16 input data. The generated transform coefficients can be arranged in the upper left 8×2 or 2×8 region according to the scan order used in the VVC standard.

[0321] Figure 21 The scanning sequence of 8×2 or 2×8 regions according to the implementation method is illustrated.

[0322] All regions except the top-left 8×2 or 2×8 region can be filled with zero values ​​(cleared), or existing transformation coefficients that have undergone a single transformation can be left as is. The predefined LFNST index can be one of the LFNST index values ​​(0, 1, 2) that are tried when calculating the RD cost while changing the LFNST index value during programming processing.

[0323] When the worst-case computational complexity is tuned to a certain level or lower (e.g., 8 multiplications / sample), for example, after generating only 8 transformation coefficients by multiplying by an 8×16 matrix that takes only the top 8 rows of a 16×16 matrix, the transformation coefficients can be... Figure 21 The scan order can be set, and zeroing can be applied to the remaining coefficient regions. Worst-case complexity control will be described later.

[0324] 4. When applying the inverse LFNST, a preset number (e.g., 16) of transform coefficients are set as the input vector, and the LFNST set obtained from process 2 and the LFNST kernel (e.g., a 16×16 matrix) derived from the selected LFNST index are selected. The output vector can then be derived by multiplying the LFNST kernel with the corresponding input vector.

[0325] In the case of M×2 (M×1) blocks, the output vector can be... Figure 20 The row priority setting in (b) is used, while in the case of 2×M (1×M) blocks, the output vector can be set to... Figure 20 The column precedence settings for (a).

[0326] Except for the region where the corresponding output vector is set in the upper left M×2 (M×1) or 2×M (M×2) region, the remaining regions in the partition block except for the upper left M×2 (M×1) or 2×M (M×2) region (the M×2 region in the partition block) can all be cleared to have zero values, or can be configured to preserve the reconstructed transform coefficients as is through residual coding and inverse quantization.

[0327] When constructing the input vector, as in point 3, the input data can be constructed according to... Figure 21 The scanning order can be arranged, and in order to keep the worst-case computational complexity to a certain extent or lower, the input vector can be constructed by reducing the number of input data (e.g., 8 instead of 16).

[0328] For example, when M=8, if 8 input data are used, the leftmost 16×8 matrix can be taken from the corresponding 16×16 matrix and multiplied to obtain 16 output data. Worst-case complexity control will be described later.

[0329] In the above implementation, when applying LFNST, the case of applying symmetry between M×2 (M×1) blocks and 2×M (1×M) blocks is shown. However, according to another example, different sets of LFNST can be applied to each of the two block shapes.

[0330] The following sections will describe various examples of mapping methods using intra-prediction mode and LFNST set configurations using ISP mode.

[0331] In ISP mode, the LFNST set configuration can differ from the existing LFNST set. In other words, a different core than the existing LFNST core can be applied, and a different mapping table can be applied than the mapping table used between the intra-prediction mode index and the LFNST set in the current VVC standard. The mapping table used in the current VVC standard can be the same as the mapping table in Table 2.

[0332] In Table 2, the preModeIntra value represents the intra-prediction mode value that has changed to take WAIP into account, and the lfnstTrSetIdx value is the index value indicating a specific LFNST set. Each LFNST set is configured with two LFNST cores.

[0333] When applying the ISP prediction mode, if both the horizontal and vertical lengths of each partition block are equal to or greater than 4, the same kernel as the LFNST kernel used in the current VVC standard can be applied, and the mapping table can be applied as is. Alternatively, mapping tables and LFNST kernels different from those in the current VVC standard can be applied.

[0334] When applying the ISP prediction mode, if the horizontal or vertical length of each block is less than 4, a mapping table and LFNST core different from the current VVC standard can be applied. In the following text, Tables 6 to 8 show the mapping table between intra-prediction mode values ​​(intra-prediction mode values ​​changed to take into account WAIP) and LFNST sets, which can be applied to M×2 (M×1) blocks or 2×M (1×M) blocks.

[0335] [Table 6]

[0336] predModeIntra lfnstTrSetIdx predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<=12 1 13<=predModeIntra<=23 2 24<=predModeIntra<=34 3 35<=predModeIntra<=44 4 45<=predModeIntra<=55 5 56<=predModeIntra<=66 6 67<=predModeIntra<=80 6 81<=predModeIntra<=83 0

[0337] [Table 7]

[0338] predModeIntra lfnstTrSetIdx predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<=23 1 24<=predModeIntra<=44 2 45<=predModeIntra<=66 3 67<=predModeIntra<=80 3 81<=predModeIntra<=83 0

[0339] [Table 8]

[0340] predModeIntra lfnstTrSetIdx predModeIntra<0 1 0<=predModeIntra<=1 0 2<=predModeIntra<=80 1 81<=predModeIntra<=83 0

[0341] The first mapping table in Table 6 is configured with seven LFNST sets, the mapping table in Table 7 is configured with four LFNST sets, and the mapping table in Table 8 is configured with two LFNST sets. As another example, when it is configured with one LFNST set, the lfnstTrSetIdx value can be fixed to 0 relative to the preModeIntra value.

[0342] The following section describes a method for maintaining the worst-case computational complexity when applying LFNST to the ISP pattern.

[0343] In ISP mode, when applying LFNST, the number of multiplications per sample (or per coefficient, per position) may be limited to a certain value or less. Depending on the size of the partition block, the number of multiplications per sample (or per coefficient, per position) can be kept to 8 or less by applying LFNST as follows.

[0344] 1. When both the horizontal and vertical lengths of the partition block are 4 or greater, the same computational complexity control method as the worst-case method for LFNST in the current VVC standard can be applied.

[0345] In other words, when the partition block is 4×4, an 8×16 matrix obtained by sampling the top 8 rows of a 16×16 matrix can be applied instead of a 16×16 matrix in the forward direction, and a 16×8 matrix obtained by sampling the left 8 columns of a 16×16 matrix can be applied in the reverse direction. Furthermore, when the partition block is 8×8, in the forward direction, instead of a 16×48 matrix, an 8×48 matrix obtained by sampling the top 8 rows of a 16×48 matrix is ​​applied, and in the reverse direction, instead of a 48×16 matrix, a 48×8 matrix obtained by sampling the left 8 columns of a 48×16 matrix can be applied.

[0346] In the case of 4×N or N×4 (N>4) blocks, when performing a forward transformation, the 16 coefficients generated after applying the 16×16 matrix only to the top-left 4×4 block can be set in the top-left 4×4 region, and other regions can be filled with values ​​of 0. Conversely, when performing an inverse transformation, the 16 coefficients in the top-left 4×4 block are set in scan order to form the input vector, and then multiplied by the 16×16 matrix to generate 16 output data. The generated output data can be set in the top-left 4×4 region, and the remaining regions outside the top-left 4×4 region can be filled with values ​​of 0.

[0347] In the case of 8×N or N×8 (N>8) blocks, when performing the forward transformation, the 16 coefficients generated after applying a 16×48 matrix to only the ROI region within the top-left 8×8 block (excluding the remaining regions from the bottom-right 4×4 block within the top-left 8×8 block) can be set in the top-left 4×4 region, and all other regions can be filled with values ​​of 0. Furthermore, when performing the inverse transformation, the 16 coefficients located in the top-left 4×4 region are set in scan order to form the input vector, which can then be multiplied by a 48×16 matrix to generate 48 output data points. The generated output data can be filled in the ROI regions, and all other regions can be filled with values ​​of 0.

[0348] 2. When the size of the partition block is N×2 or 2×N and LFNST is applied to the top left M×2 or 2×M region (M≤N), a matrix sampled according to the value of N can be applied.

[0349] With M=8, for partitions of N=8, i.e., 8×2 or 2×8 blocks, in the case of forward transformation, an 8×16 matrix obtained by sampling the top 8 rows of a 16×16 matrix can be applied instead of a 16×16 matrix, and in the case of inverse transformation, a 16×8 matrix obtained by sampling the left 8 columns of a 16×16 matrix can be applied instead of a 16×16 matrix.

[0350] When N is greater than 8, in the forward transformation, the 16×16 matrix applied to the top-left 8×2 or 2×8 block generates 16 output data points, which are then placed within that block, with the remaining areas filled with values ​​of 0. In the inverse transformation, the 16 coefficients in the top-left 8×2 or 2×8 block are arranged in scan order to form the input vector, which is then multiplied by the 16×16 matrix to generate 16 output data points. These output data points can also be placed within the top-left 8×2 or 2×8 block, with all remaining areas filled with values ​​of 0.

[0351] 3. When the size of the partition block is N×1 or 1×N and LFNST is applied to the top left M×1 or 1×M region (M≤N), a matrix sampled according to the value of N can be applied.

[0352] When M=16, for partitioned blocks of N=16, i.e., 16×1 or 1×16 blocks, in the case of forward transformation, an 8×16 matrix obtained by sampling the top 8 rows of the 16×16 matrix can be applied instead of a 16×16 matrix, and in the case of inverse transformation, a 16×8 matrix obtained by sampling the left 8 columns of the 16×16 matrix can be applied instead of a 16×16 matrix.

[0353] When N is greater than 16, in the forward transformation, the 16 output data generated by applying a 16×16 matrix to the top-left 16×1 or 1×16 block can be set within the top-left 16×1 or 1×16 block, and the remaining areas can be filled with values ​​of 0. In the inverse transformation, the 16 coefficients located in the top-left 16×1 or 1×16 block can be set in scan order to form the input vector, and then multiplied by the 16×16 matrix to generate 16 output data. The generated output data can be set within the top-left 16×1 or 1×16 block, and all remaining areas can be filled with values ​​of 0.

[0354] As another example, to keep the number of multiplications per sample (or per coefficient, per position) at a certain value or less, the number of multiplications per sample (or per coefficient, per position) can be kept to 8 or less based on the ISP coding unit size rather than the ISP block size. When only one block in the ISP block satisfies the conditions for applying LFNST, the worst-case complexity of LFNST can be calculated based on the corresponding coding unit size rather than the block size. For example, when the luma coding block of a coding unit is divided into four 4×4 blocks and encoded by ISP, and there are no non-zero transform coefficients for two of the four blocks, it can be configured such that 16 transform coefficients are generated for the other two blocks (based on the encoder) instead of eight.

[0355] The following section describes the method for signaling the LFNST index in ISP mode.

[0356] As described above, the LFNST index can have any one of the values ​​0, 1, and 2, where a value of 0 indicates that LFNST is not applied, while values ​​1 and 2 respectively indicate the two LFNST kernel matrices included in the selected LFNST set. LFNST is applied based on the LFNST kernel matrix selected by the LFNST index. The method for sending the LFNST index in the current VVC standard will be described below.

[0357] 1. The LFNST index can be sent once for each coding unit (CU), and in the case of a dual-tree type, separate LFNST indices can be signaled for both luma and chroma blocks.

[0358] 2. When no signal is sent to the LFNST index, the value of the LFNST index is set (inferred) to the default value of 0. The LFNST index is inferred to be 0 in the following cases:

[0359] A. Cases where no transformation is applied (e.g., transform skipping, BDPCM, lossless coding, etc.).

[0360] B. Cases where the transformation is not DCT-2 (DST7 or DCT8), that is, cases where the transformation in the horizontal direction or the transformation in the vertical direction is not DCT-2.

[0361] C. When the horizontal or vertical length of the luminance block of the coding unit exceeds the maximum luminance change size, for example, when the maximum luminance change size is 64, LFNST is not applied when the luminance block size of the coding block is 128×16.

[0362] In the case of a dual-tree type, for each of the coding units for the luma component and the chroma component, it is determined whether the maximum luma transform size is exceeded. That is, for a luma block, it is checked whether the maximum luma transform size is exceeded, and for a chroma block, it is checked whether the horizontal / vertical length and maximum luma transform size of the corresponding luma block for the color format are exceeded. For example, when the color format is 4:2:0, the horizontal / vertical length of the corresponding luma block is twice the horizontal / vertical length of the chroma block, and the transform size of the corresponding luma block is twice the transform size of the chroma block. In another example, when the color format is 4:4:4, the horizontal / vertical length and transform size of the corresponding luma block are the same as those of the chroma block.

[0363] A 64-length transformation or a 32-length transformation refers to a transformation applied horizontally or vertically with a length of 64 or 32 respectively, and the "transformation size" can refer to the corresponding length of 64 or 32.

[0364] In the case of a single tree type, it is possible to check whether the horizontal or vertical length of the luminance block exceeds the maximum transformable size of the luminance transform block, and when the horizontal or vertical length of the luminance block exceeds the maximum transformable size of the luminance transform block, the signaling of the LFNST index can be omitted.

[0365] D. The LFNST index can be sent only if both the horizontal and vertical lengths of the coding unit are greater than or equal to 4.

[0366] In the case of a dual-tree type, the LFNST index can be signaled only when the horizontal and vertical lengths of the corresponding component (i.e., the luminance component or the chrominance component) are both greater than or equal to 4.

[0367] In the case of a single tree type, the LFNST index can be signaled when both the horizontal and vertical lengths of the luminance component are greater than or equal to 4.

[0368] E. If the last non-zero coefficient position is not a DC position (which is located in the upper left of the block), then for a dual-tree type luma block, if the last non-zero coefficient position is not a DC position, then an LFNST index is sent. In the case of a dual-tree type chroma block, if either the last non-zero coefficient position of Cb or the last non-zero coefficient position of Cr is not a DC position, then the corresponding LNFST index is sent.

[0369] In the case of a single-tree type, the LFNST index is sent when the last non-zero coefficient position of any of the luminance component, Cb component, and Cr component is not a DC position.

[0370] Here, when the Code Block Flag (CBF), which indicates the presence of transform coefficients in a transform block, is 0, the position of the last non-zero coefficient in the corresponding transform block is not checked to determine whether to signal the LFNST index. In other words, when the corresponding CBF value is 0, the transform is not applied to the corresponding block, so the position of the last non-zero coefficient can be ignored when checking the conditions used for LFNST index signaling.

[0371] For example, 1) in the case of the luminance component in a two-tree type, if the corresponding CBF value is 0, no signal is sent to the LFNST index; 2) in the case of the chrominance component in a two-tree type, if the CBF value of Cb is 0 and the CBF value of Cr is 1, only the last non-zero coefficient position of Cr is checked and the corresponding LFNST index is sent; 3) in the case of the single-tree type, the last non-zero coefficient position of any component among the luminance component, Cb component and Cr component with a CBF value of 1 is checked.

[0372] F. When a transform coefficient is found at a location other than where the LFNST transform coefficient is allowed, the LFNST index signaling can be omitted. In the case of 4×4 and 8×8 transform blocks, according to the transform coefficient scan order in the VVC standard, the LFNST transform coefficient can exist in 8 positions starting from the DC position, with all remaining positions padded with 0. Furthermore, in cases other than 4×4 and 8×8 transform blocks, according to the transform coefficient scan order in the VVC standard, the LFNST transform coefficient can exist in 16 positions starting from the DC position, with all remaining positions padded with 0.

[0373] Therefore, when there are non-zero transform coefficients in the region that should be filled with 0 after residual coding, the LFNST index signaling can be omitted.

[0374] Furthermore, the ISP mode can be applied only to luma blocks, or it can be applied to both luma and chroma blocks. As mentioned above, when ISP prediction is applied, the corresponding coding unit is divided into two or four partitions and predicted, and a transform can be applied to each corresponding partition. Therefore, when determining the conditions for signaling the LFNST index on a coding unit basis, the fact that LFNST can be applied to each of the corresponding partitions needs to be considered. Additionally, when the ISP prediction mode is applied only to a specific component (e.g., luma block), the fact that only the corresponding component is divided into partitions should be taken into account when signaling the LFNST index. The LFNST index signaling methods available under ISP mode can be summarized as follows.

[0375] 1. The LFNST index can be sent once for each coding unit (CU), and in the case of a dual-tree type, separate LFNST indices can be sent for the luma block and the chroma block respectively.

[0376] 2. When the LFNST index is signaled, its value is (inferred) to be 0. The following are examples of cases where the LFNST index value is inferred to be 0.

[0377] A. Cases where no transformation is applied (e.g., transform skipping, BDPCM, lossless coding, etc.).

[0378] B. LFNST cannot be applied when the horizontal or vertical length of the luminance block of the coding unit exceeds the maximum luminance variable size, for example, when the maximum variable luminance size is 64, and the size of the luminance block of the coding unit is 128×16.

[0379] Whether to signal the LFNST index can be determined based on the size of the partition block rather than the size of the coding unit. That is, when the horizontal or vertical length of the partition block corresponding to the luminance block exceeds the maximum luminance changeable size, the LFNST index signaling can be omitted, and the value of the LFNST index can be inferred to be 0.

[0380] In the case of a dual-tree type, for each of the coding units or blocks for the luma component and the coding units or blocks for the chroma component, it is determined whether the maximum transform block size is exceeded. That is, the horizontal and vertical lengths of the coding units or blocks for luma are compared with the maximum luma transform size, and if either the horizontal or vertical length is greater than the maximum luma transform size, LFNST is not applied. In the case of coding units or blocks for chroma, the horizontal / vertical lengths of the corresponding luma block for the color format are compared with the maximum luma transform size. For example, when the color format is 4:2:0, the horizontal / vertical length of the corresponding luma block is twice the horizontal / vertical length of the chroma block, and the transform size of the corresponding luma block is twice the transform size of the chroma block. In another example, when the color format is 4:4:4, the horizontal / vertical lengths and transform sizes of the corresponding luma blocks are the same as those of the chroma blocks.

[0381] In the case of a single tree type, it is possible to check whether the horizontal or vertical length of the luminance block (encoding unit or partition block) exceeds the maximum transformable size of the luminance transform block, and if so, the LFNST index signaling can be omitted.

[0382] C. When applying the LFNST included in the current VVC standard, the LFNST index can be sent only when both the horizontal and vertical lengths of the partition block are greater than or equal to 4.

[0383] When an LFNST for 2×M (1×M) or M×2 (M×1) blocks is applied in addition to the LFNST included in the current VVC standard, the LFNST index can be sent only if the size of the partition block is equal to or greater than a 2×M (1×M) or M×2 (M×1) block. Here, if a P×Q block is equal to or greater than an R×S block, this means that P≥R and Q≥S.

[0384] In summary, the LFNST index can be sent only when the partition block is equal to or greater than the minimum size for which LFNST can be applied. In the case of a two-tree type, the LFNST index can be signaled only when the partition block of the luma or chroma component is equal to or greater than the minimum size for which LFNST can be applied. In the case of a single-tree type, the LFNST index can be signaled only when the partition block of the luma component is equal to or greater than the minimum size for which LFNST can be applied.

[0385] In this document, if an M×N block is equal to or greater than a K×L block, it means that M is equal to or greater than K and N is equal to or greater than L. If an M×N block is greater than a K×L block, it means that M is equal to or greater than K, N is equal to or greater than L, and M is greater than K or N is greater than L. If an M×N block is less than or equal to a K×L block, it means that M is less than or equal to K, N is less than or equal to L, and M is less than K or N is less than L.

[0386] D. If the last non-zero coefficient position is not a DC position (which is located in the upper left of the block), then for a dual-tree type luma block, the LFNST index can be sent if the last non-zero coefficient position in any of the partition blocks is not a DC position. In the case of a dual-tree type chroma block, the corresponding LNFST index can be sent if either the last non-zero coefficient position of all partition blocks for Cb (the number of partition blocks is considered 1 when the ISP mode is not applied to the chroma component) or the last non-zero coefficient position of all partition blocks for Cr (the number of partition blocks is considered 1 when the ISP mode is not applied to the chroma component).

[0387] In the case of a single-tree type, when the last non-zero coefficient position of any of the partitions of the luminance component, Cb component, and Cr component is not a DC position, the corresponding LFNST index can be sent.

[0388] Here, when the Code Block Flag (CBF) indicating whether transform coefficients exist for each partition block is 0, the position of the last non-zero coefficient of the corresponding partition block is not checked to determine whether to signal the LFNST index. That is, when the corresponding CBF value is 0, the transform is not applied to the corresponding block, so the position of the last non-zero coefficient of the corresponding partition block is not considered when checking the conditions for LFNST index signaling.

[0389] For example, 1) in the case of a two-tree type luma component, when the corresponding CBF value of each partition block is 0, the corresponding partition block is excluded when determining whether to signal the LFNST index; 2) in the case of a two-tree type chrominance component, when the CBF value of Cb of each partition block is 0 and the CBF value of Cr is 1, only the position of the last non-zero coefficient of Cr is checked when determining whether to signal the LFNST index; and 3) in the case of a single-tree type, whether to signal the LFNST index can be determined by checking only the position of the last non-zero coefficient of the block with a CBF value of 1 for all partition blocks of the luma component, Cb component and Cr component.

[0390] In ISP mode, image information can be configured to not check the position of the last non-zero coefficient, and the implementation is as follows.

[0391] i. In ISP mode, LFNST index signaling can be allowed without checking the last non-zero coefficient position of both the luma and chroma blocks. That is, LFNST index signaling can be allowed even when the last non-zero coefficient position of all partition blocks is at the DC position or the corresponding CBF value is 0.

[0392] ii. In ISP mode, the check for the last non-zero coefficient position can be omitted for the luma block, while the check for the last non-zero coefficient position of the chroma block can be performed as described above. For example, in the case of a two-tree luma block, LFNST index signaling is allowed without checking the last non-zero coefficient position, while in the case of a two-tree chroma block, the presence of a DC position with a last non-zero coefficient position can be checked as described above to determine whether to signal the corresponding LFNST index.

[0393] iii. In the case of ISP mode and single-tree type, either method i or method ii above can be applied. That is, when method i is applied to the single-tree type in ISP mode, the check for the last non-zero coefficient position can be omitted for both luma and chroma blocks, and LFNST index signaling is allowed. Alternatively, when method ii is applied, the check for the last non-zero coefficient position can be omitted for luma component blocks, and for chroma component blocks (which can be considered as having a quantity of 1 when ISP is not applied to the chroma component), the last non-zero coefficient position can be checked in the manner described above to determine whether to signal the corresponding LFNST index.

[0394] E. When it is found that even if one of the partition blocks has a transform coefficient at a location other than where the LFNST transform coefficient can exist, the LFNST index signaling can be omitted.

[0395] For example, in the case of 4×4 and 8×8 partition blocks, according to the transform coefficient scan order in the VVC standard, LFNST transform coefficients can exist in 8 positions starting from the DC position, and all remaining positions are filled with 0. Additionally, when the partition block is equal to or greater than 4×4 and is neither a 4×4 nor an 8×8 partition block, according to the transform coefficient scan order in the VVC standard, LFNST transform coefficients can exist in 16 positions starting from the DC position, and all remaining positions are filled with 0.

[0396] Therefore, when there are non-zero transform coefficients in the region where zero-padding is applied after residual coding, the LFNST index signaling can be omitted.

[0397] When LFNST can be applied even for 2×M (1×M) or M×2 (M×1) partition blocks, the regions where LFNST transform coefficients are allowed can be specified as follows. Regions outside the allowed transform coefficient regions can be filled with 0, and under the assumption of applying LFNST, LFNST index signaling can be omitted when non-zero transform coefficients exist in regions that should be filled with 0.

[0398] i. When LFNST is applicable to 2×M or M×2 blocks and M=8, only 8 LFNST transform coefficients can be generated for 2×8 or 8×2 partitioned blocks. When the transform coefficients are as follows... Figure 20 When arranged in the scanning order shown, the 8 transformation coefficients can be arranged in scanning order starting from the DC position, and the remaining 8 positions can be filled with 0.

[0399] For a 2×N or N×2 (N>8) partition, 16 LFNST transform coefficients can be generated, and when the transform coefficients are in the form of... Figure 20When arranged in the scan order shown, the 16 transform coefficients can be arranged sequentially starting from the DC position, and the remaining area can be filled with 0. That is, the area in a 2×N or N×2 (N>8) partition block, excluding the top-left 2×8 or 8×2 block, can be filled with 0. Even for a 2×8 or 8×2 partition block, 16 transform coefficients can be generated instead of 8 LFNST transform coefficients, and in this case, no area must be filled with 0. As mentioned above, when applying LFNST, if a non-zero transform coefficient is detected even for a partition block in an area set to be filled with 0, the LFNST index signaling can be omitted, and the LFNST index can be inferred as 0.

[0400] ii. When LFNST can be applied to 1×M or M×1 blocks and M=16, only 8 LFNST transform coefficients can be generated for a 1×16 or 16×1 partition block. When the transform coefficients are arranged in a scan order from left to right or from top to bottom, the 8 transform coefficients can be arranged in the corresponding scan order starting from the DC position, and the remaining 8 positions can be filled with 0.

[0401] For a 1×N or N×1 (N>16) partition block, 16 LFNST transform coefficients can be generated. When the transform coefficients are arranged in a left-to-right or top-to-bottom scan order, the 16 transform coefficients can be arranged starting from the DC position in the corresponding scan order, and the remaining area can be filled with 0. That is, the area in a 1×N or N×1 (N>16) partition block except for the top-left 1×16 or 16×1 block can be filled with 0.

[0402] Even for 1×16 or 16×1 partition blocks, 16 transform coefficients can be generated instead of 8 LFNST transform coefficients, and in this case, there will be no regions that must be filled with 0. As mentioned above, when applying LFNST, if a non-zero transform coefficient is detected for a partition block in a region that is set to be filled with 0, the LFNST index signaling can be omitted and the LFNST index can be inferred to be 0.

[0403] Furthermore, in ISP mode, according to the current VVC standard, the horizontal and vertical directions are treated independently as length conditions, and DST-7 is applied instead of DCT-2 without signaling the MTS index. The horizontal or vertical length is determined to be greater than or equal to 4 and greater than or equal to 16, respectively, and a transformation kernel is determined based on the determined result. Therefore, for cases where LFNST can be applied in ISP mode, the following transformation combinations are feasible.

[0404] 1. When the LFNST index is 0 (including cases where the LFNST index is inferred to be 0), the transformation determination condition under ISP mode can be followed, which is included in the current VVC standard. That is, if the length condition (which is equal to or greater than 4 and equal to or less than 16) is satisfied separately and independently for the horizontal and vertical directions; if so, DST-7 can be applied instead of DCT-2; otherwise, DCT-2 can be applied.

[0405] 2. When the LFNST index is greater than 0, the following two configurations can be used as a transformation.

[0406] A.DCT-2 can be applied to both the horizontal and vertical directions.

[0407] B. The conditions used to determine a single transformation in ISP mode can be followed, which are included in the current VVC standard. In other words, check that the length condition (which is equal to or greater than 4 and equal to or less than 16) is satisfied separately and independently for the horizontal and vertical directions; if so, DST-7 can be applied instead of DCT-2; otherwise, DCT-2 can be applied.

[0408] In ISP mode, image information can be configured such that the LFNST index is sent for each partition block rather than each coding unit. In this case, in the LFNST index signaling method described above, considering that there is only one partition block in the unit sending the LFNST index, it can be determined whether to signal the LFNST index.

[0409] In addition, the signaling order of the LFNST index and the MTS index will be described below.

[0410] According to the example, the LFNST index, which is signaled in residual coding, can be encoded after the coding position of the last non-zero coefficient position, and the MTS index can be encoded immediately after the LFNST index. In this configuration, the LFNST index can be signaled for each transform unit. Alternatively, even if no signal is given in residual coding, the LFNST index can be encoded after the coding of the last valid coefficient position, and the MTS index can be encoded after the LFNST index.

[0411] The syntax for residual coding based on the example is as follows.

[0412] [Table 9]

[0413]

[0414]

[0415] The meanings of the main variables shown in Table 9 are as follows.

[0416] 1. cbWidth, cbHeight: The width and height of the current coding block.

[0417] 2. log2TbWidth, log2TbHeight: The base-2 logarithmic values ​​of the width and height of the current transform block, which can be reduced to the upper left region where non-zero coefficients can exist by reflecting zeroing.

[0418] 3. `sps_lfnst_enabled_flag`: A flag indicating whether LFNST is enabled. A value of 0 indicates that LFNST is not enabled, and a value of 1 indicates that LFNST is enabled. It is defined in the Sequence Parameter Set (SPS).

[0419] 4. CuPredMode[chType][x0][y0]: The prediction mode corresponding to the variable chType and the position (x0, y0). chType can have values ​​of 0 and 1, where 0 indicates the luminance component and 1 indicates the chrominance component. The position (x0, y0) indicates the position on the image, and MODE_INTRA (intra-frame prediction) and MODE_INTER (inter-frame prediction) can be used as the values ​​of CuPredMode[chType][x0][y0].

[0420] 5. IntraSubPartitionsSplit[x0][y0]: The content at position (x0, y0) is the same as in item 4. It indicates which ISP partition was applied at position (x0, y0), and ISP_NO_SPLIT indicates that the coding unit corresponding to position (x0, y0) was not divided into a partition block.

[0421] 6. intra_mip_flag[x0][y0]: The content at position (x0, y0) is the same as in point 4 above. intra_mip_flag is a flag indicating whether matrix-based intra-frame prediction (MIP) prediction mode is applied. If the flag value is 0, it indicates that MIP is not enabled; if the flag value is 1, it indicates that MIP is enabled.

[0422] 7.cIdx: A value of 0 indicates luminance, and values ​​of 1 and 2 indicate the Cb and Cr of the chromaticity components, respectively.

[0423] 8. treeType: Indicates whether it is a single tree or a dual tree (SINGLE_TREE: single tree, DUAL_TREE_LUMA: dual tree for the luminance component, DUAL_TREE_CHROMA: dual tree for the chrominance component).

[0424] 9.tu_cbf_cb[x0][y0]: The content at position (x0, y0) is the same as in item 4. It indicates the coded block flag (CBF) of the Cb component. If its value is 0, it means that there are no non-zero coefficients in the corresponding transform unit of the Cb component, and if its value is 1, it indicates that there are non-zero coefficients in the corresponding transform unit of the Cb component.

[0425] 10. lastSubBlock: This indicates the position of the subblock (coefficient group (CG)) containing the last non-zero coefficient in the scan order. 0 indicates a subblock containing the DC component, and a value greater than 0 indicates a subblock that does not contain the DC component.

[0426] 11. lastScanPos: This indicates the position of the last valid coefficient within a sub-block in scan order. If a sub-block contains 16 positions, it can have values ​​from 0 to 15.

[0427] 12.lfnst_idx[x0][y0]: The LFNST index syntax element to be parsed. If not parsed, it is inferred to be 0. That is, the default value is set to 0, indicating that LFNST is not applied.

[0428] 13. LastSignificantCoeffX, LastSignificantCoeffY: These indicate the x and y coordinates of the last significant coefficient in the transform block. The x-coordinate starts at 0 and increases from left to right, and the y-coordinate starts at 0 and increases from top to bottom. If both variables are 0, it means the last significant coefficient is located at DC.

[0429] 14. cu_sbt_flag: A flag indicating whether Subblock Transformation (SBT) included in the current VVC standard is enabled. If the flag value is 0, it indicates that SBT is not enabled, and if the flag value is 1, it indicates that SBT is enabled.

[0430] 15. `sps_explicit_mts_inter_enabled_flag` and `sps_explicit_mts_intra_enabled_flag`: These flags indicate whether explicit MTS is applied to inter-frame CUs and intra-frame CUs, respectively. If the corresponding flag value is 0, it indicates that MTS is not enabled for inter-frame or intra-frame CUs; ​​if the corresponding flag value is 1, it indicates that MTS is enabled.

[0431] 16.tu_mts_idx[x0][y0]: The MTS index syntax element to be parsed. If not parsed, it is inferred to be 0. That is, the default value is set to 0, indicating that DCT-2 is enabled in both the horizontal and vertical directions.

[0432] As shown in Table 9, in the case of a single tree, the location condition of the last valid coefficient for luminance can be used only to determine whether to signal the LFNST index. That is, if the location of the last valid coefficient is not DC and the last valid coefficient exists in the top-left sub-block (CG) (e.g., a 4×4 block), then the LFNST index is signaled. In this case, for both 4×4 and 8×8 transform blocks, the LFNST index is signaled only if the last valid coefficient exists at positions 0 to 7 in the top-left sub-block.

[0433] In the case of a dual-tree system, the LFNST index is signaled independently of each of the luminance and chrominance components. In the case of chrominance, the LFNST index can be signaled by applying the last valid coefficient position condition only to the Cb component. For the Cr component, the corresponding condition is not checked, and if the CBF value of Cb is 0, the LFNST index can be signaled by applying the last valid coefficient position condition to the Cr component.

[0434] In Table 9, “Min(log2TbWidth,log2TbHeight)>=2” can be represented as “Min(tbWidth,tbHeight)>=4”, and “Min(log2TbWidth,log2TbHeight)>=4” can be represented as “Min(tbWidth,tbHeight)>=16”.

[0435] In Table 9, log2ZoTbWidth and log2ZoTbHeight represent the logarithmic values ​​of the width and height of the top-left region with a base of 2 (base-2), respectively, which can be zeroed out.

[0436] As shown in Table 9, the log2ZoTbWidth and log2ZoTbHeight values ​​can be updated in two places. The first is before parsing the MTS index or LFNST index value, and the second is after parsing the MTS index.

[0437] The first update occurs before parsing the MTS index (tu_mts_idx[x0][y0]) value, so log2ZoTbWidth and log2ZoTbHeight can be set regardless of the MTS index value.

[0438] After resolving the MTS index, log2ZoTbWidth and log2ZoTbHeigh are set for MTS indexes greater than 0 (DST-7 / DCT-8 combination). When DST-7 / DCT-8 is applied independently in each of the horizontal and vertical directions in a single transformation, there can be up to 16 valid coefficients per row or column in each direction. That is, after applying DST-7 / DCT-8 of length 32 or greater, up to 16 transformation coefficients can be derived for each row or column, starting from the left or top. Therefore, in a 2D block, when DST-7 / DCT-8 is applied to both the horizontal and vertical directions, there can be valid coefficients in only up to a 16×16 top-left region.

[0439] Furthermore, when DCT-2 is applied independently in each of the horizontal and vertical directions in the current transformation, there can be up to 32 valid coefficients per row or column in each direction. That is, when applying a DCT-2 of length 64 or greater, up to 32 transformation coefficients can be derived for each row or column, starting from the left or top. Therefore, in a 2D block, when DCT-2 is applied to both the horizontal and vertical directions, valid coefficients can exist in only a maximum of 32×32 in the upper left region.

[0440] Furthermore, when DST-7 / DCT-8 is applied to one side for both the horizontal and vertical directions and DCT-2 is applied to the other side, there can be 16 effective coefficients in the forward direction and 32 effective coefficients in the backward direction. For example, in the case of a 64×8 transform block, if DCT-2 is applied in the horizontal direction and DST-7 is applied in the vertical direction (which may occur when implicit MTS is applied), there can be effective coefficients in up to 32×8 regions in the upper left.

[0441] If, as shown in Table 9, log2ZoTbWidth and log2ZoTbHeight are updated in two places, that is, before resolving the MTS index, the ranges of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix can be determined by log2ZoTbWidth and log2ZoTbHeight, as shown in the table below.

[0442] [Table 10]

[0443]

[0444] Additionally, in this case, the maximum values ​​of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix can be set by reflecting the log2ZoTbWidth and log2ZoTbHeight values ​​during the binarization process of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix.

[0445] [Table 11]

[0446]

[0447] According to the example, when applying ISP mode and LFNST, the canonical text can be configured as shown in Table 12 when applying the signaling in Table 9. Compared to Table 9, the condition that the LFNST index is signaled only when ISP mode is not included has been removed (IntraSubPartitionsSplit[x0][y0] == ISP_NO_SPLIT in Table 9).

[0448] In a single tree, when the LFNST index sent for the luma component (cIdx = 0) is reused for the chroma component, the LFNST index sent for the first ISP block with valid coefficients can be applied to the chroma transform block. Alternatively, even in a single tree, the LFNST index can be signaled for the chroma component separately from the LFNST index signaled for the luma component. The variables in Table 12 are described the same as those in Table 9.

[0449] [Table 12]

[0450]

[0451] According to another example, in Table 12, when the last valid coefficient is allowed to be located only in the DC position for ISP, the conditions for resolving the LFNST index can be varied as follows.

[0452] [Table 13]

[0453]

[0454] As shown in the example, the LFNST index and / or MTS index can be signaled at the encoding unit level. As mentioned above, the LFNST index can have three values: 0, 1, and 2, where 0 indicates that LFNST is not applied, and 1 and 2 indicate that the selected LFNST set includes the first and second candidates of the two LFNST kernel candidates, respectively. The LFNST index is encoded using truncated univariate binarization, and the values ​​0, 1, and 2 can be encoded as bin strings of 0, 10, and 11, respectively.

[0455] Based on the example, LFNST can be applied only when DCT-2 is applied to both the horizontal and vertical directions in a single transformation. Therefore, if the MTS index is signaled after the LFNST index is signaled, the MTS index can be signaled only when the LFNST index is 0, and when the LFNST index is not 0, a single transformation can be performed by applying DCT-2 to both the horizontal and vertical directions without signaling the MTS index.

[0456] The MTS index can have values ​​0, 1, 2, 3, and 4, where 0, 1, 2, 3, and 4 indicate that DCT-2 / DCT-2, DST-7 / DST-7, DCT-8 / DST-7, DST-7 / DCT-8, and DCT-8 / DCT-8 are applied in the horizontal and vertical directions, respectively. Additionally, the MTS index can be encoded using truncated unary binarization, and the values ​​0, 1, 2, 3, and 4 can be encoded as bin strings of 0, 10, 110, 1110, and 1111, respectively.

[0457] The LFNST and MTS indices can be signaled at the coding unit level, and the MTS index can be encoded sequentially after the LFNST index at the coding unit level. The coding unit syntax table used for this is as follows.

[0458] [Table 14]

[0459]

[0460] The variables LfnstDcOnly and LfnstZeroOutSigCoeffFlag in Table 14 can be set as shown in Table 15 below.

[0461] The variable LfnstDcOnly is equal to 1 for a transform block with a coded block flag (CBF) of 1 (0 if at least one valid coefficient exists in the corresponding block, otherwise 0), and all last valid coefficients are located at the DC position (top left position); otherwise, it is equal to 0. Specifically, in the case of dual-tree luma, the position of the last valid coefficient is checked for a luma transform block, and in the case of dual-tree chroma, the position of the last valid coefficient is checked for both the Cb and Cr transform blocks. In the case of single-tree, the position of the last valid coefficient can be checked for luma, Cb, and Cr transform blocks.

[0462] If a valid coefficient exists at the zeroing position when applying LFNST, the variable LfnstZeroOutSigCoeffFlag equals 0; otherwise, it equals 1.

[0463] In Table 14 and subsequent tables, lfnst_idx[x0][y0] indicates the LFNST index of the corresponding coding unit, while tu_mts_idx[x0][y0] indicates the MTS index of the corresponding coding unit.

[0464] As shown in Table 14, the conditions used to signal lfnst_idx[x0][y0] can include a condition for checking whether the value of transform_skip_flag[x0][y0] is 0 (!transform_skip_flag[x0][y0]). In this case, the condition for checking whether the existing value of tu_mts_idx[x0][y0] is 0 (i.e., checking whether DCT-2 is performed in both the horizontal and vertical directions) can be omitted.

[0465] `transform_skip_flag[x0][y0]` indicates whether the encoding unit is encoded in a transform-skip mode where the transform is skipped, and this flag is signaled before the MTS and LFNST indices. That is, since `lfnst_idx[x0][y0]` is signaled before the value of `tu_mtx_idx[x0][y0]`, it is possible to check only the condition regarding the value of `transform_skip_flag[x0][y0]`.

[0466] As shown in Table 14, multiple conditions are checked when encoding tu_mts_idx[x0][y0], and as mentioned above, tu_mts_idx[x0][y0] is only signaled when the value of lfnst_idx[x0][y0] is 0.

[0467] tu_cbf_luma[x0][y0] is a flag indicating whether there are valid coefficients for the luminance component, and cbWidth and cbHeight indicate the width and height of the coding unit of the luminance component, respectively.

[0468] According to Table 14, when both the width and height of the coding unit of the luminance component are 32 or less, a signal is sent to tu_mts_idx[x0][y0]. In other words, whether to apply MTS is determined by the width and height of the coding unit of the luminance component.

[0469] According to another example, when transform block (TU) tiling occurs (e.g., when the maximum transform size is set to 32, dividing a 64×64 coding unit into four 32×32 transform blocks and encoding them), the MTS index can be signaled based on the size of each transform block. For example, when both the width and height of the transform block are 32 or less, the same MTS index value can be applied to all transform blocks in the coding unit, thus applying the same transform. Additionally, when transform block tiling occurs, the value of tu_cbf_luma[x0][y0] in Table 14 can be the CBF value of the top-left transform block, or it can be set to 1 even if the CBF value of one of the transform blocks is 1.

[0470] As shown in Table 14, even in ISP mode (IntraSubPartitionsSplitType != ISP_NO_SPLIT), lfnst_idx[x0][y0] can be configured to be signaled, and the same LFNST index value can be applied to all ISP partition blocks.

[0471] In addition, tu_mts_idx[x0][y0] can be signaled only in modes other than ISP mode (IntraSubPartitionsSplit[x0][y0] == ISP_NO_SPLIT).

[0472] As shown in Table 15, when the MTS index is signaled immediately after the LFNST index, information about the first transformation is unavailable during residual coding. In other words, the MTS index is signaled after residual coding. Therefore, in the residual coding section, the part that performs zeroing while retaining only 16 coefficients for a 32-coefficient DST-7 or DCT-8 can be modified as shown in Table 15 below.

[0473] [Table 15]

[0474]

[0475]

[0476] As shown in Table 15, in the process of determining log2ZoTbWidth and log2ZoTbHeight (where log2ZoTbWidth and log2ZoTbHeight represent the base-2 logarithmic values ​​of the width and height of the upper left region after the zeroing is performed), the value of tu_mts_idx[x0][y0] can be omitted.

[0477] The binarization of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix in Table 15 can be determined based on log2ZoTbWidth and log2ZoTbHeight as shown in Table 11.

[0478] In addition, as shown in Table 15, when determining log2ZoTbWidth and log2ZoTbHeight in the residual coding, a condition for checking sps_mts_enable_flag can be added.

[0479] The TR indicator in Table 11 is the truncated Rice binarization method, and the final effective coefficient information can be binarized based on cMax and cRiceParam as defined in Table 11 according to the method described in the table below.

[0480] [Table 16]

[0481]

[0482] As shown in the example, when recording the position information of the last valid coefficients of the luminance transform block during residual coding, the MTS index can be signaled as shown in Table 17.

[0483] [Table 17]

[0484]

[0485] In Table 17, LumaLastSignificantCoeffX and LumaLastSignificantCoeffY indicate the X and Y coordinates of the last effective coefficient position of the luminance transform block, respectively. A condition has been added to Table 17 that both LumaLastSignificantCoeffX and LumaLastSignificantCoeffY must be less than 16. When either of them is 16 or greater, applying DCT-2 in both the horizontal and vertical directions suggests that the signaling of tu_mts_idx[x0][y0] has been omitted, and that DCT-2 is applied in both the horizontal and vertical directions.

[0486] When both LumaLastSignificantCoeffX and LumaLastSignificantCoeffY are less than 16, this means that the last valid coefficients exist in the top-left 16×16 region. In the current VVC standard, when applying a 32-length DST-7 or DCT-8, this indicates the possibility that a zeroing process has been applied, retaining only 16 transform coefficients from the left or top. Therefore, the transform kernel used for a single transform can be specified by signaling tu_mts_idx[x0][y0].

[0487] In another example, the coding unit syntax table and the residual coding syntax table are as follows.

[0488] [Table 18]

[0489]

[0490] [Table 19]

[0491]

[0492] In Table 18, MtsZeroOutSigCoeffFlag is initially set to 1, and this value can be changed in the residual coding of Table 19. When there are valid coefficients in the region to be filled with 0 by zeroing (LastSignificantCoeffX>15||LastSignificantCoeffY>15), the value of the variable MtsZeroOutSigCoeffFlag changes from 1 to 0. In this case, no signal is sent to the MTS index, as shown in Table 19.

[0493] Furthermore, as shown in the example table below, when determining log2ZoTbWidth and log2ZoTbHeight in residual coding, conditions can be added for checking sps_mts_enable_flag.

[0494] [Table 20]

[0495]

[0496] As shown in Table 20, when tu_cbf_luma[x0][y0] is 1, MtsZeroOutSigCoeffFlag can be set to 1, while when tu_cbf_luma[x0][y0] is 0, the existing MtsZeroOutSigCoeffFlag value can be maintained. Therefore, when tu_cbf_luma[x0][y0] is 0 and the MtsZeroOutSigCoeffFlag value remains 0, the encoding of mts_idx[x0][y0] can be omitted. In other words, when the CBF value of the luminance component is 0, the MTS index is meaningless because no transformation is applied, so the encoding of the MTS index can be omitted.

[0497] The following figures are provided to illustrate specific examples of this specification. Since the names of particular devices or signals / messages / fields described in the figures are presented by way of example, the technical features of this specification are not limited to the specific names used in the following figures.

[0498] Figure 22 This is a flowchart illustrating the operation of a video decoding device according to an embodiment of this document.

[0499] Figure 22 Each step disclosed in the document is based on the above. Figures 5 to 21 Some content described above. Therefore, omissions or simplifications will be made. Figures 5 to 21 The description repeats specific content.

[0500] According to the embodiment, the decoding device 300 can receive residual information from the bit stream (S2210).

[0501] More specifically, the decoding device 300 can decode information about the quantization transform coefficients of the current block from the bitstream, and can deduce the quantization transform coefficients of the target block based on the information about the quantization transform coefficients of the current block. The information about the quantization transform coefficients of the target block can be incorporated into the Sequence Parameter Set (SPS) or the stripe header, and can include at least one of the following: information about whether a reduced transform (RST) is applied, information about the reduction factor, information about the minimum transform size for applying the reduced transform, information about the maximum transform size for applying the reduced transform, and information about the transform index indicating any one of the transform kernel matrix and the simplified inverse transform size included in the transform set.

[0502] In addition, the decoding device can also receive information about the intra-prediction mode of the current block and information about whether ISP encoding or ISP mode is applied to the current block. The decoding device can deduce whether the current block is divided into a predetermined number of sub-partition transform blocks by receiving and parsing flag information indicating whether ISP encoding or ISP mode is applied. Here, the current block can be a coded block. Furthermore, the decoding device can deduce the size and number of sub-partition blocks by flag information indicating the direction in which the current block will be divided.

[0503] The decoding device 300 can derive the transformation coefficients by dequantizing the residual information about the current block (i.e., the quantized transformation coefficients) (S2220).

[0504] The derived transform coefficients can be arranged in 4×4 blocks according to the reverse diagonal scan order, and the transform coefficients within a 4×4 block can also be arranged according to the reverse diagonal scan order. In other words, the transform coefficients that have been dequantized can be arranged according to the reverse scan order used in video codecs such as VVC or HEVC.

[0505] The transform coefficients derived from this residual information can be either dequantized transform coefficients as described above, or quantized transform coefficients. In other words, the transform coefficients can be any data that can be checked to see if it is non-zero data in the current block, regardless of whether it has been quantized.

[0506] Decoding devices can derive residual samples by applying an inverse transform to the quantization transform coefficients.

[0507] As described above, the decoding device can derive residual samples by applying LFNST as an inseparable transform or MTS as a separable transform, and such transforms can be performed based on the LFNST index indicating the LFNST kernel (i.e., the LFNST matrix) and the MTS index indicating the MTS kernel, respectively.

[0508] The decoding device can determine whether to parse the MTS index used to apply the MTS to the current block, and according to the example, can determine the tree type of the current block, the partition type of the current block, and whether to perform zeroing for the MTS on the current block (S2230).

[0509] When the tree type of the current block is not dual-tree chroma and the LFNST index indicating the LFNST core applied to the current block is 0, the decoding device can determine and resolve the MTS index.

[0510] In other words, when the tree type of the current block is single-tree or dual-tree luminance, the decoding device can resolve the MTS index when the LFNST index is 0 (that is, when the LFNST is not applied to the current block).

[0511] However, even without resolving the MTS index, MTS can be applied implicitly under certain conditions. For example, implicit MTS can be applied when the current block is divided into sub-blocks, when Sub-Block Transform (SBT) is applied, or when matrix-based intra-prediction (MIP) mode is not applied to intra-prediction of the current block.

[0512] Additionally, according to the example, the decoding device can resolve the MTS index when the larger of the current block's width and height is less than or equal to 32. That is, the MTS cannot be applied when the current block's width or height is greater than 32.

[0513] Additionally, according to the example, the MTS index can be resolved when the current block is not divided into multiple sub-blocks and the sub-block transform used to perform transforms by dividing coding units is not applied to the current block. As mentioned above, when ISP or SBT is applied to the current block, MTS can be performed implicitly, and the MTS index can be notified without signaling it.

[0514] Additionally, the decoding device can resolve the MTS index based on whether a zeroing process for the MTS has been performed. If a valid coefficient exists in a second region of the current block, excluding the first region in the upper left where valid transform coefficients can exist, it can be determined that a zeroing process has not been performed. In other words, if no valid coefficient exists in the second region, it is determined that a zeroing process has been performed, and the MTS index can be resolved.

[0515] The first region can be the 16×16 region at the top left of the current block.

[0516] The decoding device can deduce the variable MtsZeroOutSigCoeffFlag, which indicates whether zeroing is performed when MTS is applied. The variable MtsZeroOutSigCoeffFlag indicates whether transform coefficients exist in the region outside the top-left region (i.e., the region other than the top-left 16×16 region) after MTS is performed and can be initially set to 1. Its value can change from 1 to 0 when transform coefficients exist in the region outside the 16×16 region. When the value of the variable MtsZeroOutSigCoeffFlag is 0, no signal is sent to the MTS index.

[0517] According to the example, the decoding device can check the transform skip flag value, and when the value is 0, resolve the MTS index.

[0518] At least two conditions for resolving an MTS index can be combined into an AND condition. According to the example, a decoding device can resolve an MTS index when the current block's tree type is not dual-tree chroma, the LFNST index indicating the LFNST core is 0, the larger of the current block's width and height is less than or equal to 32, the current block has not been divided into sub-blocks, no sub-block transformation is applied to the current block, and zeroing according to the MTS has been performed.

[0519] The decoding device can receive and parse at least one of the LFNST index or MTS index at the encoding unit level, and can parse the LFNST index indicating the LFNST core before (i.e. immediately before) the MTS index indicating the MTS core.

[0520] After parsing the MTS index, the decoding device can deduce the residual sample of the current block by applying the MTS to the current block based on the MTS index (S2240).

[0521] Subsequently, the decoding device 300 can generate a reconstructed sample based on the residual sample of the current block and the predicted sample of the current block (S2250).

[0522] The following figures are provided to illustrate specific examples of this specification. Since the names of particular devices or signals / messages / fields described in the figures are presented by way of example, the technical features of this specification are not limited to the specific names used in the following figures.

[0523] Figure 23 This is a flowchart illustrating the operation of a video encoding device according to an embodiment of this document.

[0524] Figure 23 Each step disclosed in the document is based on the above. Figures 5 to 21 Some of the content described above. Therefore, the above will be omitted or briefly explained. Figure 2 and Figures 5 to 21 Explanation of the specific content that is repeated in the description.

[0525] According to the implementation, the coding device 200 can derive the prediction sample of the current block based on the intra-prediction mode applied to the current block (S2310).

[0526] When the ISP is applied to the current block, the encoding device can perform prediction for each sub-partition transform block.

[0527] The encoding device can determine whether to apply ISP encoding or ISP mode to the current block (i.e., the encoding block), and based on the determination result, it can determine the direction in which the current block should be divided, and deduce the size and number of sub-blocks to be divided.

[0528] The encoding device 200 can derive the residual sample of the current block based on the predicted sample (S2320).

[0529] The encoding device 200 can derive the transform coefficients of the current block by applying at least one of LFNST or MTS to the residual sample. The transform coefficients can be arranged according to a predetermined scan order. According to the example, the transform coefficients of the current block can be derived based on the MTS of the residual sample (S2330).

[0530] A transformation can be performed using multiple transform kernels, such as MTS, and in this case, the transform kernel can be selected based on the intra-frame prediction mode.

[0531] After deriving the transform coefficients by applying MTS, the encoding device can zero out the rest of the current block except for a specific region in the upper left corner (e.g., a 16×16 region).

[0532] The encoding device can encode the MTS index based on the tree type of the current block, the partition type of the current block, and whether the MTS is cleared for the current block, and can also encode the residual information derived by quantization of the transform coefficients (S2340).

[0533] When the tree type of the current block is not dual-tree chroma and the LFNST index indicating the LFNST core applied to the current block is 0, the encoding device can configure the image information to signal the MTS index, and can also signal the MTS index.

[0534] In other words, when the tree type of the current block is single-tree or dual-tree luminance, the encoding device can signal the MTS index when the LFNST index is 0 (that is, when LFNST is not applied to the current block).

[0535] However, even without signaling the MTS index, MTS can be applied implicitly when certain conditions are met. For example, implicit MTS can be applied when the current block is divided into sub-blocks or when Sub-Block Transform (SBT) is applied, or when matrix-based intra-prediction (MIP) mode is not applied to intra-prediction of the current block.

[0536] Furthermore, according to the example, the encoding device can be configured with image information such that it signals the MTS index when the larger of the width and height of the current block is less than or equal to 32. In other words, the MTS cannot be applied when the width or height of the current block is greater than 32.

[0537] Furthermore, according to the example, when the current block is not divided into multiple sub-blocks and the sub-block transformation performed by dividing coding units is not applied to the current block, the MTS index may not be signaled. As mentioned above, when ISP or SBT is applied to the current block, MTS may be performed implicitly, and the MTS index may not be signaled.

[0538] Furthermore, the encoding device can signal the MTS index to indicate whether a zeroing operation for the MTS has already been performed. If a valid coefficient exists in a second region of the current block, excluding the first region in the upper left where valid transform coefficients can exist, the encoding device can determine that a zeroing operation is not required. In other words, if no valid coefficient exists in the second region, it is determined that a zeroing operation has been performed and the MTS index can be signaled.

[0539] The first region can be the 16×16 region at the top left of the current block.

[0540] The encoding device can derive the variable `MtsZeroOutSigCoeffFlag`, which indicates whether zeroing should be performed when applying MTS, and can configure it as image information for MTS indexing signaling. The variable `MtsZeroOutSigCoeffFlag` indicates whether transform coefficients exist in the region outside the top-left region (i.e., outside the top-left 16×16 region) after MTS execution due to zeroing, and can be initially set to 1. Its value can change from 1 to 0 when transform coefficients exist in the region outside the 16×16 region. When the value of the variable `MtsZeroOutSigCoeffFlag` is 0, no signal is sent to notify MTS indexing.

[0541] According to the example, the encoding device can check the transform skip flag value and signal the MTS index when the value is 0.

[0542] At least two conditions for resolving the MTS index can be combined into an AND condition. According to the example, the encoding device can signal the MTS index when the current block's tree type is not dual-tree chroma, the LFNST index indicating the LFNST kernel is 0, the larger of the current block's width and height is less than or equal to 32, the current block has not been divided into sub-blocks, no sub-block transformation is applied to the current block, and zeroing according to the MTS has been performed.

[0543] The encoding device can signal at least one of the LFNST index or MTS index at the encoding unit level, and can be configured with image information such that the LFNST index indicating the LFNST core is signaled before (i.e. immediately before) the MTS index indicating the MTS core.

[0544] The encoding device can generate residual information that includes information about the quantization transform coefficients. The residual information can include the transform-related information / syntax elements mentioned above. The encoding device can encode image / video information including the residual information and output the encoded image / video information as a bitstream.

[0545] More specifically, the encoding device can generate information about the quantization transform coefficients and can encode the information about the generated quantization transform coefficients.

[0546] When LFNST can be applied, the image information may include LFNST indices that indicate the LFNST matrix.

[0547] The syntax elements of the LFNST index according to this embodiment can indicate whether (inverse) LFNST is applied and any LFNST matrix included in the LFNST set, and when the LFNST set includes two transformation kernel matrices, the syntax elements of the LFNST index can have three values.

[0548] According to the implementation method, when the partitioning tree structure of the current block is a dual-tree type, each of the luma block and chroma block can be encoded with an LFNST index.

[0549] According to the implementation, the syntax element values ​​of the transform index can be deduced as 0, 1, and 2, where 0 indicates that (inverse) LFNST is not applied to the current block, 1 indicates the first LFNST matrix in the LFNST matrix, and 2 indicates the second LFNST matrix in the LFNST matrix.

[0550] In this disclosure, at least one of quantization / dequantization and / or transformation / inverse transformation may be omitted. When quantization / dequantization is omitted, the quantization transformation coefficients may be referred to as transformation coefficients. When transformation / inverse transformation is omitted, the transformation coefficients may be referred to as coefficients or residual coefficients, or, for the sake of consistency, may still be referred to as transformation coefficients.

[0551] Furthermore, in this disclosure, quantization transform coefficients and transform coefficients can be referred to as transform coefficients and scaling transform coefficients, respectively. In this case, residual information can include information about the transform coefficients, and this information can be signaled via residual coding syntax. Transform coefficients can be derived based on residual information (or information about transform coefficients), and scaling transform coefficients can be derived through the inverse transform (scaling) of the transform coefficients. Residual samples can be derived based on the inverse transform (scaling) of the scaling transform coefficients. These details can also be applied / expressed in other parts of this disclosure.

[0552] In the above embodiments, the method is explained based on a flowchart using a series of steps or blocks. However, this disclosure is not limited to the order of the steps, and a step may be performed in a different order or sequence than described above, or a step may be performed concurrently with other steps. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and another step may be incorporated or one or more steps in the flowchart may be deleted without affecting the scope of this disclosure.

[0553] The methods described above according to this disclosure can be implemented in software form, and the encoding and / or decoding devices according to this disclosure can be included in devices for image processing such as televisions, computers, smartphones, set-top boxes, and display devices.

[0554] When the embodiments of this disclosure are implemented by software, the above methods can be implemented as modules (steps, functions, etc.) for performing the above functions. These modules can be stored in memory and can be executed by a processor. The memory can be internal or external to the processor and can be connected to the processor in various well-known ways. The processor may include application-specific integrated circuits (ASICs), other chipsets, logic circuits, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. That is, the embodiments described in this disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure can be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0555] Furthermore, the decoding and encoding devices using this disclosure can include multimedia broadcast transceivers, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices (such as video communication), mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, over-the-top (OTT) video devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, and medical video devices, and can be used to process video signals or data signals. For example, over-the-top (OTT) video devices can include game consoles, Blu-ray players, internet access TVs, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0556] Furthermore, the processing methods of this disclosure can be produced in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data having the data structure according to this disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include various storage devices and distributed storage devices for storing computer-readable data. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. In addition, computer-readable recording media include media implemented in the form of a carrier wave (e.g., transmission over the Internet). Furthermore, bitstreams generated by encoding methods can be stored in computer-readable recording media or transmitted via wired or wireless communication networks. Additionally, embodiments of this disclosure can be implemented as computer program products by program code, and the program code can be executed on a computer according to embodiments of this disclosure. The program code can be stored on a computer-readable carrier.

[0557] The claims disclosed herein can be combined in various ways. For example, the technical features of the method claims can be combined to be implemented or performed in a device, and the technical features of the device claims can be combined to be implemented or performed in a method. Furthermore, the technical features of the method claims and the device claims can be combined to be implemented or performed in a device, and the technical features of the method claims and the device claims can be combined to be implemented or performed in a method.

Claims

1. An image decoding device, the image decoding device comprising: Memory; as well as At least one processor, connected to the memory, is configured to: Receive residual information from the bitstream. The transformation coefficients of the current block are derived based on the residual information. The residual samples of the current block are derived by applying the inverse transform to the transform coefficients based on the multi-transform selection MTS index of the current block, and A reconstructed image is generated based on the residual samples. The MTS index is obtained from the bitstream based on the tree type of the current block, the partition type of the current block, the LFNST index associated with the low-frequency inseparable transform LFNST kernel of the current block, and a variable related to whether there is at least one effective coefficient in a second region other than the first region within the current block. The first region is a 16×16 region including the top-left sample position of the current block. The LFNST index and the MTS index are obtained from the coding unit syntax, and The MTS index is obtained immediately after the LFNST index is obtained from the encoding unit syntax.

2. The image decoding device according to claim 1, wherein, The MTS index is obtained from the bitstream based on the fact that the tree type of the current block is not a dual-tree chroma and the LFNST index is 0.

3. The image decoding device according to claim 2, wherein, The MTS index is obtained from the bitstream based on the fact that the larger of the width and height of the current block is less than or equal to 32.

4. The image decoding device according to claim 3, wherein, The MTS index is obtained from the bitstream based on the fact that the current block has not been divided into multiple sub-blocks and the sub-block transformation performed by dividing the coding unit to perform the inverse transformation has not been applied to the current block.

5. The image decoding device according to claim 4, wherein, The MTS index is obtained from the bitstream based on the absence of the at least one valid coefficient in the second region.

6. An image encoding device, the image encoding device comprising: Memory; as well as At least one processor, connected to the memory, is configured to: Derive the predicted samples for the current block. Based on the predicted samples, derive the residual samples for the current block. The transformation coefficients of the current block are derived based on the transformation applied to the residual samples, and The residual information derived from the quantization of the transform coefficients and the multi-transform selection (MTS) index associated with the transform kernel for the transform are encoded. Specifically, the MTS index is encoded into the bitstream based on the tree type of the current block, the partition type of the current block, the LFNST index associated with the low-frequency inseparable transform LFNST kernel of the current block, and a variable related to whether there is at least one valid coefficient in a second region other than the first region within the current block. The first region is a 16×16 region including the top-left sample position of the current block. Specifically, the LFNST index and the MTS index are encoded into the coding unit syntax, and Specifically, the MTS index is encoded immediately after the LFNST index is encoded into the encoding unit syntax.

7. The image encoding device according to claim 6, wherein, Based on the fact that the tree type of the current block is not dual-tree chroma and the LFNST index is 0, the MTS index is encoded into the bitstream.

8. The image encoding device according to claim 7, wherein, The MTS index is encoded into the bitstream based on the fact that the larger of the width and height of the current block is less than or equal to 32.

9. The image encoding device according to claim 8, wherein, Based on the fact that the current block has not been divided into multiple sub-blocks and the sub-block transformation performed by dividing the coding unit has not been applied to the current block, the MTS index is encoded into the bitstream.

10. The image encoding device according to claim 9, wherein, The MTS index is encoded into the bitstream based on the absence of the at least one valid coefficient in the second region.

11. An apparatus for transmitting image information data, the apparatus comprising: At least one processor, configured to obtain a bitstream of image information including residual information, wherein the bitstream is generated by the following steps: deriving a prediction sample of the current block, deriving a residual sample of the current block based on the prediction sample, deriving transform coefficients of the current block based on a transform for the residual sample, and encoding the residual information derived by quantization of the transform coefficients and a Multiple Transform Selection (MTS) index associated with a transform kernel for the transform; and A transmitter configured to transmit the data, including the bitstream of the image information. Specifically, the MTS index is encoded into the bitstream based on the tree type of the current block, the partition type of the current block, the LFNST index associated with the low-frequency inseparable transform LFNST kernel of the current block, and a variable related to whether there is at least one valid coefficient in a second region other than the first region within the current block. The first region is a 16×16 region including the top-left sample position of the current block. Specifically, the LFNST index and the MTS index are encoded into the coding unit syntax, and Specifically, the MTS index is encoded immediately after the LFNST index is encoded into the encoding unit syntax.

Images