Transform-based image encoding method and apparatus therefor

Abstract

According to the present disclosure, a method of image decoding performed by a decoding device includes the steps of receiving image information including residual information about a current block; deriving transform coefficients for the current block based on the residual information; and deriving residual samples for the current block based on the transform coefficients, wherein the image information includes at least one of multi-transform selection (MTS) index information, low frequency non-separable transform (LFNST) index information, and non-separable primary transform (NSPT) index information.

Description

Technical Field

[0001] This disclosure relates to image / video coding methods and image / video coding devices. Background Technology

[0002] Video coding is used in a variety of applications, such as digital storage media, television broadcasting, video streaming services, and real-time communications. Recently, the demand for high-resolution and high-quality video has been growing across various sectors.

[0003] As images / videos reach higher resolutions and higher quality, their data size increases, leading to a relative increase in the amount of information or bits transmitted.

[0004] Furthermore, there is increasing interest and demand for immersive media, such as virtual reality (VR), augmented reality (AR), mixed reality (MR) content, and holograms. Additionally, there is a growing trend of using immersive media to deliver immersive experiences across various sectors, including gaming, education, healthcare, real estate, and marketing.

[0005] Therefore, efficient video coding techniques are needed to efficiently compress, transmit, store, and reproduce high-resolution and high-quality video information with various characteristics as described above. Summary of the Invention

[0006] Technical solution

[0007] According to embodiments of this disclosure, methods and apparatus for enhancing video / image coding efficiency are provided.

[0008] According to embodiments of this disclosure, a method and apparatus for transform-based video coding are provided.

[0009] According to embodiments of this disclosure, an image decoding method performed by a decoding device is provided. The method includes the following steps: receiving image information including residual information for a current block; deriving transform coefficients for the current block based on the residual information; and deriving residual samples for the current block based on the transform coefficients, wherein the image information includes at least one of multiple transform selection (MTS) index information, low-frequency non-separable transform (LFNST) index information, or non-separable master transform (NSPT) index information.

[0010] According to embodiments of this disclosure, an image encoding method performed by an encoding device is provided. The method includes the following steps: deriving residual samples for a current block; deriving transform coefficients for the current block based on the residual samples; generating residual information for the current block based on the transform coefficients; and encoding image information including the residual information, wherein the image information includes at least one of Multiple Transform Selection (MTS) index information, Low Frequency Inseparable Transform (LFNST) index information, or Inseparable Master Transform (NSPT) index information.

[0011] According to embodiments of this disclosure, a decoding apparatus for image decoding is provided. The decoding apparatus includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the following operations: receiving image information including residual information for a current block; deriving transform coefficients for the current block based on the residual information; and deriving residual samples for the current block based on the transform coefficients, wherein the image information includes at least one of multiple transform selection (MTS) index information, low-frequency non-separable transform (LFNST) index information, or non-separable master transform (NSPT) index information.

[0012] According to embodiments of this disclosure, an encoding apparatus for image encoding is provided. The encoding apparatus includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the following operations: deriving residual samples for a current block; deriving transform coefficients for the current block based on the residual samples; generating residual information for the current block based on the transform coefficients; and encoding image information including the residual information, wherein the image information includes at least one of multiple transform selection (MTS) index information, low-frequency non-separable transform (LFNST) index information, or non-separable master transform (NSPT) index information.

[0013] According to embodiments of the present disclosure, a method for transmitting video / video data is provided, the video / video data comprising a bitstream generated by a video / image encoding method according to at least one of embodiments of the present disclosure.

[0014] According to embodiments of the present disclosure, an apparatus is provided for transmitting video / video data, the video / video data comprising a bitstream generated by a video / image encoding method according to at least one of embodiments of the present disclosure.

[0015] According to embodiments of the present disclosure, a computer-readable storage medium is provided that stores a program for performing a method according to at least one of the embodiments of the present disclosure.

[0016] According to embodiments of the present disclosure, a computer-readable digital storage medium is provided that stores encoded video / image information generated by a video / image encoding method according to at least one of the embodiments of the present disclosure.

[0017] According to embodiments of the present disclosure, a computer-readable digital storage medium is provided that stores encoded information or encoded video / image information such that a decoding device performs a video / image decoding method according to at least one of the embodiments of the present disclosure.

[0018] Beneficial effects

[0019] According to the embodiments of this disclosure, the overall video / image compression efficiency can be enhanced.

[0020] According to embodiments of this disclosure, transformation performance for the current block can be improved.

[0021] According to embodiments of this disclosure, signal-related information can be efficiently communicated.

[0022] According to embodiments of this disclosure, when NSPT is applied to the current block, transform-related information can be efficiently notified by signals to efficiently perform the transform, and the amount of transform-related information can be reduced, thereby improving the overall coding efficiency.

[0023] According to embodiments of this disclosure, when LFNST is applied to the current block, transform-related information can be efficiently notified by signals to efficiently perform the transform, and the amount of transform-related information can be reduced, thereby improving the overall coding efficiency.

[0024] According to embodiments of this disclosure, LFNST or NSPT can be adaptively executed by signaling index information based on the size of the current block, thereby improving overall coding efficiency. Attached Figure Description

[0025] Figure 1 Examples of video / image coding systems to which embodiments of the present disclosure may be applied are illustrated schematically.

[0026] Figure 2 This is a diagram that schematically illustrates the configuration of a video / image encoding device to which embodiments of the present disclosure may be applied.

[0027] Figure 3 This is a diagram that schematically illustrates the configuration of a video / image decoding device to which embodiments of the present disclosure may be applied.

[0028] Figure 4 An example of a video / image coding method based on residual processing is shown.

[0029] Figure 5An example of a video / image decoding method based on residual processing is shown.

[0030] Figure 6 Multiple transformation techniques according to this disclosure are illustrated by way of example.

[0031] Figure 7 This is a diagram illustrating an example of LFNST.

[0032] Figure 8 This is a diagram illustrating an example of deriving the MST set.

[0033] Figure 9 An example of deriving DIMM-based intra-frame modes for MTS and LFNST is given.

[0034] Figure 10 An NSPT core based on block size is illustrated as an example.

[0035] Figure 11 An ROI for LFNST 16 is illustrated as an example.

[0036] Figure 12 An ROI for LFNST 8 is illustrated as an example.

[0037] Figure 13 An exemplary MIP prediction sample for constructing HoG is illustrated.

[0038] Figure 14 Context modeling for the LFNST and NSPT transform coefficients is illustrated exemplarily.

[0039] Figure 15 A video / image coding method according to an embodiment of the present disclosure is illustrated schematically.

[0040] Figure 16 A video / image decoding method according to an embodiment of the present disclosure is illustrated schematically. Detailed Implementation

[0041] Because this disclosure can have various modifications and implementations, specific implementations are illustrated in the accompanying drawings and will be described in detail. However, it should be understood that there is no intention to limit the implementations of this disclosure to that specific implementation. The terminology used herein is for the purpose of describing specific implementations only and is not intended to limit the technical spirit of this disclosure. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise. As used herein, the term "and / or" includes any and all combinations of the associated listed items. As used herein, the terms "comprising," "including," and "having" specify the presence of the stated features, quantities, operations, elements, components, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, elements, components, and / or combinations thereof. In this disclosure, the use of the term "may" (e.g., regarding what the example or implementation may include or implement) in conjunction with an example or implementation indicates the existence of at least one example or implementation that includes or implements that feature, but all examples are not limited thereto, and the corresponding feature or configuration may be omitted.

[0042] Each component in the accompanying drawings described in this disclosure is shown independently to facilitate the explanation of its different features and functions, which does not imply that each component is implemented as separate hardware or separate software. For example, two or more of these components may be combined to form a single component, or a single component may be divided into multiple components. Embodiments in which components are integrated and / or separated are also included within the scope of this disclosure without departing from its spirit.

[0043] In this disclosure, “A or B” can mean “A only”, “B only”, or “both A and B”. In other words, “A or B” can be interpreted in this disclosure as “A and / or B”. For example, “A, B or C” in this disclosure can mean “A only”, “B only”, “C only”, or “any and all combinations of A, B and C”.

[0044] As used in this article, a forward slash ( / ) or a comma can mean "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0045] In this disclosure, "at least one of A and B" can mean "only A", "only B" or "both A and B". Furthermore, the expressions "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same way as "at least one of A and B".

[0046] In this disclosure, "at least one of A, B, and C" can mean "A only", "B only", "C only", or "any and all combinations of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" can mean "at least one of A, B, and C".

[0047] The brackets used in this disclosure may mean "for example". Specifically, when indicated as "prediction (intra-frame prediction)", "intra-frame prediction" can be presented as an example of "prediction". In other words, "prediction" in this disclosure is not limited to "intra-frame prediction", and "intra-frame prediction" can be presented as an example of "prediction". Furthermore, even when indicated as "prediction (i.e., intra-frame prediction)", "intra-frame prediction" can be presented as an example of "prediction".

[0048] In this disclosure, the technical features described in a single figure may be implemented independently or simultaneously.

[0049] This disclosure relates to video / image coding. For example, the methods / implementations described in this disclosure can be applied to enhanced compression models or methods disclosed in the H.267 standard. Furthermore, the methods / implementations disclosed herein can be applied to methods disclosed in the AOMedia Video 2 (AV2) standard or next-generation video / image coding standards (e.g., H.268 and H.269).

[0050] In this disclosure, encoding may include encoding and / or decoding. In this disclosure, image encoding may be used interchangeably with video encoding.

[0051] In this disclosure, video can refer to a collection of images over time. An image typically refers to a unit representing a single image at a particular time, and a slice / tile refers to a unit that forms part of an image in encoding. A slice / tile may include one or more coding tree units (CTUs). A single image may include one or more slices / tiles. A tile may represent a rectangular area of ​​a CTU within a specific tile row and a specific tile column of an image.

[0052] A single image can be divided into two or more sub-images. A sub-image can be a rectangular region of one or more slices of the image.

[0053] A pixel or cell can refer to the smallest unit that makes up a picture (or image). A "sample" can be used as the term corresponding to a pixel. A sample can generally represent a pixel or pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0054] A unit can represent the basic unit of image processing. A unit may include a specific region of an image and at least one piece of information associated with that region. A single unit may include a luminance block and two chrominance (e.g., Cb and Cr) blocks. The term "unit" may be used interchangeably with terms such as "block" or "region" in some cases. Typically, an M×N block may include a set (or array) of samples or transform coefficients in M ​​columns and N rows.

[0055] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Furthermore, the same reference numerals may be used throughout the drawings to indicate the same elements, and redundant descriptions of the same elements may be omitted.

[0056] Figure 1 Examples of video / image coding systems to which embodiments of the present disclosure may be applied are illustrated schematically.

[0057] refer to Figure 1 A video / image encoding system may include a first device (encoding device) and a second device (decoding device). The first device may deliver encoded video / image information or data to the second device in the form of a file or stream via a digital storage medium or network.

[0058] A video / image encoding system may also include a video / image acquisition device and a video / image renderer. The video / image acquisition device may be included in the encoding device, or it may be configured as a separate device or an external component. The video / image renderer may be included in the decoding device, or it may be configured as a separate device or an external component.

[0059] The first device may include a transmitter as an internal component, or a transmitter as a separate device or an external component.

[0060] The second device may include a receiver as an internal component, or as a receiver as a separate device or an external component.

[0061] An encoding device can be called an encoder, and a decoding device can be called a decoder. A transmitter can be included in the encoding device. A receiver can be included in the decoding device. A renderer can include a display, and the display can be configured as a separate device or an external component.

[0062] The decoding and encoding devices applied in one or more embodiments of this disclosure can be included in multimedia broadcasting transmitters / receivers, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video conferencing devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, over-the-air (OTT) video devices, internet streaming service providers, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, video telephony devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicle terminals), aircraft terminals, and ship terminals), and medical video devices, and can be used to process video signals or data signals. For example, over-the-air (OTT) video devices can include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0063] A video / image acquisition device can acquire video / image sources. The video / image acquisition device can acquire video / images through processes of capturing, compositing, or generating video / images. The video / image acquisition device may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras and a video / image archive including previously captured video / images. The video / image generation device may include, for example, a camera, a computer, a tablet PC, and a smartphone, and can (electronically) generate video / images. For example, virtual video / images can be generated by a computer, in which case the video / image capture process can be replaced by a process of generating relevant data. The video / image source can perform a video / image preprocessing process to input the optimized video / image into the encoder.

[0064] Encoding devices can encode input video / images. They can perform a series of processes, such as prediction, transformation, and quantization, to achieve compression and encoding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0065] The transmitter can send encoded images / image information or data, output as a bitstream, to a receiver of a receiving device via a digital storage medium or network, either as a file or a stream. The encoded images / image information or data output as a bitstream can be sent to the receiver via a streaming server. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiver can receive / extract the bitstream and send the received bitstream to a decoding device.

[0066] A streaming server can temporarily store bitstreams during the sending or receiving of bitstreams. Based on user requests via a web server, the streaming server sends multimedia data to the user's device, with the web server acting as an intermediary to notify the user of available services. When a user requests a desired service from the web server, the web server forwards the request to the streaming server, and the streaming server sends the multimedia data to the user. The content streaming system may include a separate control server, in which case the control server manages the commands / responses between devices within the content streaming system.

[0067] A streaming server can receive content from media storage devices and / or encoding devices. For example, when receiving content from an encoding device, the content can be received in real time. In this case, the streaming server can store the bitstream for a certain period of time to provide a smooth streaming service.

[0068] Decoding devices can decode video / images by performing a series of processes such as dequantization, inverse transform, and prediction, which correspond to the operations of encoding devices.

[0069] The renderer can render the decoded video / images. The rendered video / images can then be displayed on the monitor.

[0070] Figure 2 This diagram schematically illustrates the configuration of a video / image encoding apparatus to which embodiments of the present disclosure may be applied. Hereinafter, the encoding apparatus may include image encoding apparatus and / or video encoding apparatus.

[0071] refer to Figure 2 The encoding device 200 may include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor and an intra-frame predictor. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image partitioner 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured as at least one hardware component (e.g., an encoder chipset or processor). The memory 270 may include a decoded picture buffer (DPB) or may be configured as a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0072] Image partitioner 210 can partition an input image (or picture or frame) input to encoding device 200 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). In this case, the coding unit can be recursively partitioned from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree-binary-tritree (QTBTTT) structure. For example, a single coding unit can be partitioned into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. In this case, for example, a quadtree structure can be applied first, and a binary tree structure and / or a ternary tree structure can be applied later. Alternatively, a binary tree structure can be applied first. The encoding process according to this disclosure can be performed based on the final coding unit that is no longer partitioned. In this case, based on the encoding efficiency according to the image characteristics, the maximum coding unit can be used as the final coding unit, or, if necessary, the coding unit can be recursively partitioned into deeper coding units, thereby using the coding unit with the optimal size as the final coding unit. Here, the encoding process may include prediction, transformation, and reconstruction processes, which will be described below. In another example, the processing unit may also include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can be split or partitioned from the aforementioned final encoding unit. The prediction unit may be a unit for sample prediction, and the transformation unit may be a unit for deriving the transform coefficients and / or a unit for deriving the residual signal from the transform coefficients.

[0073] The term "unit" may be used interchangeably with the terms "cell" or "region" depending on the context. Typically, an M×N block can represent an array of samples or transform coefficients arranged in M ​​rows and N columns. A sample can typically represent a pixel or pixel value, and may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component. "Sample" can be used as a term corresponding to a pixel or cell in a single picture (or image).

[0074] Encoding device 200 generates a residual signal (residual signal, residual block, or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the predictor from the input image signal (original block or original sample array), and the generated residual signal is sent to converter 232. In this case, as illustrated, the component in encoder 200 used to subtract the prediction signal (prediction block or prediction sample array) from the input image signal (original block or original sample array) can be referred to as subtractor 231. The predictor can perform prediction on the processing target block (hereinafter referred to as the current block) and can generate a prediction block including prediction samples for the current block. The predictor can determine whether intra-frame prediction or inter-frame prediction is applied based on the current block or CU. The predictor can generate various information about the prediction, such as prediction mode information, and can send the generated information to entropy encoder 240, as described below in the description of each prediction mode. The information about the prediction can be encoded by entropy encoder 240 and output as a bitstream.

[0075] An intra-frame predictor can refer to samples within the current image to predict the current block. The referenced samples can be located adjacent to (near) the current block, or, depending on the prediction mode, located far from the current block. In intra-frame prediction, prediction modes can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC modes and planar modes. Directional modes can include, for example, 33 or 65 directional prediction modes based on the granularity of the prediction direction. However, this example is for illustration only, and more or fewer directional prediction modes can be used depending on the configuration. The intra-frame predictor can determine the prediction mode to be applied to the current block based on the prediction modes applied to neighboring blocks.

[0076] Inter-frame predictors can derive predicted blocks for the current block based on reference blocks (reference sample arrays) specified by motion vectors on a reference image. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted based on the correlation between the motion information of neighboring blocks and the current block, on a block, sub-block, or sample basis. Motion information can include motion vectors and reference image indices. Motion information can also include inter-frame prediction direction (L0 prediction, L1 prediction, and bidirectional prediction) information. In inter-frame prediction, neighboring blocks can include spatially adjacent blocks existing within the current image and temporally adjacent blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally adjacent block can be the same or different. Temporally adjacent blocks can be referred to as collinear reference blocks or collinear CUs (colCUs), and the reference image including temporally adjacent blocks can also be referred to as collinear images (colPics). For example, the inter-frame predictor can configure a motion information candidate list based on neighboring blocks and can generate information indicating candidates for deriving the motion vectors and / or reference image indices for the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip and merge modes, the inter-frame predictor can use motion information about neighboring blocks as motion information about the current block. In skip mode, unlike merge mode, the residual signal may not be sent. In motion vector prediction (MVP) mode, the motion vector of the current block can be indicated by using the motion vectors of neighboring blocks as the motion vector predictor and signaling the motion vector difference.

[0077] Predictor 220 can generate prediction signals based on various prediction methods described below. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply intra-frame prediction and inter-frame prediction simultaneously, which can be referred to as combined inter-frame and intra-frame prediction (CIIP). Furthermore, the predictor can predict blocks based on an intra-block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or palette mode can be used, for example, for screen content coding (SCC). IBC essentially performs prediction within the current frame, but can be performed similarly to inter-frame prediction in that it derives a reference block within the current frame based on a block vector. That is, IBC can utilize at least one of the inter-frame prediction techniques described in this disclosure.

[0078] The predicted signal generated by predictor 220 can be used to generate a reconstructed signal or a residual signal. Transformer 232 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT).

[0079] Quantizer 233 can quantize the transform coefficients and send the quantized transform coefficients to entropy encoder 240, which can encode the quantized signal (information about the quantized transform coefficients) and output the encoded signal as a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 233 can rearrange the block-form quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order, and can generate information about the transform coefficients based on the one-dimensional vector form of the quantized transform coefficients. Entropy encoder 240 can perform various encoding methods, such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), and Context Adaptive Binary Arithmetic Coding (CABAC). Entropy encoder 240 can encode information necessary for video / image reconstruction (e.g., values ​​of syntax elements) other than the quantized transform coefficients, either together or separately. The encoded information (e.g., encoded video / image information) can be sent or stored as a bitstream on a Network Abstraction Layer (NAL) basis. The video / image information may also include information about various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Furthermore, the video / image information may also include general constraint information. In this disclosure, information and / or syntax elements transmitted / signed from the encoding device to the decoding device can be included in the video / image information. The video / image information can be encoded by the aforementioned encoding process and included in a bitstream. The bitstream can be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) and / or storage device (not shown) for transmitting and / or storing signals output from the entropy encoder 240 may be configured as an internal / external element of the encoding device 200, or the transmitter may be included in the entropy encoder 240.

[0080] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients using dequantizer 234 and inverse transform unit 235. Adder 250 can add the reconstructed residual signal to the prediction signal output from the predictor to generate a reconstructed signal (reconstructed image, reconstructed block, or array of reconstructed samples). When there is no residual for the processing target block, such as when a skip mode is applied, the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current image, or it can be used for inter-frame prediction of the next image after being filtered as follows.

[0081] Luminance mapping with chroma scaling (LMCS) can be applied in image encoding and / or reconstruction processes.

[0082] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be stored in memory 270, specifically in the DPB of memory 270. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, and bilateral filtering. Filter 260 can generate filtering-related information and can send the generated information to entropy encoder 240. The filtering-related information can be encoded by entropy encoder 240 and output as a bitstream.

[0083] The modified reconstructed image sent to memory 270 can be used as a reference image in the inter-frame predictor. When inter-frame prediction is applied via the modified reconstructed image, the encoding device can avoid prediction mismatch between the encoding device 200 and the decoding device, and can improve encoding efficiency.

[0084] The DPB of memory 270 can store modified reconstructed images for use as reference images in the inter-frame predictor. Memory 270 can store motion information about blocks in the current image from which motion information is derived (or encoded) and / or about blocks in the reconstructed image. The stored motion information can be sent to the inter-frame predictor to be used as motion information about spatially adjacent blocks or about temporally adjacent blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current image and can send these reconstructed samples to the intra-frame predictor.

[0085] Figure 3 This diagram schematically illustrates the configuration of a video / image decoding device to which embodiments of the present disclosure can be applied. Hereinafter, the decoding device may include an image decoding device and / or a video decoding device.

[0086] refer to Figure 3The decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor and an intra-frame predictor. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to embodiments, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 may be configured as a single hardware component (e.g., a decoder chipset or processor). The memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The hardware component may also include the memory 360 as an internal / external component.

[0087] When a bitstream including video / image information is input, the decoding device 300 can, based on the... Figure 2 The decoding device 300 reconstructs an image by processing video / image information in an encoding device. For example, the decoding device 300 can derive units / blocks based on information related to block partitions obtained from the bitstream. The decoding device 300 can perform decoding using processing units applied to the encoding device. Therefore, the processing unit used for decoding can be, for example, an encoding unit, and the encoding unit can be partitioned from encoding tree units or maximum encoding units according to a quadtree structure, binary tree structure, and / or ternary tree structure. One or more transform units can be derived from the encoding unit. The reconstructed image signal decoded and output by the decoding device 300 can be reproduced via a reproduction device.

[0088] Decoding device 300 can receive signals output from encoding device in the form of a bitstream, and the received signals can be decoded by entropy decoder 310. For example, entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). Furthermore, the video / image information may also include general constraint information. Decoding device can further decode the picture based on the information about the parameter sets and / or general constraint information. In this disclosure, the information and / or syntax elements that are signaled / received, as described below, can be decoded through a decoding process and can be obtained from the bitstream. For example, entropy decoder 310 can decode information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and can output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients for the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine a context model using information about the target syntax element and decoding information about adjacent and target blocks or information about symbols / bins decoded in previous stages, and generate symbols corresponding to the values ​​of each syntax element by predicting the occurrence probability of bins based on the determined context model and performing arithmetic decoding on the bins. Here, after determining the context model, the CABAC entropy decoding method can update the context model using information about the decoded symbols / bins for use in the context model of the next symbol / bin. Prediction-related information from the information decoded by the entropy decoder 310 can be provided to the predictor 330, and the residual values ​​(i.e., quantized transform coefficients and related parameter information) obtained by entropy decoding in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive residual signals (residual blocks, residual samples, or arrays of residual samples). Filtering-related information from the information decoded by the entropy decoder 310 can be provided to the filter 350. A receiver (not shown) for receiving signals output from the encoding device may be further configured as an internal / external element of the decoding device 300, or the receiver may be a component of the entropy decoder 310. The decoding device according to this disclosure may be referred to as a video / image / picture decoding device and may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, and a predictor 330.

[0089] Dequantizer 321 can dequantize the quantized transform coefficients to output transform coefficients. Dequantizer 321 can rearrange the quantized transform coefficients into two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device. Dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0090] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block or residual sample array).

[0091] The predictor can perform prediction for the current block and generate a prediction block that includes prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on prediction-related information output from the entropy decoder 310, and determine the specific intra-frame / inter-frame prediction mode.

[0092] Predictor 330 can generate a prediction signal based on various prediction methods described below. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply intra-frame prediction and inter-frame prediction simultaneously, which can be referred to as combined inter-frame and intra-frame prediction (CIIP). Furthermore, the predictor can predict blocks based on an intra-block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or palette mode can be used, for example, for screen content coding (SCC). IBC essentially performs prediction within the current frame, but can be performed similarly to inter-frame prediction in that it derives a reference block within the current frame based on a block vector. That is, IBC can utilize at least one of the inter-frame prediction techniques described in this disclosure.

[0093] An intra-frame predictor can refer to samples within the current image to predict the current block. The referenced samples can be located adjacent to (near) the current block, or, depending on the prediction mode, located far from the current block. In intra-frame prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. The intra-frame predictor can determine the prediction mode to be applied to the current block based on the prediction modes applied to neighboring blocks.

[0094] An inter-frame predictor can deduce a predicted block for the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. Here, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between the motion information of neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, and bidirectional prediction) information. In inter-frame prediction, neighboring blocks may include spatially adjacent blocks existing within the current image and temporally adjacent blocks existing in the reference image. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and can deduce the motion vector and / or reference image index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode for the current block.

[0095] Adder 340 can add the obtained residual signal to the prediction signal (prediction block or prediction sample array) output from predictor 330 to generate a reconstruction signal (reconstructed image, reconstruction block, or reconstruction sample array). When there is no residual for the processing target block, such as when a skip mode is applied, the prediction block can be used as a reconstruction block.

[0096] Adder 340 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current image, or it can be output after being filtered as described below, or used for inter-frame prediction of the next image.

[0097] Luminance mapping with chroma scaling (LMCS) can be applied in the image decoding process.

[0098] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be sent to memory 360, specifically to the DPB in memory 360. Various filtering methods can include, for example, unblocking filtering, sample adaptive shifting, adaptive loop filtering, and bilateral filtering.

[0099] The (modified) reconstructed image stored in the DPB of memory 360 can be used as a reference image in the inter-frame predictor. Memory 360 can store motion information about blocks in the current image from which motion information is derived (or decoded) and / or about blocks in the reconstructed image. The stored motion information can be sent to the inter-frame predictor to be used as motion information about spatially adjacent blocks or about temporally adjacent blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current image and can send the reconstructed samples to the intra-frame predictor.

[0100] The embodiments described in this specification for the filter 260 and predictor 210 of the encoding device 200 can also be applied equally or correspondingly to the filter 350 and predictor 330 of the decoding device 300.

[0101] As described above, during video encoding, prediction is performed to increase compression efficiency. Prediction generates prediction blocks that include prediction samples for the current block, which is the target block for encoding. Prediction blocks include prediction samples in the spatial domain (or pixel domain). Prediction blocks are derived identically in both the encoding and decoding devices, and the encoding device can enhance image coding efficiency by signaling information about the residuals between the original block and the prediction blocks (residual information) to the decoding device, rather than the original sample values ​​of the original block. The decoding device can derive residual blocks including residual samples based on the residual information, generate reconstructed blocks including reconstructed samples by combining the residual blocks with the prediction blocks, and generate a reconstructed image including the reconstructed blocks.

[0102] Residual information can be generated through transformation and quantization processes. For example, an encoding device can derive a residual block between the original block and the prediction block, perform a transformation process on the residual samples (residual sample array) included in the residual block to derive transform coefficients, perform a quantization process on the transform coefficients to derive quantized transform coefficients, and signal the relevant residual information to a decoding device (via a bitstream). Residual information may include information such as the values ​​and locations of the quantized transform coefficients, the transform technique, the transform kernel, and the quantization parameters. The decoding device can perform dequantization / inverse transform processes based on the residual information to derive residual samples (or residual blocks). The decoding device can generate a reconstructed image based on the prediction block and the residual block. The encoding device can perform dequantization / inverse transform on the quantized transform coefficients to derive residual blocks used for reference in inter-frame prediction of subsequent images, and can generate a reconstructed image based on the residual blocks.

[0103] In this disclosure, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. When quantization / dequantization is omitted, the quantized transform coefficients may be referred to as transform coefficients. When transform / inverse transform is omitted, the transform coefficients may be referred to as coefficients or residual coefficients, or, for the sake of consistency, may still be referred to as transform coefficients.

[0104] Furthermore, in this disclosure, quantized transform coefficients and transform coefficients can be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, residual information may include information about one or more transform coefficients, and the information about one or more transform coefficients may be signaled via residual coding syntax. Transform coefficients can be derived based on residual information (or information about one or more transform coefficients), and scaled transform coefficients can be derived by inverse transforming (scaling) the transform coefficients. Residual samples can be derived based on inverse transforming (scaling) the scaled transform coefficients. These details can be equally applied to other parts of this disclosure or described in other parts of this disclosure.

[0105] Intra-frame prediction can refer to generating a prediction sample for the current block based on reference samples within the image to which the current block belongs (hereinafter, the current image). When intra-frame prediction is applied to the current block, neighboring reference samples to be used for intra-frame prediction of the current block can be derived. The neighboring reference samples for the current block can include H+W samples located to the left of the W×H current block, W+H samples located above the current block, and at least one sample adjacent to the upper left of the current block. Alternatively, the neighboring reference samples for the current block can include multiple rows of upper adjacent samples and multiple columns of left adjacent samples.

[0106] Some neighboring reference samples for the current block may not have been decoded or may be unavailable. In this case, the decoder can configure the neighboring reference samples to be used for prediction by filling in or replacing unavailable samples with available samples.

[0107] When deriving neighboring reference samples, the predicted samples for the current block can be derived based on the neighboring reference samples and intra-prediction mode / type information. Here, the intra-prediction mode can indicate either a directional prediction mode or a non-directional prediction mode that represents the spatial correlation of intra-prediction. The directional prediction mode can be referred to as the angular prediction mode, and the non-directional prediction mode can be referred to as the non-angular prediction mode. The intra-prediction type can indicate various prediction types used to perform intra-prediction. Intra-prediction types can include, for example, multiple reference lines (MRL), intra-segmentation (ISP), position-dependent intra-prediction (PDPC), matrix-weighted intra-prediction or matrix-based intra-prediction (MIP), cross-component linear model (CCLM), multi-model linear model (MMLM), decoder-side intra-mode derivation (DIMD), fusion of chroma intra-prediction modes, intra-template matching, fusion for template-based intra-mode derivation (TIMD), intra-prediction fusion, cross-component convolutional model (CCCM), cross-component prediction (CCP), and spatial geometric partitioning mode (SGPM). Depending on the circumstances, intra-prediction mode and / or intra-prediction type can be used to perform intra-prediction.

[0108] Specifically, the intra-frame prediction process may include intra-frame prediction mode / type determination, reference sample derivation, and prediction sample derivation based on the intra-frame prediction mode / type. Furthermore, post-filtering operations can be performed on the derived prediction samples as needed.

[0109] Residual processing will be described below. Residual processing can be performed in both encoding and decoding devices. Residual processing may include coefficient encoding, transformation, and / or quantization processes.

[0110] The residual processing on the encoding side may include the process of generating and / or encoding residual information from residual samples for the derived current block. The residual processing may also include the process of deriving residual samples based on predicted samples. The residual processing on the decoding side may include the process of deriving residual samples from residual information in the received bitstream. For example, the residual processing may include (inverse) transform and / or (inverse) quantization processes. Furthermore, the residual processing may include encoding / decoding processes for residual information. The residual information may include residual data and / or transform / quantization related parameters.

[0111] Specifically, for residual processing, methods can be used to derive (quantized) transform coefficients in the derivation block and generate and encode residual information based on the derived (quantized) transform coefficients, as well as methods to derive (quantized) residual coefficients in the block with applied transform skip and generate and encode residual information for transform skip based on the derived (quantized) residual coefficients. The encoded information can be output as a bitstream as described above.

[0112] Furthermore, on the decoding side, the (quantized) transform coefficients or (quantized) residual coefficients in the block can be derived from the residual information included in the bitstream (or the residual information for transform skipping), and the residual samples can be derived by performing inverse quantization / inverse transform (if necessary).

[0113] The encoding process based on residual processing can, for example, include the following:

[0114] Figure 4 An example of a video / image coding method based on residual processing is shown.

[0115] refer to Figure 4 S400 can be executed by the predictor of the encoding device, S410 by the residual processor of the encoding device, and S420 by the entropy encoder of the encoding device. Specifically, prediction-related information can be derived by the predictor and encoded by the entropy encoder. Residual information can be derived by the residual processor and encoded by the entropy encoder. Residual information is information about residual samples. Residual information may include information about the quantized transform coefficients for the residual samples. As described above, residual samples can be derived as transform coefficients by the transformer of the encoding device, and transform coefficients can be derived as quantized transform coefficients by the quantizer. Information about the quantized transform coefficients can be encoded in the entropy encoder through the residual encoding process.

[0116] The encoding device derives the prediction sample for the current block (S400). The encoding device can derive the prediction sample for the current block based on the above inter-frame prediction and / or intra-frame prediction.

[0117] The encoding device can perform residual processing (S410) based on the predicted samples. The encoding device can derive residual samples based on the predicted samples. The encoding device can derive residual samples by comparing the original samples of the current block with the predicted samples. As described above, residual processing includes transformation and / or quantization processes for the residual samples. The encoding device can generate residual information from the residual samples through residual processing. The residual information can include information about the quantized transformation coefficients as described above.

[0118] The encoding device encodes image information, including prediction-related information and / or residual information (S420). The encoding device may output the encoded image information as a bitstream. The prediction-related information may include information related to the prediction process. The residual information is information about the residual samples. The residual information may include information about the quantized transform coefficients for the residual samples.

[0119] The output bitstream can be stored in (digital) storage media and sent to the decoding device, or it can be sent to the decoding device via a network.

[0120] Furthermore, as mentioned above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks) based on reference samples and residual samples. This is to derive the same prediction result in the encoding device as the prediction result performed in the decoding device, thereby improving encoding efficiency. Therefore, the encoding device can store the reconstructed image (or reconstructed samples, reconstructed blocks) in memory and use it as a reference image for inter-frame prediction. As mentioned above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0121] Decoding devices can perform operations corresponding to those performed in encoding devices. A video / image decoding process based on residual processing may include, for example, the following.

[0122] Figure 5 An example of a video / image decoding method based on residual processing is shown.

[0123] refer to Figure 5 S500 can be executed by the entropy decoder of the decoding device, S510 can be executed by the predictor of the decoding device, S520 can be executed by the residual processor of the decoding device, and S530 can be executed by the adder or reconstructor of the decoding device.

[0124] Specifically, the decoding device obtains image / video information from the bitstream (S500). The image / video information may include prediction-related information and / or residual information.

[0125] The decoding device performs prediction (including inter-frame prediction and / or intra-frame prediction) based on prediction-related information (S510). The decoding device can deduce the prediction mode / type for the current block based on the prediction-related information and generate prediction samples within the current block based on the prediction mode / type. In this case, the decoding device can perform a prediction sample filtering process. Prediction sample filtering can be referred to as post-filtering. The prediction samples can be partially or completely filtered through the prediction sample filtering process. Depending on the circumstances, the prediction sample filtering process can be omitted.

[0126] The decoding device performs residual processing based on the residual information (S520). The decoding device can derive residual samples for the current block based on the residual information. Specifically, the dequantizer of the residual processor can perform dequantization based on the quantized transform coefficients derived from the residual information to derive the transform coefficients, and the inverse transformer of the residual processor can perform inverse transform on the transform coefficients to derive residual samples for the current block.

[0127] The decoding device generates a reconstructed block / image (S530). The decoding device can generate reconstructed samples for the current block based on predicted samples and / or residual samples, and derive a reconstructed block including the reconstructed samples. A reconstructed image for the current image can be generated based on the reconstructed block. As described above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0128] The residual information can be encoded / decoded using the binarization and encoding methods described in this disclosure. For example, the residual information can be binarized using fixed-length binarization, truncated Ricean binarization, truncated univariate binarization, etc. For example, the residual information can be encoded / decoded using entropy coding (e.g., CABAC, CAVLC) or bypass coding.

[0129] The maximum transform size and zeroing of the transform coefficients will be described below.

[0130] For example, the CTU size and maximum transform size (for all MTS cores) can be extended to 256. In this case, the maximum size of the intra-prediction block can be set to 128×128. In UHD sequences, the maximum CTU size can be set to 256, otherwise it can be set to 128.

[0131] During the primary transform, the transform coefficients may not need to be zeroed out. However, in the secondary transform using LFNST, the primary transform coefficients outside the ROI where LFNST is applied can be zeroed out.

[0132] Furthermore, zeroing of the transform coefficients can be applied during the main transform. As the transform block size increases, zeroing may be necessary after the main transform, and this can be determined based on whether the MTS (Multi-Target Transform) is applied. For example, when the MTS is applied, zeroing can be performed at any of 16, 20, 24, 32, or 40, and when the MTS is not applied, zeroing can be performed at a preset specific value. Specific values ​​could be 32 or 64. The number of transform coefficients to be zeroed when the MTS is applied can vary depending on the transform block size. For example, if the width or height of the transform block to which the MTS is applied is 4, 8, or 16 (i.e., 4-point transform, 8-point transform, or 16-point transform), zeroing is not applied, and if the width or height of the transform block to which the MTS is applied is 32 (i.e., 32-point transform), zeroing can be performed at 16. Alternatively, when the MTS is applied to a width or height greater than 32, zeroing can be performed at 32.

[0133] Furthermore, residual processing includes transform / inverse transform and / or quantization / inverse quantization processes. According to this disclosure, a master transform and / or a quadratic transform can be applied to the residual block to derive the transform coefficient block (transform coefficients), and an inverse quadratic transform and / or an inverse master transform can be applied to the transform coefficient block (transform coefficients) to derive the residual block.

[0134] As described above, the transformation used for the residuals can be performed via a primary transformation and / or a secondary transformation optionally performed after the primary transformation. The primary transformation can be referred to as the master transformation and can be a DCT (Discrete Cosine Transform) and a DST (Discrete Sine Transform) applied to all rows and all columns of the residual block. After the primary transformation, a secondary transformation can be additionally applied to specific transform coefficients at the top left of the transform block, based on the result of the primary transformation. In the inverse transformation performed during the decoding step, an inverse secondary transformation can be applied to specific residuals (or transform coefficients) at the top left of the residual block corresponding to the (dequantized) residual information, and an inverse primary transformation can be applied to the transform block based on the result of the inverse secondary transformation.

[0135] Furthermore, as will be described later, the NSPT transform can be used as appropriate. The NSPT transform can be a transform that integrates and replaces the main transform and the secondary transform.

[0136] The following text will describe multiple transformation selection (MTS) as a transformation method.

[0137] Figure 6 Multiple transformation techniques according to this disclosure are illustrated by way of example.

[0138] refer to Figure 6 The converter can correspond to the one described above. Figure 2 The converter in the encoding device, and the inverse converter can correspond to the one described above. Figure 2 Inverse converter in encoding devices or Figure 3 The inverse converter in the decoding device.

[0139] The transformer can perform a master transform based on residual samples (residual sample array) in the residual block to derive the (master) transform coefficients (S600). This master transform can be referred to as the core transform. Here, the master transform can be based on multiple transform selection (MTS), and when multiple transforms are applied to the master transform, they can be referred to as multiple core transforms.

[0140] The converter can perform a secondary transformation based on the (primary) transform coefficients to derive modified (secondary) transform coefficients (S610). The primary transform can refer to a transformation from the spatial domain to the frequency domain, and the secondary transform can refer to a transformation into a more compact representation by utilizing the correlations present in the (primary) transform coefficients.

[0141] For example, a quadratic transform can include an inseparable transform. In this case, the quadratic transform can be called an inseparable quadratic transform (NSST) or a mode-dependent inseparable quadratic transform (MDNSST). An inseparable quadratic transform can refer to a transform based on an inseparable transform matrix, performing a quadratic transform on the (master) transform coefficients derived through the master transform to generate modified transform coefficients (or quadratic transform coefficients) for the residual signal. Here, instead of applying the vertical and horizontal transforms separately (or independently to the (master) transform coefficients), the transform can be applied once based on the inseparable transform matrix.

[0142] In other words, an inseparable quadratic transform can represent a transform method that, without separating the vertical and horizontal components of the (master) transform coefficients, rearranges a two-dimensional signal (transform coefficients) into a one-dimensional signal in a specific predetermined direction, and then generates modified transform coefficients (or quadratic transform coefficients) based on an inseparable transform matrix. In other words, it can represent a transform method that rearranges the signal into a one-dimensional signal in a row-first or column-first direction and then generates modified transform coefficients (or quadratic transform coefficients) based on an inseparable transform matrix.

[0143] Furthermore, the inverse transformer can perform a series of processes in the reverse order of the process performed by the transformer described above. The inverse transformer can receive (dequantized) transform coefficients, derive (primary) transform coefficients by performing a second (inverse) transform (S620), and obtain residual blocks (residual samples) by performing a primary (inverse) transform on the (primary) transform coefficients (S630). Here, the primary transform coefficients can be referred to as modified transform coefficients on the inverse transformer side. As described above, the encoding and / or decoding devices can generate reconstructed blocks based on the residual blocks and prediction blocks, and can generate reconstructed images based on these blocks.

[0144] In this disclosure, the master transform may be referred to as the core transform. Here, the master transform may be based on multiple transform selection (MTS), and when a transform kernel selected from multiple transform kernel types is applied as the master transform, it may be referred to as a multi-core transform.

[0145] Multi-core transform refers to a scheme that uses DCT (Discrete Cosine Transform) 2, DST (Discrete Sine Transform) 7, DCT 8, etc., to perform the transform. In other words, multi-core transform can refer to a transform method that uses multiple transform cores selected from DCT 2, DST 7, DCT 8, and DST 1 to transform the spatial domain residual signal (or residual block) into frequency domain transform coefficients (or master transform coefficients). Here, from the perspective of the transformer, the master transform coefficients can be referred to as temporary transform coefficients.

[0146] In other words, when applying conventional transform methods, transform coefficients can be generated by applying a spatial-to-frequency domain transform to the residual signal (or residual block) based on DCT-2. Conversely, when applying multi-core transforms, transform coefficients (or master transform coefficients) can be generated by applying a spatial-to-frequency domain transform to the residual signal (or residual block) based on DCT-2, DST-7, DCT-8, and / or DST-1. Here, DCT-2, DST-7, DCT-8, and DST-1 can be referred to as transform types, transform kernels, or transform cores. These DCT / DST transform types can be defined based on basis functions.

[0147] When performing multi-core transformation, a vertical transformation kernel and a horizontal transformation kernel can be selected from the transformation kernels for the target block (the block undergoing transformation). A vertical transformation for the target block can be performed based on the vertical transformation kernel, and a horizontal transformation for the target block can be performed based on the horizontal transformation kernel. Here, the horizontal transformation can represent the transformation of the horizontal components of the target block, and the vertical transformation can represent the transformation of the vertical components of the target block. The vertical / horizontal transformation kernel can be adaptively determined based on the prediction mode and / or the transformation index of the target block (CU or sub-block) including the residual block.

[0148] Furthermore, according to the example, when performing the master transform by applying the MTS, a mapping relationship for the transform kernel can be established by setting specific basis functions to predetermined values ​​and combining which basis functions are applied in the vertical or horizontal transform. For example, when the horizontal transform kernel is represented as trTypeHor and the vertical transform kernel is represented as trTypeVer, trTypeHor or trTypeVer value 0 can be set to DCT2, trTypeHor or trTypeVer value 1 can be set to DST7, and trTypeHor or trTypeVer value 2 can be set to DCT8.

[0149] In this case, the MTS index information can be encoded and sent to the decoding device to indicate any one of the multiple transform core sets. For example, MTS index 0 can indicate that both trTypeHor and trTypeVer values ​​are 0, MTS index 1 can indicate that both trTypeHor and trTypeVer values ​​are 1, MTS index 2 can indicate that trTypeHor is 2 and trTypeVer is 1, MTS index 3 can indicate that trTypeHor is 1 and trTypeVer is 2, and MTS index 4 can indicate that both trTypeHor and trTypeVer are 2.

[0150] Such an MTS can be applied explicitly via signaling as described above, or implicitly based on specific conditions. In an implicit MTS, the transform kernel can be derived independently for each direction based on the width or height of the transform block. For example, when applying a sub-block transform that applies the transform to only one sub-block, an implicit MTS can be applied if the width or height meets specific conditions, or conditions for a specific intra-frame mode. For instance, an implicit MTS can be applied to blocks that apply SBT and where the larger value between the width and height is 32 or less, or 64 or less. Alternatively, an implicit MTS can be applied to blocks that apply ISP, or to blocks that apply intra-frame prediction but do not apply LFNST and MIP.

[0151] Furthermore, implicit MTS can infer the primary transform pair using a LUT based on the intra-prediction mode of the current block and the size of the TU. For example, the same intra-prediction mode as explicit MTS can be considered, and the maximum TU size can be 32×32. In this case, the LUT can include transform pairs based on separable primary transforms, and no additional primary transforms need to be added. For example, DCT-2, DCT-5, DCT-8, DST-1, DST-4, and DST-7 can be used.

[0152] For example, in the case of an ISP block, the sizeIdx used as input to the LUT can be based on the position of the current ISP sub-partition within the ISP block. Furthermore, as an example, implicit MTS can be applied only to ISP blocks depending on the CTC configuration settings. As another example, implicit MTS can be applied to angular intra-prediction modes. For example, implicit MTS for ISP blocks can be used by using implicit signaling for all modes (non-CTC). For example, the default implicit MTS (i.e., based on the shape of the TU) can be maintained in TIMD and DIMD modes. Additionally, DCT-2 can be used in MIP, EIP, SGPM, and IntraTMP modes. Furthermore, the default implicit MTS can be used for ISP blocks.

[0153] The following text will describe the Low Frequency Inseparable Transform (LFNST) as an example of a transformation method.

[0154] Furthermore, the transformer of the encoding device can derive modified (secondary) transform coefficients by performing a secondary transform based on the (primary) transform coefficients. Here, the primary transform refers to the transform from the spatial domain to the frequency domain, and the secondary transform refers to transforming into a more compact representation by utilizing the correlation between the (primary) transform coefficients. The secondary transform can include an inseparable transform. In this case, the secondary transform can be called an inseparable secondary transform (NSST). An inseparable secondary transform can represent a transform that generates modified transform coefficients (or secondary transform coefficients) for the residual signal by performing a secondary transform based on the (primary) transform coefficients of the inseparable transform matrix derived via the primary transform. Here, based on the inseparable transform matrix, the transform can be applied simultaneously to the (primary) transform coefficients without applying the vertical and horizontal transforms separately (or without applying the horizontal and vertical transforms independently). In other words, an inseparable quadratic transform can be represented as follows: without separating the vertical and horizontal components of the (master) transform coefficients, a two-dimensional signal (transform coefficients) is rearranged into a one-dimensional signal in a specific predetermined direction (e.g., row-first or column-first direction), and then modified transform coefficients (or quadratic transform coefficients) are generated based on the inseparable transform matrix. For example, the row-first order is arranged in a straight line in the order of the first row, second row, ..., and Mth row of an M×N block, and the column-first order is arranged in a straight line in the order of the first column, second column, ..., and Nth column of an M×N block. The inseparable quadratic transform can be applied to the upper left region of a block consisting of (master) transform coefficients (hereinafter referred to as a transform coefficient block). For the inseparable quadratic transform, the transform kernel (or transform type) can be selected based on mode relevance. Here, the mode can include intra-frame prediction mode and / or inter-frame prediction mode.

[0155] Inseparable quadratic transformations can be performed using either an 8×8 transformation or a 4×4 transformation, determined based on the width (W) and height (H) of the transform coefficient block. An 8×8 transformation applies to the 8×8 region within the corresponding transform coefficient block when both W and H are equal to or greater than 8, and this 8×8 region can be the top-left 8×8 region within the corresponding transform coefficient block. Similarly, a 4×4 transformation applies to the 4×4 region within the corresponding transform coefficient block when both W and H are equal to or greater than 4, and this 4×4 region can be the top-left 4×4 region within the corresponding transform coefficient block. For example, the 8×8 transform kernel matrix can be a 64×64 / 16×64 matrix, and the 4×4 transform kernel matrix can be a 16×16 / 8×16 matrix.

[0156] Furthermore, for mode-dependent transform kernel selection, transform sets can be configured for inseparable quadratic transforms, where each transform set consists of k inseparable quadratic transform kernels. For example, there can be 4 or 35 transform sets. A specific set can be selected from the transform sets based on, for example, the intra-prediction mode of the target block (such as a CU or sub-block).

[0157] For example, when a specific set is determined to be used for an inseparable quadratic transform, a transform kernel can be selected from k transform kernels within that specific set using an inseparable quadratic transform index. The encoding device can derive the inseparable quadratic transform index indicating the specific transform kernel based on rate distortion (RD) checks, and can signal this index to the decoding device. The decoding device can then select a transform kernel from k transform kernels within the specific set based on the inseparable quadratic transform index.

[0158] The inverse transformers of encoding and decoding devices can perform a series of processes in the reverse order of those performed by the aforementioned transformers. The inverse transformer can receive (dequantized) transform coefficients and perform a second (inverse) transform to derive the (primary) transform coefficients. From the perspective of the inverse transformer, the primary transform coefficients can be referred to as the modified transform coefficients.

[0159] In this disclosure, to reduce the computational and memory requirements involved in the inseparable quadratic transform, a reduced quadratic transform (RST) that reduces the size of the transform matrix (kernel) can be applied within the concept of NSST. RST can be referred to by various terms such as reduced transform, reduced quadratic transform, reduction transform, simplified transform, or simple transform, and the name of RST can be limited to the examples listed. Alternatively, since RST is primarily performed in the low-frequency region of the transform block, including non-zero coefficients, RST can also be called low-frequency inseparable transform (LFNST).

[0160] In LFNST, the reduced transformation matrix can be determined by mapping an N-dimensional vector to an R-dimensional vector in another space, where R is less than N. N can represent the square of the length of one side of the block to which the transformation is applied or the total number of transformation coefficients corresponding to the block to which the transformation is applied, and the reduction factor can be represented by the value of R / N.

[0161] According to the example, the size of the LFNST matrix is ​​R×N, which is smaller than the size of the regular transformation matrix N×N, and can be defined as shown in Equation 1 below.

[0162] [Formula 1]

[0163] The transformation coefficients for the target block can be derived by multiplying the LFNST matrix TR×N by the residual samples for the target block. When the size of the block to which the transformation is applied is 8×8 and R=16 (i.e., R / N=16 / 64=1 / 4), the LFNST can be represented by the matrix operations in Equation 2 below.

[0164] [Equation 2]

[0165] In Equation 2, r1 to r64 can represent the residual samples for the target block, and more specifically, can be the transform coefficients generated by applying the master transform. As a result of Equation 2, the transform coefficients ci for the target block can be derived. With respect to ci, c1 to cR can be derived. That is, when R=16, the transform coefficients c1 to c16 for the target block can be derived.

[0166] If a conventional transform is applied instead of LFNST to multiply a 64×64 (N×N) transform matrix by a 64×1 (N×1) residual sample, 64 (N) transform coefficients for the target block are derived; however, due to the application of LFNST, only 16 (R) transform coefficients for the target block are derived. Since the total number of transform coefficients for the target block is reduced from N to R, the amount of data sent from the encoding device to the decoding device is reduced, thus improving the transmission efficiency between the encoding and decoding devices.

[0167] Inverse LFNST matrix T N×R The size is N×R, which is smaller than the size of the traditional inverse transformation matrix N×N, and is similar to the LFNST matrix T shown in Equation 1. R×N They are transposes. Tt can refer to the inverse LFNST matrix T. R×N T (The superscript T indicates transpose). When the inverse LFNST matrix T is... R×N T Multiplying by the transform coefficients for the target block allows for the derivation of the modified transform coefficients for the target block or the residual samples for the target block. The inverse LFNST matrix T R×N T It can also be represented as (T) R×N ) T N×R .

[0168] According to the example, when the size of the block to which the inverse transformation is applied is 8×8 and R=16 (i.e., when R / N=16 / 64=1 / 4), the inverse LFNST can be represented by the matrix operation in Equation 3 below.

[0169] [Formula 3]

[0170] c1 to c16 can represent the transformation coefficients for the target block. As a result of Equation 3, rj can be derived, representing the modified transformation coefficients for the target block or the residual samples for the target block. Regarding rj, r1 to rN can be derived. That is, when N=64 (or when R=16), r1 to r64 for the target block can be derived.

[0171] Figure 7 This is a diagram illustrating an example of LFNST.

[0172] refer to Figure 7 4×4 LFNST can be applied to blocks with min(width, height) < 8, and 8×8 LFNST can be applied to blocks with min(width, height) > 4.

[0173] For example, 16 (primary) transform coefficients can be input for a 4×4 forward LFNST, and 64 (primary) transform coefficients can be input for an 8×8 forward LFNST. Since 8 or 16 transform coefficients can be derived from the forward LFNST respectively, 8 or 16 transform coefficients can be input for the inverse LFNST respectively. When performing the 4×4 inverse LFNST, 16 modified transform coefficients can be output from 8 transform coefficients, and when performing the 8×8 inverse LFNST, 64 modified transform coefficients can be output from 16 transform coefficients.

[0174] Furthermore, according to the example, in the transformation of the encoding process, a maximum 16×48 transform kernel matrix can be applied instead of using a 16×64 transform kernel matrix by selecting only 48 data points out of the 64 data points constituting the 8×8 region. Here, the term "maximum" means that for an m×48 transform kernel matrix that can generate m coefficients, the maximum value of m is 16. That is, when performing LFNST by applying an m×48 transform kernel matrix (m≤16) to an 8×8 region, m coefficients can be generated by receiving 48 data points as input. When m is 16, 16 coefficients are generated by receiving 48 data points as input. In other words, assuming that the 48 data points form a 48×1 vector, a 16×1 vector can be generated by sequentially multiplying the 16×48 matrix with the 48×1 vector. At this point, the 48×1 vector can be configured by appropriately arranging the 48 data points constituting the 8×8 region. At this point, when matrix operations are performed by applying the maximum 16×48 transformation kernel matrix, 16 modified transformation coefficients are generated. These 16 modified transformation coefficients can be arranged in the upper left 4×4 region according to the scan order, and the upper right 4×4 region and the lower left 4×4 region can be filled with 0.

[0175] In the inverse transform of the decoding process, the transpose of the aforementioned transform kernel matrix can be used. That is, when the inverse LFNST is performed as the inverse transform process executed in the decoding device, the input coefficient data to be applied to the inverse LFNST is configured as a 1D vector according to a predetermined arrangement order, and the modified coefficient vector obtained by multiplying the 1D vector by the corresponding inverse LFNST matrix from the left can be arranged in a 2D block according to a predetermined arrangement order. In this case, matrix operations can be performed using a (48×16 matrix). The transformation coefficient vector (16×1) = (48×1 modified transformation coefficient vector) is represented as follows. Here, since an n×1 vector can be interpreted as having the same meaning as an n×1 matrix, it can also be represented as an n×1 column vector. This represents a matrix multiplication operation. When performing this matrix operation, 48 modified transformation coefficients can be derived, and these 48 modified transformation coefficients can be arranged in the upper left, upper right, and lower left regions of an 8×8 region, excluding the lower right region.

[0176] Furthermore, LFNST can be applied to sub-blocks. For example, when a coded block is partitioned into sub-blocks, the size of the sub-block can be "width / 2 × height / 2" or "width / 4 × height / 4". In this case, sub-blocks can include corner sub-blocks or center sub-blocks. For example, LFNST can be applied to corner sub-blocks. In this case, similar to regular SBT (Sub-Block Transform), signals can be used to indicate the partitioning pattern and position of non-zero sub-blocks.

[0177] The following section will describe the enhanced MTS (Multi-Transform Selection) for intra-frame prediction.

[0178] For example, when MTS is applied to intra-prediction blocks, in addition to DCT 2, DST 7, and DCT 8, DCT 5 and DST 4 can also be used as transform kernels. Furthermore, when MTS is applied to intra-prediction blocks, in addition to DCT 2, DST 7, and DCT 8, DCT 5, DST 4, DST 1, and the Identity Transform (IDT) can also be used as transform kernels.

[0179] Furthermore, intra-MTS candidates, i.e., the MTS set, can be derived based on TU size and intra-prediction mode information. For example, a total of 16 TU sizes can be applied, and individual TU sizes can be classified into 5 groups (categories) according to the intra-prediction mode. In this case, if the 5 groups are applied to each of the 16 TU sizes, a total of 80 groups can be considered; however, since the transform set can be shared, a number less than 80 can be considered, for example, 58 groups. Moreover, the number of groups is not limited to 58 groups, and a positive integer N groups can be considered.

[0180] Furthermore, when the intra-prediction mode is a directional mode, the symmetry of the TU shape and intra-prediction direction can be considered. For example, mode i (i>34) with A×B and mode j (j=68-i) with B×A can be mapped to the same group, and in this case, the transform pairs for the vertical and horizontal kernels can be swapped. For example, a 16×4 block with intra-prediction mode number 18 and a 4×16 block with intra-prediction mode number 50 can be mapped to the same group. When the intra-prediction mode is a wide-angle intra-prediction mode, the closest regular directional mode can be used to determine the transform set. For example, to derive the group used to determine the transform set, wide-angle intra-prediction modes between -2 and -14 can use mode 2, and wide-angle intra-prediction modes between 67 and 80 can use mode 66.

[0181] Furthermore, multiple transform candidates (transform pairs) can be applied to each individual group. For example, one, four, or six transform sets can be applied. The number of transform sets applied can be determined based on the position of the last transform coefficient or based on the absolute value of the transform coefficient. For example, the position of the last transform coefficient can be compared with two thresholds, and the application of one, four, or six transform sets can be selected based on the comparison result. Alternatively, the sum of the absolute values ​​of the transform coefficients can be compared with two thresholds, and the application of one, four, or six transform sets can be selected.

[0182] Furthermore, the number of transformation sets can be determined based on the sum of the absolute values ​​of the transformation coefficients. For example, when the sum of the absolute values ​​of the transformation coefficients is less than th0, one transformation set can be applied; when the sum of the absolute values ​​of the transformation coefficients is greater than th0 and equal to or less than th1, four transformation sets can be applied; and when the sum of the absolute values ​​of the transformation coefficients is greater than th1, six transformation sets can be applied (1 candidate: sum <= th0, 4 candidates: th0 < sum <= th1, 6 candidates: sum > th1). Here, the sum can represent the sum of the absolute values ​​of the transformation coefficients. Furthermore, th0 can be set to 6, and th1 can be set to 32.

[0183] Figure 8 An example of deriving the MTS set is shown.

[0184] refer to Figure 8 Four transform sets can be determined based on TU size and intra-frame prediction mode information.

[0185] For example, using a total of 16 TU sizes, a total of 36 intra-prediction modes can be applied, including intra-prediction modes 0 to 34 and additional MIP (matrix-based intra-prediction) modes. As described above, the intra-prediction modes of the transform block can be mapped to any of the 1 to 34 intra-prediction modes based on the intra-prediction mode information. A total of 5 groups can be mapped for each individual TU size. For example, intra-prediction modes 0 and 1, intra-prediction modes 2 to 12, intra-prediction modes 13 to 23, intra-prediction modes 24 to 34, and the MIP mode can each be mapped to a group.

[0186] Furthermore, for each transform pair, the five transform kernels of DCT 8, DST 7, DCT 5, DST 4, and DST 1 can be applied as a pair, and each transform pair can be indexed to any of 0 to 24.

[0187] As described above, if four transformation sets are applied to a single group, then a transformation set consisting of four transformation pairs can be applied to each of the 80 groups. In this case, some of the 80 groups can share transformation sets, and the 80 groups can be reduced to 58 groups.

[0188] Figure 9 An example of deriving DIMM-based intra-frame modes for MTS and LFNST is given.

[0189] refer to Figure 9 The intra-frame mode based on DIMM can be derived to determine the MTS set and LFNST set.

[0190] Furthermore, when the intra-prediction mode is IntraTMP, the intra-prediction mode derived via DIMD can be applied as the intra-prediction mode for determining the MTS transform set. For example, the intra-prediction mode with the largest histogram magnitude can be used as the intra-prediction mode for determining the MTS set and the LFNST set.

[0191] Furthermore, according to another example, the intra-mode for IntraTMP can be derived from the reference block or the adjacent block of the reference block.

[0192] Furthermore, when the intra-prediction mode is any of IBC (Intra-Block Copy), SGPM (Spatial Geometric Partition Mode), TIMD (Template-Based Intra-Mode Derivation), and Palette Mode, DIMD can be applied based on the prediction samples derived through the corresponding mode to derive the intra-prediction mode used to determine the MTS transform set.

[0193] Furthermore, according to the example, MTS can be applied even when the transform block size is greater than 32. Alternatively, considering complexity, for each transform block size, the intra-prediction modes can be grouped into 3 groups or 2 groups instead of 5 groups. Alternatively, in addition to Figure 8 In addition to the transformation pairs, identity transformations (IDTs) can also be applied. For example, either the vertical transformation or the horizontal transformation can be like this: Figure 8 Any one of DST 7, DCT 8, DCT 5, DST 4, and DST 1 shown, and the other one can be an identity transformation. In this case, the number of transformation pairs can be increased.

[0194] In addition, the signaling of the MTS index (mts_idx) is described in detail below.

[0195] According to the example, `mts_enabled_flag` can be signaled at a high level, and `mts_flag` and `mts_idx` can be signaled at either the CU (coding unit) level or the residual coding level. In this case, if `mts_flag` is 1, `mts_idx` is signaled to determine the transform set, and if `mts_flag` is 0, DCT 2 can be applied to the main transform.

[0196] Furthermore, if the number of transform sets is 4, one of the 4 transform pairs is selected via `mts_idx`, and if the number of transform sets is 6, one of the 6 transform pairs can be selected via `mts_idx`. Conversely, when only one transform set exists, the first transform pair of either the 4-transform set or the 6-transform set can be selected without any explicit signaling. That is, if the number of transform sets is 1, `mts_flag` is signaled with a value of 1, but `mts_idx` does not need to be signaled. Furthermore, when `mts_idx` is not signaled, a predefined specific transform pair can be used, or `mts_idx` can be implicitly deduced as 0 or 1. Alternatively, if the number of transform sets is 1, `mts_flag` is signaled as 1, and `mts_idx` equal to 0 indicates the application of the first transform pair of the 4-transform set, while `mts_idx` equal to 1 indicates the application of the first transform pair of the 6-transform set.

[0197] For example, the binarization of the value of the MTS index (mts_idx) can be shown in the table below.

[0198] [Table 1]

[0199] Furthermore, according to another example, in the absence of a signaling flag (mts_flag), when mts_idx is 0, it is possible to omit the application of MTS; that is, it can indicate that DCT 2 is applied to the main transform. In the above case, the binarization of the MTS index can be shown in the table below.

[0200] [Table 2]

[0201] As shown in the table above, MTS indexes can be binarized using a truncated Rice (or truncated unary) scheme, or they can be binarized based on a fixed-length encoding scheme.

[0202] Alternatively, according to another example, when the number of transform sets is 1, a specific transform pair other than 4 or 6 transform sets can be used, and in this case, mts_idx can indicate the specific transform pair. In this case, mts_idx indicating 1 transform set can be signaled as 6 in Table 1 and as 7 in Table 2. Alternatively, mts_idx can be signaled as an intermediate value, for example, the value 3 or 4 in Table 1 or Table 2.

[0203] Alternatively, when an IDT (identity transformation) is applied such that any one of the six transformation kernels is applied, a signal can be used after `mts_idx` to indicate whether the IDT is applied in the horizontal or vertical direction. For example, the `idt_flag` can be used to indicate whether the IDT is applied, and when `idt_flag` is 1, the direction of application of the identity transformation can be further indicated by a signal. Alternatively, separate signals can be used to indicate whether the identity transformation is applied in the horizontal and vertical directions.

[0204] Alternatively, when applying the IDT (Identity Transformation), the `mts_inter_enabled_flag` or `mts_intra_enabled_flag` can be signaled at a high level, and the kernel indices of the six kernels can be signaled at a low level for each direction. For example, the kernel index for the horizontal direction can be signaled as `mts_horizontal_idx`, and the kernel index for the vertical direction can be signaled as `mts_vertical_idx`.

[0205] The following section describes inter-frame MTS applied to inter-prediction blocks.

[0206] For example, when MTS is applied to inter-prediction blocks, DST 7 and DCT 8 can be applied as transform kernels, and four transform pairs ((DCT 8, DCT 8), (DCT 8, DST 7), (DST 7, DCT 8), (DST 7, DST 7)) can be applied to each CU. For example, in the cases of 4-point, 8-point, and 16-point transforms (that is, when the width or height of the transform block is 16 or less), a separable KLT core kernel can be used instead of DST 7 and DCT 8. Furthermore, in large-resolution sequences such as those with a width > 1080, the maximum CU size for using Inter-MTS can be limited to 32×32 or less, and can be set to 16 for the remaining sequences.

[0207] Furthermore, although it is an intra-frame mode, an inter-frame MTS core can be applied instead of an intra-frame MTS core in IBC mode or IntraTMP mode. An inter-frame MTS core can be applied to IBC mode, an inter-frame MTS core can be applied to IntraTMP mode, or an inter-frame MTS core can be applied to both IBC and IntraTMP modes.

[0208] For example, in the case of inter-frame MTS, mts_idx can be signaled and binarized. For instance, binarization can be performed for transform pairs applying DCT 8 or DST 7 and KLT0 or KLT1, as shown in the table below. In other words, mts_idx can be binarized for transform pairs applying DCT 8 or DST 7 and KLT0 or KLT1, as shown in the table below.

[0209] [Table 3]

[0210] Furthermore, according to the example, DCT 8 or DST 7 can be applied in the horizontal direction, and KLT0 or KLT1 can be applied in the vertical direction. In this case, the transform kernel can be derived using the mapping relationship between DCT 8 or DST 7 and KLT0 or KLT1. In this case, mts_idx can be binarized, as shown in the table below.

[0211] [Table 4]

[0212] Furthermore, since the size of the transform block determines whether DCT 8 or DST 7, or KLT0 or KLT1, is used as the transform kernel, `mts_idx` can indicate which of DCT 8 and DST 7, or KLT0 and KLT1, is applied. For example, in the case of an 8×32 inter-frame block, 8 points of KLT0 or KLT1 can be applied horizontally, and either DCT 8 or DST 7 can be applied vertically. When KLT0 is applied horizontally and DST 7 is applied vertically, a signal notification value of 1 can be used as `mts_idx`.

[0213] The following section will describe the Inseparable Master Transform (NSPT) for the transformation.

[0214] For example, as described above, a separable transformation that applies the transform kernel to each of the vertical and horizontal directions can be applied as the principal transform or the inverse principal transform, and the LFNST, as an inseparable transform, can be applied as the quadratic transform or the inverse quadratic transform, and when the MTS is applied as the principal transform and the inverse principal transform, the LFNST may not be applied.

[0215] Furthermore, when MTS is not applied, and (DCT 2, DCT 2) is applied as the primary transform, and LFNST is applied, the inseparable primary transform can be applied as the primary transform. That is, the inseparable primary transform is performed as the primary transform, and no additional secondary transform is performed. This transform can be applied only to the luma component, or it can be applied to both the luma and chroma components. Alternatively, considering the application of the inseparable transform to intra-predicted blocks, it can be applied only to intra-predicted blocks. In addition, NSPT can be applied to each condition according to a specific tree structure. For example, in the case of dual-tree chroma, NSPT can be performed when the coded_flag for the chroma component is 0 or when transform_skip_flag[x0][y0][1] and transform_skip_flag[x0][y0][2] are 0. Furthermore, in the case of a two-tree luma component, NSPT can be performed when the coded_flag for the luma component is 0 or transform_skip_flag[x0][y0][0] is 0. Additionally, in the case of a single-tree structure, NSPT can be performed when the coded_flag for the luma component is 0 or transform_skip_flag[x0][y0][0] is 0, and the coded_flag for the chroma component is 0 or transform_skip_flag[x0][y0][1] and transform_skip_flag[x0][y0][2] are both 0. That is, in the case of a two-tree chroma component, NSPT can be applied only when the chroma block is not a transform skip, and in the case of a two-tree luma component, NSPT can be applied only when the luma block is not a transform skip. Furthermore, in the case of a single tree, LFNST can be applied only to the luma component, and in this case, NSPT can be applied only when all blocks are not transform skips.

[0216] Furthermore, similar to LFNST, 35 or 67 transform sets can be applied to NSPT, and three transform candidates can be applied to each individual transform set. The transform kernel can be derived based on the size of the block to which NSPT is applied.

[0217] Figure 10 An NSPT core based on block size is illustrated as an example.

[0218] refer to Figure 10Available NSPT cores may include NSPT4×4 applied to 4×4, NSPT4×8 applied to 4×8, NSPT8×4 applied to 8×4, NSPT8×8 applied to 8×8, NSPT4×16 applied to 4×16, NSPT16×4 applied to 16×4, NSPT8×16 applied to 8×16, NSPT16×8 applied to 16×8, NSPT4×32 applied to 4×32, NSPT32×4 applied to 32×4, NSPT8×32 applied to 8×32, and NSPT32×8 applied to 32×8.

[0219] Specifically, Figure 10 (a) can represent the NSPT applied to a 4×N / N×4 block. Figure 10 (b) can represent the NSPT applied to an 8×N / N×8 block, and Figure 10 (c) can represent the transformation applied to a 16×N / N×16 block. As mentioned above, in a 4×N / N×4 block, the dimensions of the NSPT kernel can be as follows.

[0220] - NSPT4×4: 16×16

[0221] - NSPT4×8 / NSPT8×4: 32×20

[0222] - NSPT8×8: 64×3

[0223] - NSPT4×16 / NSPT16×4: 64×24

[0224] - NSPT8×16 / NSPT16×8: 128×40

[0225] - NSPT4×32 / NSPT32×4: 128×20

[0226] - NSPT8×32 / NSPT32×8: 256×24

[0227] For example, when applying NSPT4×8 / NSPT8×4, 12 transform coefficients can be zeroed; when applying NSPT8×8, 32 transform coefficients can be zeroed; when applying NSPT4×16 / NSPT16×4, 40 transform coefficients can be zeroed; when applying NSPT8×16 / NSPT16×8, 88 transform coefficients can be zeroed; when applying NSPT4×32 / NSPT32×4, 108 transform coefficients can be zeroed; and when applying NSPT8×32 / NSPT32×8, 232 transform coefficients can be zeroed.

[0228] like Figure 10 As shown and as stated above, NSPT can be applied only to transform blocks of a specific size, and LFNST can be omitted when NSPT is applied.

[0229] Furthermore, the NSPT index (nspt_idx) of the NSPT core can be signaled only if the size condition of the transform block is met. In this case, if the NSPT index is 0, it can indicate that the NSPT has not been applied to the transform block, and if the NSPT index is not signaled, it can be inferred to be 0.

[0230] According to the example, since LFNST is not applied when NSPT is applied, the NSPT index can be signaled before lfnst_idx. For example, when the NSPT index is signaled, lfnst_idx can be left unsigned. On the other hand, when the NSPT index is not signaled or the NSPT index is 0, lfnst_idx can be signaled so that LFNST is performed as the inverse quadratic transform, and DCT 2 can be applied as the inverse principal transform. In this case, mts_idx can be left unsigned.

[0231] Furthermore, according to another example, `lfnst_idx` can be signaled first, and the NSPT index and `mts_idx` can be signaled after `lfnst_idx`. Additionally, when `lfnst_idx` is not signaled or is 0, the NSPT index and `mts_idx` can be signaled. In this case, the NSPT index and `mts_idx` can be signaled sequentially, or not sequentially. Furthermore, since the NSPT index must be signaled before parsing the transform coefficients, it can be signaled after the `last_sig_coeff_pos` syntax element, similar to `lfnst_idx`. Furthermore, the NSPT index can be signaled at the residual level (residual coding syntax), and `mts_idx` can be signaled at the CU level, TU tree level, or TU level.

[0232] Furthermore, the NSPT index can be replaced by signaling the LFNST index. That is, the NSPT index is not signaled, and the NSPT core can be selected based on the value of the LFNST index when a specific transform block size is met. For example, as... Figure 8As shown, when a specific block size is met, NSPT (i.e., the non-separable transform) is applied to the transform, and in this case, the core of the non-separable transform (i.e., NSPT) can be selected based on the value of the LFNST index (or index information indicating the non-separable transform core) as signaled. If the transform block size is a transform block for which NSPT is not applied, LFNST can be applied, and in this case, the core of the non-separable transform (i.e., LFNST) can be selected based on the value of the LFNST index (or index information indicating the non-separable transform core) as signaled. Furthermore, when LFNST is not applied, a separable transform can be applied, and in this case, either DCT 2 transform or MTS can be applied.

[0233] Furthermore, the NSPT kernel set and LFNST kernel set can be selected based on the size of the transform block and the intra-prediction mode. For example, NSPT can be used for block shapes of 4×4, 4×8, 4×16, 8×8, 8×16, and 8×32, as well as the corresponding transposed blocks (e.g., 4×4, 8×4, 16×4, 8×8, 16×8, and 32×8 blocks), and LFNST can be used for the remaining block shapes.

[0234] Furthermore, NSPT can be used in most intra-frame prediction tools, such as regular intra-frame prediction, DIMD, TIMD, SGPM, MIP, EIP, IntraTMP, etc. Additionally, NSPT can be applied to inter-frame CUs. In this case, the existing NSPT kernel set can be replaced with three kernel sets by using two additional NSPT kernel sets. For example, the first NSPT kernel set can be applied to a block using regular intra-frame prediction, the second NSPT kernel set can be applied to a block using TIMD, DIMD, EIP, MIP, or SGPM, and the third NSPT kernel set can be applied to a block using IntraTMP and inter-frame CUs. That is, three kernel sets can be used depending on the intra-frame prediction mode, and additional signaling is not required. Furthermore, for cases other than regular intra-frame prediction modes, such as DIMD, TIMD, SGPM, MIP, EIP, IntraTMP, or inter-frame prediction modes, additional kernel sets can be used, and additional signaling information is not required. Furthermore, information about the three kernel sets can be indicated by signaling. Additionally, a flag indicating whether to use additional NSPT transform kernel sets for DIMD, TIMD, EIP, MIP, SGPM, etc., can be indicated by signaling. For example, when the value of the flag is 1, the additional transformation kernel set can be used, and when the value of the flag is 0, the additional transformation kernel set can be omitted.

[0235] Furthermore, multiple transform set selection (MTSS) for intra-frame LFNST / NSPT can be used. For example, multiple LFNST / NSPT transform sets can be used for DIMD, TIMD, OBIC, SGPM, MIP, EIP, and IntraTMP modes. In this case, the DIMD method used to derive the intra-frame prediction mode for transform set selection can operate on subsampled neighboring or prediction block samples. For example, a second intra-frame prediction mode for selecting the LFNST / NSPT transform kernel for MIP, EIP, SGPM, and IntraTMP modes can be derived from the second high HoG of the prediction block or neighboring block. In this case, the first and second transform sets can be different transform sets or different transform types.

[0236] Furthermore, there may be limitations on the block size and the number of transformation candidates for the second set. For example, MTSS can be used based on the block size. For instance, MTSS can be applied to CUs with a width × height equal to or greater than 128. Furthermore, for CUs with a width × height less than 256, only the first two sets of candidates can be used, and for larger CUs, all three sets of candidates can be used. In this case, the block size to which MTSS can be applied is not limited to 128. Additionally, whether to apply MTSS can be determined based on the shape of the current block.

[0237] Furthermore, the MTSS for LFNST / NSPT can include a first transform set and a second transform set. In this case, the intra-prediction mode for the first transform set can vary depending on the prediction mode. For example, in the cases of DIMD, OBIC, and TIMD, the first intra-prediction mode for the first transform set can be the prediction mode used in the current prediction mode, and in the cases of SGPM, MIP, EIP, and IntraTMP, the first intra-prediction mode for the first transform set can be a prediction mode derived based on HoG. Furthermore, for example, in the cases of DIMD, OBIC, and TIMD, the second intra-prediction mode for the second transform set can be the prediction mode used in the current prediction mode, and in the cases of SGPM, MIP, EIP, and IntraTMP, the second intra-prediction mode for the second transform set can be a prediction mode derived based on HoG. Additionally, a flag indicating whether MTSS is used can be signaled. For example, when the flag value is 1, MTSS can be used, and when the flag value is 0, MTSS can not be used. Furthermore, a flag or index indicating the first or second transform set can be signaled. That is, the value of the flag or index can indicate either the first or second transform set.

[0238] Furthermore, the transform set for intra-chroma blocks encoded using LFNST / NSPT and CCP modes can be derived. For example, the intra-prediction mode for the current block can be derived based on CCP prediction samples using DIMD. Specifically, DIMD can be applied to CCP prediction samples, and at this point, the LFNST / NSPT transform set can be determined using a first DIMD mode. For example, the horizontal and vertical gradients can be computed for each prediction sample to derive HoG, and the LFNST / NSPT transform set can be determined by using the intra-prediction mode with the largest histogram magnitude.

[0239] Furthermore, in the case of SGPM mode, the partitioning direction can be used to determine the transform kernel set. As another example, a Virtual Intra-Prediction Mode (VIPM) can be calculated by applying the DIMD process to the SGPM prediction signal, allowing the determination of the LFNST / NSPT transform kernel set. In this case, a threshold can be used to compare the VIPM with the intra-prediction modes of the SGPM. For example, when the value of the VIPM is close to at least one intra-prediction mode in the SGPM, the VIPM can be used, and the LFNST / NSPT transform kernel set can be derived using the VIPM. Otherwise, the partitioning direction can be used to derive the LFNST / NSPT transform kernel set. A threshold can be used to determine whether the value of the VIPM is close to at least one intra-prediction mode in the SGPM. For example, when the difference between an intra-prediction mode in the SGPM and the value of the VIPM is less than N, the LFNST / NSPT transform kernel set can be derived using the VIPM. Here, N can be an integer greater than or equal to 0.

[0240] The following section will describe the extension of LFNST for the transformation.

[0241] For example, the transform set and transform kernel of LFNST can be extended. For example, the number of LFNST transform sets can be 35, and each transform set can consist of three transform kernels (that is, transform candidates). The transform set (lfnstTrSetIdx) for the intra-prediction mode can have the mapping relationship shown in the table below.

[0242] [Table 5]

[0243] Referring to Table 5 above, when the intra-prediction mode (predModeIntra) is less than 0, lfnstTrSetIdx can be mapped to 2. When the intra-prediction mode is 0 to 34, lfnstTrSetIdx can also be mapped to 0 to 34. Furthermore, when the intra-prediction mode is 35 to 66, lfnstTrSetIdx can be mapped to (68 - predModeIntra). Additionally, when the intra-prediction mode is greater than 66, lfnstTrSetIdx can be mapped to 2. In this case, lfnstTrSetIdx can be referred to as the LFNST set index.

[0244] Furthermore, LFNST cores can include LFNST 4 cores, LFNST 8 cores, and LFNST 16 cores. For example, in addition to LFNST 4 cores and LFNST 8 cores, LFNST 16 cores can be applied. For instance, an LFNST 4 core can be applied to blocks of 4×N / N×4 (N≥4), an LFNST 8 core can be applied to blocks of 8×N / N×8 (N≥8), and an LFNST 16 core can be applied to blocks of 16×N / N×16 (N≥16).

[0245] Furthermore, regions of interest (ROIs) and zeroing can be applied in LFNST. Here, an ROI can represent an operation that is of interest to a specific part of an image, or it can represent a sample within a dataset for a specific purpose of recognition. For example, forward LFNST is applied to an ROI that is a specific region of interest located at the top left of the target block. Therefore, when LFNST is applied, the principal transform coefficients existing in regions other than the ROI can be zeroed out.

[0246] Figure 11 An ROI for LFNST 16 is shown as an example.

[0247] refer to Figure 11 For LFNST 16, the ROI can consist of six 4×4 sub-blocks arranged consecutively in the scan direction, starting from the top left of the target block. Since a total of 96 transform coefficients are input to the forward LFNST, the dimension of the forward LFNST matrix can be R×96. Here, R can be 32, 48, or 64, which is less than 96. As an example, R can be 32, and in this case, LFNST 16 is a 32×96 matrix. Furthermore, for example, when a 32×96 matrix is ​​used for LFNST 16, the principal transform coefficients existing in regions other than the ROI can be zeroed out.

[0248] Figure 12 An example of the ROI for LFNST 8 is shown.

[0249] refer to Figure 12 The ROI for LFNST 8 can consist of four 4×4 sub-blocks located in the upper left of the target block (that is, an 8×8 region located in the upper left of the target block). Since a total of 64 transform coefficients are input into the forward LFNST, the dimension of the forward LFNST matrix can be R×64. Here, R can be 32 or 48, which is less than 64. As an example, R can be 32, and in this case, LFNST 8 is a 32×64 matrix. When a 32×64 matrix is ​​used for LFNST 8, the principal transform coefficients existing in regions other than the ROI can be zeroed out. Furthermore, for example, in the case of an 8×8 block, since 64 transform coefficients are input into the forward LFNST, the entire block is the ROI, so zeroing out is not required.

[0250] Figure 13 An exemplary MIP prediction sample for HoG construction is shown.

[0251] refer to Figure 13 HoG can be constructed using DIMD based on MIP-based prediction samples. For example, in the case of target blocks predicted by MIP or IntraTMP, DIMD can be used to derive the intra-prediction mode of the target block based on the MIP- or IntraTMP-based prediction samples. For example, in the case of target blocks applying MIP, DIMD can be applied based on the MIP prediction samples before upsampling. For example, to construct HoG, the horizontal and vertical gradients are calculated for each prediction sample. Subsequently, the LFNST transform set and LFNST transpose flag can be determined by using the intra-prediction mode with the largest histogram magnitude. Furthermore, the LFNST transpose flag is information indicating whether the LFNST kernel is transposed and can be signaled after the LFNST index signaling. Alternatively, for example, the MIP transpose flag (mip_transposed_flag) can be used as the LFNST transpose flag. In this case, the signaling of the LFNST transpose flag can be omitted.

[0252] Furthermore, when the intra-prediction mode is IntraTMP, the intra-prediction mode derived via DIMD can be applied as the intra-prediction mode for determining the LFNST transform set. In this case, the intra-prediction mode with the largest histogram magnitude can be used as the intra-prediction mode for determining the LFNST set.

[0253] Alternatively, according to the example, when the intra-prediction mode is any of IBC, SGPM (Spatial Geometric Partitioning Mode), TIMD (Template-based Intra-mode Derivation), and Palette Mode, the intra-prediction mode for determining the LFNST transform set can be derived by applying DIMD based on prediction samples derived via the corresponding mode.

[0254] Alternatively, according to the example, LFNST or NSPT can be applied to inter-frame prediction blocks instead of intra-frame prediction. For example, a transform set can be mapped for each inter-frame prediction mode, and LFNST or NSPT can be performed by applying any of a plurality of transform kernels to the mapped transform set. Alternatively, according to the example, DIMD can be applied based on inter-frame prediction samples, and the horizontal and vertical gradients can be computed for each prediction sample to construct HoG. The LFNST transform set can then be derived using the prediction mode with the largest histogram magnitude.

[0255] Furthermore, for the encoding of LFNST / NSPT coefficients, a modified context model can be used that uses the previous five coefficients in encoding order instead of adjacent 2D coefficients. That is, when applying LFNST / NSPT, the five previous transform coefficients in encoding (scanning) order, rather than adjacent coefficients at 2D positions, can be used for context modeling or context information derivation regarding the currently parsed transform coefficients (e.g., which may include at least one of sig_coeff_flag, gt1_flag, or gt2_flag).

[0256] Figure 14 An exemplary context modeling framework for LFNST and NSPT transform coefficients is illustrated.

[0257] refer to Figure 14 (a) illustrates the context modeling of adjacent 2D coefficients (No. 6, No. 7, No. 9, No. 10, and No. 13) for the currently parsed transform coefficient (No. 5). In this case, the currently parsed transform coefficient can be No. 5, and the adjacent 2D coefficients can be No. 6, 7, 9, 10, and 13. Furthermore, refer to... Figure 14 (b) can illustrate context modeling of the five previous coefficients (No.2, No.3, No.6, No.9, and No.12) used for the transformation coefficient (No.5) to be resolved. In this case, the transformation coefficient to be resolved can be No.5, and the five previous coefficients can be No.2, No.3, No.6, No.9, and No.12. In this scenario, the values ​​or absolute values ​​of the previous five coefficients can be used. For example, contextual information (context index or context increment) about the transformation coefficient can be derived based on whether the sum of the values ​​or absolute values ​​of the five coefficients is greater than a threshold. Alternatively, for example, contextual information (context index or context increment) about the transformation coefficient can be derived based on whether the average of the values ​​or absolute values ​​of the five coefficients is greater than a threshold.

[0258] Furthermore, since lfnstIdx (lfnst_idx) is required for parsing the transform coefficients, lfnstIdx can be signaled after all last_sig_coeff_pos syntax elements of the CU. For example, when applying LFNST or NSPT, the DCT-2 transform coefficients can be arranged within a coefficient block by using diagonal reordering. Alternatively, when applying LFNST or NSPT, the transform coefficient scan order can be changed / derived due to zeroing. According to the example, lfnstIdx and / or the NSPT index can be signaled after last_sig_coeff_pos at the residual coding level.

[0259] Furthermore, in the case of inseparable transforms (such as LFNST or NSPT), one-dimensional (1D) orientation (such as horizontal or vertical) may not significantly affect the encoding of transform coefficients or may be unimportant for the encoding of transform coefficients. Therefore, when performing a transform via LFNST or NSPT, the transform can be performed by reordering the transform coefficients in the diagonal direction, and by reflecting this, contextual modeling of the transform coefficients can be performed by utilizing modeling information of the coefficients encoded first according to the scan order of the diagonal direction. Thus, by signaling the LFNST index before encoding the transform coefficients, contextual modeling for the transform coefficients can be adaptively performed by distinguishing between cases where LFNST (or NSPT) is applied and cases where LFNST (or NSPT) is not applied.

[0260] In this scenario, mts_idx can be signaled at the same level as lfnstIdx and / or the NSPT index (e.g., at the residual coding level), or it can be signaled at the CU level, TU tree level, or TU level. Furthermore, mts_idx can be signaled immediately after lfnstIdx signaling or immediately after the NSPT index signaling.

[0261] As an example, the transformation-related syntax can be shown in the table below.

[0262] [Table 6]

[0263] Referring to Table 6 above, signals can be used to notify lfnst_idx, nspt_idx, mts_flag, and / or mts_idx in the same syntax, or some of them can be notified using signals in another syntax.

[0264] For example, after signaling lfnst_idx, you can signal nspt_idx, mts_flag, and / or mts_idx. Furthermore, you can signal nspt_idx, or you can signal mts_flag and / or mts_idx based on the value of lfnst_idx. For example, you can signal lfnst_idx when the LFNST application condition is true. Additionally, for example, you can signal nspt_idx when the value of lfnst_idx is 0 and the NSPT application condition is true; otherwise, you can signal mts_flag and / or mts_idx.

[0265] As another example, the transformation-related syntax can be shown in the table below.

[0266] [Table 7]

[0267] Referring to Table 7 above, nspt_idx, lfnst_idx, mts_flag and / or mts_idx can be signaled in the same syntax, or some of them can be signaled in another syntax.

[0268] For example, after signaling nspt_idx, you can signal lfnst_idx, mts_flag, and / or mts_idx. For example, you can signal nspt_idx when the NSPT application condition is true. Furthermore, for example, you can signal lfnst_idx when the value of nspt_idx is 0 and the LFNST application condition is true. Furthermore, for example, you can signal mts_flag and / or mts_idx when the value of lfnst_idx is 0.

[0269] As another example, the transformation-related syntax can be shown in the table below.

[0270] [Table 8]

[0271] Referring to Table 8 above, nspt_idx, lfnst_idx, mts_flag and / or mts_idx can be signaled in the same syntax, or some of them can be signaled in another syntax.

[0272] For example, when the NSPT application condition is true, nspt_idx can be notified by a signal. Furthermore, when the NSPT application condition is not true and the LFNST application condition is true, lfnst_idx can be notified by a signal. Additionally, for example, when the value of lfnst_idx is 0, mts_flag and / or mts_idx can be notified by a signal.

[0273] Furthermore, lfnst_idx can indicate the LFNST transform kernel and can conditionally indicate the NSPT transform kernel. For example, the value of lfnst_idx can indicate one of the NSPT candidates when the NSPT conditions described in this disclosure (e.g., the NSPT application block size condition) are met.

[0274] As another example, the transformation-related syntax can be shown in the table below.

[0275] [Table 9]

[0276] Referring to Table 9 above, signals can be used to notify lfnst_idx, nspt_idx, mts_flag, and / or mts_idx in the same syntax, or some of them can be notified using signals in another syntax.

[0277] For example, when the LFNST application condition is true, lfnst_idx can be notified using a signal. Otherwise, mts_flag and / or mts_idx can be notified using a signal.

[0278] As another example, the transformation-related syntax can be shown in the table below.

[0279] [Table 10]

[0280] Referring to Table 10 above, when the value of lfnst_idx is 0 and the value of nspt_idx is 0, mts_idx can be notified at the coding unit level by signaling.

[0281] In addition, the signaling for the transform_tree syntax can be shown in the table below.

[0282] [Table 11]

[0283] Referring to Table 11 above, the transform_unit syntax can be signaled at the transform_tree syntax level.

[0284] In addition, the signaling for the residual_coding syntax can be shown in the table below.

[0285] [Table 12]

[0286] Referring to Table 12 above, the residual_coding syntax can be signaled at the transform_unit syntax level.

[0287] Furthermore, as another example, the transformation syntax can be as follows.

[0288] [Table 13]

[0289] Referring to Table 13 above, after notifying lfnst_idx with a signal, you can also notify nspt_idx with a signal.

[0290] For example, when the last valid coefficient is located in the lfnst zero-return region, LfnstZeroOutSigCoeffFlag can be set to 0. Furthermore, when the last valid coefficient is located in the lfnst coefficient region (that is, the region other than the lfnst zero-return region), LfnstZeroOutSigCoeffFlag can be set to 1. In this case, the last valid coefficient position can be derived based on the last valid coefficient position information (e.g., including at least one of last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, or last_sig_coeff_y_suffix).

[0291] Specifically, `last_sig_coeff_x_prefix` represents the column position prefix of the last valid coefficient in scan order within the transform block, `last_sig_coeff_y_prefix` represents the row position prefix of the last valid coefficient in scan order within the transform block, `last_sig_coeff_x_suffix` represents the column position suffix of the last valid coefficient in scan order within the transform block, and `last_sig_coeff_y_suffix` represents the row position suffix of the last valid coefficient in scan order within the transform block. Here, valid coefficients can represent non-zero coefficients. Although described in units of transform blocks (TB), this is an example, and TB can be used interchangeably with code blocks (CB).

[0292] For example, when the value of LfnstZeroOutSigCoeffFlag is 1, lfnst_idx can be notified by a signal. Furthermore, when the value of lfnst_idx is 0 and the NSPT application condition is true, nspt_idx can be notified by a signal.

[0293] Furthermore, for example, when applying LFNST and / or NSPT, the signaling of sb_coded_flag[xS][yS] of the last sub-block and the sub-block within the DC sub-block, including the last valid coefficient, can be omitted, and its value can be derived as 1.

[0294] In addition, for example, when the value of lfnst_idx or nspt_idx is greater than 0, some or all of sb_coded_flag[xS][yS] can be omitted.

[0295] Furthermore, transformation-related information can be encoded based on contextual information, and the relevant contextual information can be represented, for example, as follows.

[0296] For example, even when the NSPT condition (e.g., the block size condition for NSPT application) is met (that is, when lfnst_idx replaces nspt_idx or is used interchangeably with nspt_idx), and the value of lfnst_idx can indicate one of the NSPT candidates, a different context model (context information) may be needed depending on whether NSPT or LFNST is applied. For this purpose, the following structure can be provided. For example, when the NSPT condition (e.g., the block size condition for NSPT application) is met, ApplyNsptFlag can be set to 1.

[0297] For example, examples of a context model for assigning ctxInc to syntax elements using context-encoded bin can be shown in Tables 14 through 17 below.

[0298] [Table 14]

[0299] [Table 15]

[0300] [Table 16]

[0301] [Table 17]

[0302] Furthermore, when nspt_idx and lfnst_idx are used separately, examples of the context model used to assign ctxInc to syntax elements using context-encoded bins can be shown in Tables 18 to 21 below.

[0303] [Table 18]

[0304] [Table 19]

[0305] [Table 20]

[0306] [Table 21]

[0307] Figure 15 A video / image coding method according to an embodiment of the present disclosure is illustrated schematically. Figure 15 The method disclosed in the article can be derived from Figure 2 The encoding device disclosed in the document executes the code. Specifically, for example, Figure 15 S1500 to S1520 can be executed by the residual processor 230 of the encoding device 200, and S1530 can be executed by the entropy encoder 240 of the encoding device 200. Figure 15 The methods disclosed herein may include the embodiments described above.

[0308] refer to Figure 15 The encoding device derives a residual sample for the current block (S1500). For example, the encoding device can derive a residual sample for the current block.

[0309] The encoding device derives the transform coefficients for the current block (S1510). For example, the encoding device can derive the transform coefficients for the current block based on the residual samples.

[0310] For example, a transformation for the residual samples can be performed using either a primary or secondary transformation. Furthermore, the NSPT transformation can be applied depending on the situation. In this case, the NSPT transformation can be an integration and replacement of both the primary and secondary transformations.

[0311] Furthermore, LFNST can be applied to the transform when the transform block size is the same as the transform block size without NSPT. In this case, the LFNST core can be selected based on the value of the LFNST index information.

[0312] Furthermore, when LFNST is not applied, a separable transform can be applied. In this case, either the DCT-2 transform or the MTS can be applied.

[0313] The encoding device generates residual information for the current block (S1520). For example, the encoding device may generate residual information for the current block based on the transform coefficients.

[0314] The encoding device encodes image information including residual information (S1530). For example, the encoding device can encode image information including residual information.

[0315] Image information may include at least one of the following: Multi-Transform Selection (MTS) index information, Low-Frequency Non-Separable Transform (LFNST) index information, or Non-Separable Master Transform (NSPT) index information.

[0316] Furthermore, based on the size of the current block, it can be determined whether to signal the NSPT index information in the image information. For example, NSPT index information can only be included in the image information and signaled if the current block size condition is met. For example, if NSPT index information is not signaled, the value of the NSPT index can be configured to be inferred as 0. Here, NSPT index information with a value of 0 can indicate that NSPT has not been applied to the current block. That is, if the value of the NSPT index is 0, it can indicate that NSPT has not been applied to the current block. In this case, the current block can be a transform block.

[0317] Furthermore, at least one of the LFNST index information or MTS index information can be signaled after the NSPT index information. Additionally, if the value of the NSPT index information is 0, the MTS index information can be signaled without signaling the LFNST index information. In this case, the signaling of the MTS index information can be explicit signaling. Furthermore, based on whether the NSPT index information is signaled, it can be determined whether the LFNST index information is signaled. For example, if the NSPT index information is signaled, the LFNST index information can be signaled without signaling. Furthermore, for example, if the NSPT index information is not signaled, the LFNST index information can be signaled.

[0318] In addition, at least one of the LFNST index information or MTS index information can be signaled after the NSPT index information.

[0319] For example, when NSPT index information is notified using a signal, LFNST index information can be notified without a signal. Furthermore, when NSPT index information is not notified using a signal, LFNST index information can be notified using a signal.

[0320] Furthermore, for example, when the NSPT index information is signaled and the value of the NSPT index information is 0, the LFNST index information can be signaled. In this case, LFNST can be performed as an inverse quadratic transform, and DCT-2 can be performed as an inverse principal transform. In this case, the MTS index information can be signaled to perform a DCT-2-based inverse principal transform, or the DCT-2-based inverse principal transform can be performed without signaling the MTS index information.

[0321] Furthermore, for example, when the NSPT index information is notified by signal and the value of the NSPT index information is 0, the MTS index information can be notified by signal without notifying the LFNST index information.

[0322] Furthermore, if the NSPT index information is not notified by a signal or the value of the NSPT index information is 0, the LFNST index information can be notified by a signal. In this case, when the LFNST index information is included in the image information and is notified by a signal, LFNST can be performed as a secondary transform, and DCT-2 can be performed as the primary transform. In this situation, the MTS index information can be notified without a signal.

[0323] Furthermore, at least one of the NSPT index information or MTS index information can be signaled after the LFNST index information. Additionally, based on whether the LFNST index information is signaled, it can be determined whether the NSPT index information or MTS index information is signaled. For example, if the LFNST index information is signaled, it is possible not to signal the NSPT index information or MTS index information. Furthermore, for example, if the LFNST index information is not signaled, it can be signaled to the NSPT index information or MTS index information. Furthermore, for example, if the LFNST index information is not signaled or the value of the LFNST index information is 0, it can be signaled to the NSPT index information or MTS index information.

[0324] In addition, at least one of the NSPT index information or MTS index information can be signaled after the LFNST index information.

[0325] For example, when signaling LFNST index information, it is not necessary to signal at least one of NSPT or MTS index information. Furthermore, when not signaling LFNST index information, it is possible to signal at least one of NSPT or MTS index information.

[0326] Furthermore, for example, when the LFNST index information is signaled and the value of the LFNST index information is 0, at least one of the NSPT index information or the MTS index information can be signaled.

[0327] Furthermore, the NSPT index information and MTS index information can be notified sequentially using signals, or they can be notified separately. For example, when notifying the NSPT index information and MTS index information sequentially using signals, the MTS index information can be notified after the NSPT index information is notified, or the NSPT index information can be notified after the MTS index information is notified.

[0328] Furthermore, the NSPT index information or LFNST index information can be signaled after the position information indicating the position of the last valid coefficient. Specifically, the NSPT index information or LFNST index information can be signaled after the position information of the last valid coefficient (e.g., including at least one of last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, or last_sig_coeff_y_suffix).

[0329] In addition, NSPT index information can be signaled in the residual coding syntax, and MTS index can be signaled in the coding unit syntax, transform tree syntax, or transform unit syntax.

[0330] Furthermore, the NSPT index can be replaced by the LFNST index. For example, when the image information does not include the NSPT index and the size of a specific transform block is satisfied, the NSPT kernel can be selected based on the value of the LFNST index. For example, as described above... Figure 10 In this context, NSPT can be applied to the transform when a specific block size is met. In this case, the NSPT core can be selected based on the value of the LFNST index information. That is, the NSPT index can be replaced by signaling the LFNST index.

[0331] For example, based on the fact that the current block size is a specific size, the NSPT core can be selected based on the LFNST index information. That is, when the current block size is the size for which NSPT is applied, the NSPT core can be selected based on the value of the LFNST index information notified by a signal.

[0332] Furthermore, for example, when the size of the current block is the size of the block before NSPT is applied, LFNST can be applied to the current block. In this case, the LFNST core can be selected based on the value of the LFNST index information notified by a signal.

[0333] In addition, for example, when the size of the current block is the size of the block without NSPT and LFNST is not applied to the current block, the DCT-2 transform can be applied to the current block, or the MTS can be applied to the current block.

[0334] Furthermore, according to embodiments of this disclosure, the image information may include various types of information. For example, the image information may include information disclosed in at least one of the tables described above.

[0335] Furthermore, the encoded image information can be output as a bitstream. The bitstream can be sent to a decoding device via a network or storage medium.

[0336] Furthermore, as mentioned above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks) based on reference samples and residual samples. This is to derive the same prediction result in the encoding device as the prediction result performed in the decoding device, thereby improving encoding efficiency. Therefore, the encoding device can store the reconstructed image (or reconstructed samples, reconstructed blocks) in memory and use it as a reference image for inter-frame prediction. As mentioned above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0337] According to the above implementation method, when applying NSPT to the current block, by efficiently notifying the MTS index information, LFNST index information and NSPT index information with signals, the transformation can be performed efficiently and the amount of information required for the transformation can be reduced, thereby improving the overall coding efficiency.

[0338] According to the above implementation method, when applying LFNST to the current block, by efficiently notifying the MTS index information, LFNST index information and NSPT index information with signals, the transformation can be performed efficiently and the amount of information required for the transformation can be reduced, thereby improving the overall coding efficiency.

[0339] According to the above implementation method, LFNST or NSPT can be adaptively executed by signaling the LFNST index information based on the size of the transform block, thereby improving the overall coding efficiency.

[0340] Figure 16 A video / image decoding method according to an embodiment of the present disclosure is illustrated schematically. Figure 16 The method disclosed in the article can be derived from Figure 3 The decoding device disclosed in the document performs the operation. Specifically, for example, Figure 16 S1600 can be executed by the entropy decoder 310 of the decoding device 300, and S1610 to S1620 can be executed by the residual processor 320 of the decoding device 300. Figure 16 The methods disclosed herein may include the embodiments described above.

[0341] refer to Figure 16 The decoding device receives image information including residual information for the current block (S1600). For example, the decoding device may receive image information including residual information for the current block. As described above, the image information may also include prediction-related information.

[0342] Image information may include at least one of MTS (Multiple Transform Selection) index information, LFNST (Low Frequency Inseparable Transform) index information, or NSPT (Inseparable Master Transform) index information.

[0343] Furthermore, based on the size of the current block, it can be determined whether the image information includes NSPT index information. For example, NSPT index information can only be included in the image information and signaled if the current block size condition is met. For example, if the image information does not include NSPT index information, the value of the NSPT index can be inferred to be 0. Here, based on the value of the NSPT index being 0, NSPT may not be applied to the current block. In this case, if the value of the NSPT index is 0, it can indicate that NSPT is not applied to the current block. In this case, the current block can be a transform block.

[0344] Furthermore, at least one of the LFNST index information or MTS index information may have a signaling order following the NSPT index information. Additionally, if the NSPT index information has a value of 0, the MTS index information can be signaled without signaling the LFNST index information. In this case, the signaling for the MTS index information can be explicit signaling. Furthermore, whether the image information includes LFNST index information can be determined based on whether the image information includes NSPT index information. For example, if the image information includes NSPT index information, it may not include LFNST index information. Conversely, if the image information does not include NSPT index information, it may include LFNST index information.

[0345] In addition, at least one of the LFNST index information or MTS index information can be signaled after the NSPT index information.

[0346] For example, when NSPT index information is notified using a signal, LFNST index information can be notified without a signal. Furthermore, when NSPT index information is not notified using a signal, LFNST index information can be notified using a signal.

[0347] Furthermore, for example, when the NSPT index information is signaled and the value of the NSPT index information is 0, the LFNST index information can be signaled. In this case, LFNST can be performed as an inverse quadratic transform, and DCT-2 can be performed as an inverse principal transform. Alternatively, the MTS index information can be signaled to perform a DCT-2-based inverse principal transform, or the DCT-2-based inverse principal transform can be performed without signaling the MTS index information.

[0348] Furthermore, for example, when the NSPT index information is notified by signal and the value of the NSPT index information is 0, the MTS index information can be notified by signal without notifying the LFNST index information.

[0349] Furthermore, based on the fact that the image information does not include NSPT index information or the value of the NSPT index information is 0, the image information may include LFNST index information. In this case, when the LFNST index information is included in the image information and is signaled, LFNST can be performed as an inverse quadratic transform, and DCT-2 can be performed as an inverse principal transform. In this situation, the MTS index information does not need to be signaled.

[0350] Furthermore, at least one of the NSPT index information or MTS index information may have a signaling order following the LFNST index information. In this case, the signaling of the MTS index information may be explicit signaling. Additionally, whether the image information includes NSPT index information or MTS index information can be determined based on whether the image information includes LFNST index information. For example, if the image information includes LFNST index information, the image information may not include NSPT index information or MTS index information. Furthermore, for example, if the image information does not include LFNST index information, the image information may include NSPT index information or MTS index information. Furthermore, for example, if the image information does not include LFNST index information or the value of LFNST index information is 0, the image information may include NSPT index information or MTS index information.

[0351] In addition, at least one of the NSPT index information or MTS index information can be signaled after the LFNST index information.

[0352] For example, when signaling LFNST index information, it is not necessary to signal at least one of NSPT or MTS index information. Furthermore, when not signaling LFNST index information, it is possible to signal at least one of NSPT or MTS index information.

[0353] Furthermore, for example, when the LFNST index information is signaled and the value of the LFNST index information is 0, at least one of the NSPT index information or the MTS index information can be signaled. Alternatively, the NSPT index information and the MTS index information can be signaled sequentially, or they can be signaled without being sequentially. For example, when the NSPT index information and the MTS index information are signaled sequentially, the MTS index information can be signaled after parsing the NSPT index information, or the NSPT index information can be signaled after parsing the MTS index information.

[0354] Furthermore, the NSPT index information or LFNST index information can be signaled after the position information indicating the position of the last valid coefficient. Specifically, the NSPT index information or LFNST index information can be signaled after the position information of the last valid coefficient (e.g., including at least one of last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, or last_sig_coeff_y_suffix).

[0355] In addition, NSPT index information can be signaled in the residual coding syntax, and MTS index can be signaled in the coding unit syntax, transform tree syntax, or transform unit syntax.

[0356] The NSPT index can be replaced by the LFNST index. For example, when the image information does not include the NSPT index and a specific transform block size is satisfied, the NSPT kernel can be selected as the value of the LFNST index. For example, as described above. Figure 10 In this context, NSPT can be applied to the transform when a specific block size is met. In this case, the NSPT core can be selected based on the value of the LFNST index information. That is, the NSPT index can be replaced by signaling the LFNST index.

[0357] For example, if the current block size is a specific size, the NSPT core can be selected based on the LFNST index information. In other words, when the current block size is the size for which the NSPT is applied, the NSPT core can be selected as the value of the LFNST index information signaled by the NSPT.

[0358] Furthermore, for example, when the size of the current block is the size before NSPT is applied, LFNST can be applied to the current block. In this case, the LFNST core can be selected as the value of the LFNST index information notified by a signal.

[0359] In addition, for example, when the size of the current block is the size of the block without NSPT and LFNST is not applied to the current block, the DCT-2 transform can be applied to the current block, or the MTS can be applied to the current block.

[0360] Furthermore, the image information may include various types of information according to embodiments of this disclosure. For example, the image information may include information disclosed in at least one of the tables described above.

[0361] The decoding device derives the transform coefficients for the current block (S1610). For example, the decoding device can derive the transform coefficients for the current block based on the residual information.

[0362] The decoding device derives a residual sample for the current block (S1620). For example, the decoding device can derive a residual sample for the current block based on the transform coefficients.

[0363] For example, a transformation of the transform coefficients can be performed using either a primary or secondary transform. Furthermore, depending on the circumstances, the NSPT transform can be applied. In this case, the NSPT transform can be an integration and replacement of both the primary and secondary transforms.

[0364] Furthermore, for example, when the size of the transform block is the same as the size of the transform block without NSPT, LFNST can be applied to the transform. In this case, the LFNST kernel can be selected based on the value of the LFNST index information.

[0365] Furthermore, when LFNST is not applied, a separable transform can be applied. In this case, either the DCT-2 transform or the MTS can be applied.

[0366] The decoding device can generate reconstructed samples based on the predicted samples of the current block. For example, the decoding device can generate reconstructed samples for the current block based on the residual samples and predicted samples for the current block. Residual samples for the current block can be generated based on the received residual information. Furthermore, as an example, the decoding device can generate a reconstructed image including the reconstructed samples. Thereafter, as described above, the decoding device can apply in-loop filtering processes (such as deblocking filtering and / or SAO processes) to the reconstructed image to improve the subjective / objective image quality as needed.

[0367] According to the above implementation method, when applying NSPT to the current block, by efficiently notifying the MTS index information, LFNST index information and NSPT index information with signals, the transformation can be performed efficiently and the amount of information required for the transformation can be reduced, thereby improving the overall coding efficiency.

[0368] According to the above implementation method, when applying LFNST to the current block, by efficiently notifying the MTS index information, LFNST index information and NSPT index information with signals, the transformation can be performed efficiently and the amount of information required for the transformation can be reduced, thereby improving the overall coding efficiency.

[0369] According to the above implementation method, LFNST or NSPT can be adaptively executed by signaling the LFNST index information based on the size of the transform block, thereby improving the overall coding efficiency.

[0370] Although the method is described as a series of steps or blocks based on the flowchart in the above embodiments, the embodiments are not limited to the order of the steps, and a step may occur in a different order than another step described above or simultaneously with another step described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and other steps may be included or one or more steps in the flowchart may be deleted without affecting the scope of the embodiments of this disclosure.

[0371] The methods described above according to embodiments of this disclosure can be implemented in software, and the encoding and / or decoding devices according to this disclosure can be included in devices for performing image processing, such as televisions, computers, smartphones, set-top boxes, and display devices.

[0372] The embodiments described above can be implemented in the form of a recording medium including computer-executable (program) instructions, such as a program module executed by a computer. The module can be stored in memory and executed by a processor. The memory can be located inside or outside the processor and can be connected to the processor by various known means. The computer-readable medium can be any available medium accessible to a computer and can include both volatile and non-volatile media, as well as removable and non-removable media. Furthermore, the computer-readable medium can include both computer storage media and communication media. Computer storage media can include both volatile and non-volatile media, as well as removable and non-removable media, implemented using any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Communication media typically include computer-readable instructions, data structures, program modules, other data in modulated data signals (such as carrier waves), or other transmission mechanisms, and include any information delivery medium.

[0373] Furthermore, the embodiments described above in this disclosure can be implemented as a computer program (or computer program product) including computer-executable instructions. The computer program may include programmable machine instructions processed by a processor and may be implemented in a high-level programming language, an object-oriented programming language, assembly language, or machine language. Additionally, the computer program may be recorded on a tangible computer-readable recording medium (e.g., memory, hard disk, magnetic / optical media, or solid-state drive (SSD)).

[0374] Therefore, embodiments of this disclosure can be implemented by executing the computer program described above using a computing device. The computing device may include at least some of a processor, memory, storage devices, high-speed interfaces connected to the memory and high-speed expansion ports, and low-speed interfaces connected to low-speed buses and storage devices. These components may be interconnected via various buses and may be mounted on a common motherboard or otherwise suitable.

[0375] A processor can process instructions within a computing device. These instructions may include those stored in memory or storage devices to display graphical information on an external input / output device (such as a display) connected to a high-speed interface, providing a graphical user interface (GUI). In another embodiment, multiple processors and / or multiple buses may be appropriately utilized along with multiple memories and memory types. Furthermore, the processor may be implemented as a chipset comprising multiple independent analog and / or digital processors.

[0376] Memory stores information within a computing device. For example, memory may include volatile memory cells or a collection of volatile memory cells. In another example, memory may include non-volatile memory cells or a collection of non-volatile memory cells. Memory may also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0377] Storage devices can provide large-capacity storage space for computing devices. Storage devices can be computer-readable media or components that include computer-readable media. For example, storage devices can include devices or other components within a storage area network (SAN) and can be floppy disk devices, hard disk devices, optical disk devices, magnetic tape devices, flash memory, other similar semiconductor storage devices, or device arrays.

[0378] The network can be implemented as a wired network, such as a local area network (LAN), a wide area network (WAN), or a value-added network (VAN), or various types of wireless networks, such as mobile radio communication networks or satellite communication networks.

[0379] Although this disclosure has been described with reference to embodiments illustrated in the accompanying drawings, these embodiments are merely exemplary. Those skilled in the art will understand that various modifications and variations are possible. That is, the scope of this disclosure is not limited to the described embodiments, and various modifications and alterations made by those skilled in the art based on the basic concepts defined in the appended claims also fall within the scope of the claims. Therefore, the true technical scope of this disclosure should be determined by the technical spirit of the appended claims.

Claims

1. An image decoding method performed by a decoding device, the image decoding method comprising the following steps: Receive image information including residual information for the current block; Based on the residual information, the transformation coefficients for the current block are derived; as well as Based on the transformation coefficients, the residual samples for the current block are derived. The image information includes at least one of the following: Multi-Transform Selection (MTS) index information, Low-Frequency Inseparable Transform (LFNST) index information, or Inseparable Main Transform (NSPT) index information.

2. The image decoding method according to claim 1, wherein, The image information is determined based on the size of the current block to determine whether it includes the NSPT index information.

3. The image decoding method according to claim 2, wherein, Since the image information does not include the NSPT index information, the value of the NSPT index is inferred to be 0, and based on the value of the NSPT index being 0, the NSPT is not applied to the current block.

4. The image decoding method according to claim 1, wherein, At least one of the LFNST index information or the MTS index information has a signaling order following the NSPT index information.

5. The image decoding method according to claim 4, wherein, Based on the value of the NSPT index information being 0, the MTS index information is notified by signaling without signaling the LFNST index information.

6. The image decoding method according to claim 4, wherein, Whether the image information includes the LFNST index information is determined based on whether the image information includes the NSPT index information. Wherein, the image information includes the NSPT index information, but the image information does not include the LFNST index information, and The image information does not include the NSPT index information, but includes the LFNST index information.

7. The image decoding method according to claim 1, wherein, The image information includes the LFNST index information if the image information does not include the NSPT index information or the value of the NSPT index information is 0.

8. The image decoding method according to claim 1, wherein, At least one of the NSPT index information or the MTS index information has a signaling order following the LFNST index information.

9. The image decoding method according to claim 8, wherein, Based on whether the image information includes the LFNST index information, determine whether the image information includes the NSPT index information or the MTS index information. Wherein, the image information includes the LFNST index information, but the image information does not include the NSPT index information or the MTS index information, and The image information does not include the LFNST index information; the image information includes the NSPT index information or the MTS index information.

10. The image decoding method according to claim 1, wherein, Based on the fact that the image information does not include the LFNST index information or the value of the LFNST index information is 0, the image information includes the NSPT index information and the MTS index information.

11. The image decoding method according to claim 1, wherein, The NSPT index information or the LFNST index information is signaled after the position information indicating the position of the last valid coefficient.

12. The image decoding method according to claim 1, wherein, The NSPT index information is included in the residual coding syntax, and the MTS index is included in the coding unit syntax, transform tree syntax, or transform unit syntax.

13. An image encoding method performed by an encoding device, the image encoding method comprising the following steps: Derive the residual samples for the current block; Based on the residual samples, derive the transformation coefficients for the current block; Residual information for the current block is generated based on the transformation coefficients; as well as The image information, including the residual information, is encoded. The image information includes at least one of the following: Multi-Transform Selection (MTS) index information, Low-Frequency Inseparable Transform (LFNST) index information, or Inseparable Main Transform (NSPT) index information.

14. The image encoding method according to claim 13, wherein, The image information is determined based on the size of the current block to determine whether it includes the NSPT index information.

15. The image encoding method according to claim 14, wherein, Based on the unused signal notification of the NSPT index information, the value of the NSPT index is configured to be inferred as 0, and the NSPT index with a value of 0 indicates that the NSPT has not been applied to the current block.

16. The image encoding method according to claim 13, wherein, Based on whether to signal the NSPT index information, determine whether to signal the LFNST index information. Specifically, the NSPT index information is notified by signal, while the LFNST index information is not notified by signal. The LFNST index information is notified by signaling when the NSPT index information is not used.

17. The image encoding method according to claim 13, wherein, The LFNST index information is signaled if the NSPT index information is not notified by a signal or if the value of the NSPT index information is 0.

18. The image encoding method according to claim 13, wherein, Whether to signal the NSPT index information or the MTS index information is determined based on whether the LFNST index information is signaled. Specifically, the LFNST index information is notified by signal, while the NSPT index information or the MTS index information is not notified by signal. Based on the absence of signal notification for the LFNST index information, signal notification is used for the NSPT index information or the MTS index information.

19. The image encoding method according to claim 13, wherein, Based on the image information not including the LFNST index information or the value of the LFNST index information being 0, the LFNST index information is notified by a signal.

20. A method for transmitting image data, the method comprising the steps of: Obtain a bitstream for the image, wherein the bitstream is generated based on the following operations: deriving residual samples for the current block, deriving transform coefficients for the current block based on the residual samples, generating residual information for the current block based on the transform coefficients, and encoding image information including the residual information; and Send the data including the bit stream, and The image information includes at least one of the following: Multi-Transform Selection (MTS) index information, Low-Frequency Inseparable Transform (LFNST) index information, or Inseparable Main Transform (NSPT) index information.

Images