Transform-based video coding method and apparatus thereof

Abstract

The invention relates to a transform-based video coding method and an apparatus therefor. A video decoding method according to the present document includes a step of deriving transform coefficients for a current block based on residual information, in which the step of deriving transform coefficients includes a step of deriving a return-to-zero block indicating a region in which valid transform coefficients may be present in the current block, wherein the return-to-zero block is derived on the basis of flag information indicating whether multiple transform selection (MTS) in which a plurality of transform cores are used can be applied to the current block.

Description

[0001] This application is a divisional application of the patent application with application number 202080058618.7 (PCT / KR2020 / 010487) filed on February 18, 2022, with an international application date of August 7, 2020, and the invention name is “Transformation-based video coding method and device thereof”. Technical Field

[0002] The present disclosure generally relates to image coding technology, and more particularly, to a transformation-based image coding method and device in an image coding system. Background Art

[0003] Recently, demand for high-resolution and high-quality images and videos, such as ultra-high-definition (HUD) images and 4K, 8K, or even higher-resolution video, has been increasing in various fields. As image and video data becomes higher in resolution and quality, the amount of information or number of bits required to transmit increases compared to existing image and video data. Consequently, using media such as existing wired or wireless broadband lines to transmit image data, or using existing storage media to store image and video data, increases transmission and storage costs.

[0004] Furthermore, interest and demand for immersive media such as virtual reality (VR), artificial reality (AR) content, or holograms have recently increased, and broadcasting of images and videos, such as game images, whose image characteristics differ from those of real images, has increased.

[0005] Therefore, in order to efficiently compress and transmit or store and play back information of high-resolution and high-quality images and videos having such various characteristics, efficient image and video compression technology is required. Summary of the Invention

[0006] Technical issues

[0007] The technical aspect of the present disclosure is to provide a method and apparatus for improving image coding efficiency.

[0008] Another technical aspect of the present disclosure is to provide a method and apparatus for improving residual coding efficiency.

[0009] Yet another technical aspect of the present disclosure is to provide a method and apparatus for improving residual coding efficiency by coding transform coefficients based on high-frequency zeroing.

[0010] Yet another technical aspect of the present disclosure is to provide a method and apparatus for improving efficiency of image coding in which high frequency zeroing is performed based on multiple transform selections.

[0011] Yet another technical aspect of the present disclosure is to provide a method and apparatus for coding an image, which can reduce data loss when performing high-frequency zeroing.

[0012] Another technical aspect of the present disclosure is to provide a method and apparatus for deriving a context model for last significant transform coefficient position information based on a current block size when encoding transform coefficients for a current block (or current transform block) based on high-frequency zeroing.

[0013] Technical Solution

[0014] According to an embodiment of the present disclosure, an image decoding method performed by a decoding device is provided. The method includes deriving transform coefficients for a current block based on residual information, wherein deriving the transform coefficients includes deriving a zero-out block indicating an area where valid transform coefficients may exist in the current block, and deriving the zero-out block based on flag information indicating whether multiple transform selection (MTS) using multiple transform kernels is applicable to the current block.

[0015] When MTS is applied, the width or height of the return-to-zero block may be set to 16, and when MTS is not applied, the width or height of the return-to-zero block may be set to 32 or less.

[0016] When flag information indicating whether sub-block transform for performing transform on a partitioned coding unit is applied to the current block is 1, the width or height of the zeroing block may be 16.

[0017] When the height of the partition sub-block is less than 64 and the width of the sub-block is 32, the width of the zeroing block may be set to 16.

[0018] When the width of the partition sub-block is less than 64 and the height of the sub-block is 32, the height of the zeroing block may be set to 16.

[0019] The transform kernel may be derived based on the partition direction of the current block and the position of the sub-block to which the transform is applied.

[0020] The residual information may include last significant coefficient prefix information, and a maximum value of the last significant coefficient prefix information may be derived based on the size of the return-to-zero block.

[0021] According to another embodiment of the present disclosure, an image encoding method performed by an encoding device is provided. The method includes deriving residual samples for a current block and deriving transform coefficients based on the residual samples for the current block, wherein deriving the transform coefficients may include deriving a zeroing block indicating an area where valid transform coefficients may exist in the current block based on whether multiple transform selection (MTS) using multiple transform kernels is applied to the current block.

[0022] According to still another embodiment of the present disclosure, a digital storage medium storing image data including encoded image information and a bit stream generated according to an image encoding method performed by an encoding device may be provided.

[0023] According to yet another embodiment of the present disclosure, a digital storage medium may be provided that stores image data including encoded image information and a bit stream so that a decoding device performs an image decoding method.

[0024] Beneficial effects

[0025] According to the present disclosure, the overall image / video compression efficiency can be improved.

[0026] According to the present disclosure, the efficiency of residual coding can be improved.

[0027] According to the present disclosure, the efficiency of residual coding can be improved by coding transform coefficients based on high-frequency zeroing.

[0028] According to the present disclosure, the efficiency of image coding in which high-frequency zeroing is performed based on multiple transform selections can be improved.

[0029] According to the present disclosure, data loss when performing high-frequency zeroing can be reduced, thereby improving the efficiency of image coding.

[0030] The effects that can be achieved through the specific examples of the present disclosure are not limited to the effects listed above. For example, there may be various technical effects that can be understood or derived from the present disclosure by a person of ordinary skill in the relevant field. Therefore, the specific effects of the present disclosure are not limited to the effects explicitly described in the present disclosure, and may include various effects that can be understood or derived from the technical features of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 An example of a video / image coding system to which the present disclosure is applicable is schematically illustrated.

[0032] Figure 2 is a diagram schematically illustrating the configuration of a video / image encoding device to which the present disclosure is applicable.

[0033] Figure 3 is a diagram schematically illustrating a configuration of a video / image decoding device to which the present disclosure is applicable.

[0034] Figure 4 The multiple transformation technology according to the embodiment of the present disclosure is schematically illustrated.

[0035] Figure 5 Illustrated is an MTS applied to sub-block transform according to an example of the present disclosure.

[0036] Figure 6 Illustrated is 32-point zeroing applied to sub-block transform according to an example of the present disclosure.

[0037] Figure 7 is a flowchart illustrating the operation of a video decoding apparatus according to an embodiment of the present disclosure.

[0038] Figure 8 is a flowchart illustrating a process for deriving transform coefficients by a video decoding apparatus according to an embodiment of the present disclosure.

[0039] Figure 9 is a flowchart illustrating the operation of a video encoding apparatus according to an embodiment of the present disclosure.

[0040] Figure 10 is a flowchart illustrating a process for encoding transform coefficients and information by a video encoding apparatus according to an embodiment of the present disclosure.

[0041] Figure 11 The structure of a content streaming system to which the present disclosure is applied is illustrated. DETAILED DESCRIPTION

[0042] This document can be modified in various ways and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit this document to a specific embodiment. The terms commonly used in this specification are used to describe specific embodiments rather than to limit the technical spirit of this document. Unless otherwise clearly indicated in the context, singular expressions include plural expressions. Terms such as "including" or "having" in this specification should be understood to indicate the presence of characteristics, numbers, steps, operations, elements, components, or combinations thereof described in this specification, without excluding the possibility of the presence or addition of one or more characteristics, numbers, steps, operations, elements, components, or combinations thereof.

[0043] In addition, to facilitate descriptions related to different feature functions, the elements in the drawings described in this document are illustrated independently. This does not mean that each element is implemented as separate hardware or separate software. For example, at least two elements can be combined to form a single element, or a single element can be divided into multiple elements. Embodiments in which elements are combined and / or separated are also included in the scope of the rights of this document unless it deviates from the essence of this document.

[0044] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Hereinafter, in the accompanying drawings, the same reference numerals are used for the same elements, and redundant descriptions of the same elements may be omitted.

[0045] This document relates to video / image coding. For example, the methods and examples disclosed in this document may relate to the Versatile Video Coding (VVC) standard (ITU-T Recommendation H.266), the next-generation video / image coding standard after VVC, or other video coding-related standards (for example, the High Efficiency Video Coding (HEVC) standard (ITU-T Recommendation H.265), the Essential Video Coding (EVC) standard, the AVS2 standard, etc.).

[0046] In this document, various embodiments related to video / image coding may be provided, and unless otherwise specified, the embodiments may be combined with each other and performed.

[0047] In this document, video may refer to a collection of images over time. Generally, a picture refers to a unit of image that represents a specific time region, and a slice / tile is a unit that constitutes a portion of a picture. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles.

[0048] A pixel or picture element (pel) can refer to the smallest unit that constitutes a picture (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luma component or only the pixel / pixel value of the chroma component.

[0049] A unit can represent the basic unit of image processing. A unit can include at least one of a specific region and information related to the region. A unit can include a luma block and two chroma (e.g., CB, CR) blocks. Depending on the situation, terms such as unit and block, region, etc. can be used interchangeably. In general, an MxN block can include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0050] In this document, the terms " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" may mean "A and / or B". In addition, "A, B" may mean "A and / or B". In addition, "A / B / C" may mean "at least one of A, B, and / or C". In addition, "A / B / C" may mean "at least one of A, B, and / or C".

[0051] Additionally, in this document, the term "or" should be interpreted as meaning "and / or." For example, the expression "A or B" may include 1) "only A," 2) "only B," and / or 3) "both A and B." In other words, the term "or" in this document should be interpreted as meaning "additionally or alternatively."

[0052] In the present disclosure, “at least one of A and B” may mean “only A”, “only B”, or “both A and B”. In addition, in the present disclosure, the expression “at least one of A or B” or “at least one of A and / or B” may be interpreted as “at least one of A and B”.

[0053] In addition, in the present disclosure, “at least one of A, B, and C” may mean “only A,” “only B,” “only C,” or “any combination of A, B, and C.” In addition, “at least one of A, B, or C” or “at least one of A, B, and / or C” may mean “at least one of A, B, and C.”

[0054] Additionally, parentheses used in this disclosure may refer to "for example." Specifically, when "prediction (intra-frame prediction)" is indicated, this may mean that "intra-frame prediction" is presented as an example of "prediction." That is, "prediction" in this disclosure is not limited to "intra-frame prediction," and "intra-frame prediction" may be presented as an example of "prediction." Additionally, when "prediction (i.e., intra-frame prediction)" is indicated, this may also mean that "intra-frame prediction" is presented as an example of "prediction."

[0055] Technical features described individually in one drawing in the present disclosure may be implemented individually or may be implemented simultaneously.

[0056] Figure 1 An example of a video / image coding system to which embodiments of this document can be applied is schematically illustrated.

[0057] Reference Figure 1 The video / image coding system may include a first device (source device) and a second device (receiver device). The source device may transmit the encoded video / image information or data to the receive device in the form of a file or stream transmission via a digital storage medium or a network.

[0058] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0059] A video source can obtain a video / image through a process of capturing, synthesizing, or generating a video / image. A video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured videos / images, etc. The video / image generation device may include, for example, a computer, a tablet computer, or a smartphone, and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capture process may be replaced by a process of generating relevant data.

[0060] An encoding device encodes input video / images. It performs a series of processes, such as prediction, transform, and quantization, for compression and coding efficiency. The encoded data (encoded video / image information) is output as a bitstream.

[0061] The transmitter can transmit the encoded video / image information or data, output as a bitstream, to a receiver in a receiving device via a digital storage medium or network in the form of a file or streaming. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter can include components for generating a media file in a predetermined file format and can also include components for transmitting via a broadcast / communication network. The receiver can receive / extract the bitstream and transmit the received / extracted bitstream to a decoding device.

[0062] The decoding device may decode a video / image by performing a series of processes such as dequantization, inverse transformation, prediction, etc. corresponding to the operation of the encoding device.

[0063] The renderer can render the decoded video / image, and the rendered video / image can be displayed on a display.

[0064] Figure 2 The figure schematically illustrates the configuration of a video / image encoding device to which the present document can be applied. Hereinafter, the so-called video encoding device may include an image encoding device.

[0065] Reference Figure 2The encoding device 200 may include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. Depending on the embodiment, the image partitioner 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filter 260 described above may be comprised of one or more hardware components (e.g., an encoder chipset or processor). Furthermore, the memory 270 may include a decoded picture buffer (DPB) and may be comprised of a digital storage medium. The hardware components may also include the memory 270 as an internal / external component.

[0066] The image partitioner 210 partitions the input image (or picture, or frame) input to the encoding device 200 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). In this case, starting with a coding tree unit (CTU) or a largest coding unit (LCU), the coding units may be recursively partitioned according to a quadtree, binary tree, ternary tree (QTBTTT) structure. For example, a coding unit may be divided into multiple coding units of increasing depth based on a quadtree, binary tree, and / or ternary tree structure. In this case, for example, the quadtree structure may be applied first, followed by the binary tree and / or ternary tree structure. Alternatively, the binary tree structure may be applied first. The coding process according to this document may be performed based on the final coding unit that has not been further partitioned. In this case, the largest coding unit may be directly used as the final coding unit based on coding efficiency according to image characteristics. Alternatively, the coding unit may be recursively partitioned into coding units of increasing depth as needed, so that the optimally sized coding unit can be used as the final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction, which will be described later. As another example, a processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the final coding unit described above. A prediction unit may be a unit for sample prediction, and a transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal based on the transform coefficients.

[0067] Depending on the situation, terms such as unit and block, region, etc. can be used interchangeably. In general, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or pixel value, and can represent only the pixel / pixel value of the luma component or only the pixel / pixel value of the chroma component. Sample can be used as a term corresponding to a pixel or pel of a picture (or image).

[0068] Subtractor 231 subtracts the prediction signal (prediction block, prediction sample, or prediction sample array) output from predictor 220 from the input image signal (original block, original sample, or original sample array) to generate a residual signal (residual block, residual sample, or residual sample array). The generated residual signal is sent to transformer 232. Predictor 220 may perform prediction on a processing target block (hereinafter referred to as the "current block") and may generate a prediction block including prediction samples of the current block. Predictor 220 may determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. As discussed later in the description of each prediction mode, the predictor may generate various information related to the prediction, such as prediction mode information, and send the generated information to entropy encoder 240. The information regarding the prediction may be encoded in entropy encoder 240 and output in the form of a bitstream.

[0069] The intra-frame predictor 222 can predict the current block by referring to samples in the current picture. Depending on the prediction mode, the reference sample can be located near the current block or separated from the current block. In intra-frame prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. The non-directional mode can include, for example, a DC mode and a planar mode. Depending on the level of detail of the prediction direction, the directional mode can include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is merely an example, and more or fewer directional prediction modes can be used depending on the settings. The intra-frame predictor 222 can determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[0070] The inter-frame predictor 221 can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector in a reference picture. To reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information can include a motion vector and a reference picture index. It can also include information about the inter-frame prediction direction (e.g., L0 prediction, L1 prediction, or Bi prediction). In the case of inter-frame prediction, neighboring blocks can include spatially neighboring blocks in the current picture and temporally neighboring blocks in a reference picture. The reference picture containing the reference block and the reference picture containing the temporally neighboring block can be the same or different. Temporally neighboring blocks can be referred to as collocated reference blocks, collocated CUs (colCUs), etc., and the reference picture containing temporally neighboring blocks can be referred to as collocated pictures (colPics). For example, the inter-frame predictor 221 can configure a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index for the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 221 can use the motion information of the neighboring blocks as the motion information of the current block. In skip mode, unlike merge mode, a residual signal cannot be transmitted. In motion information prediction (motion vector prediction, MVP) mode, the motion vector of the neighboring blocks can be used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0071] The predictor 220 can generate a prediction signal based on various prediction methods. For example, the predictor can apply intra prediction or inter prediction to predict a block, and can also apply intra prediction and inter prediction at the same time. This can be referred to as combined inter and intra prediction (CIIP). In addition, the predictor can perform prediction on the block based on an intra block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or the palette mode can be used for content image / video encoding such as screen content coding (SCC) for games. Although IBC basically performs prediction in the current block, it can be performed similarly to inter prediction in terms of deriving a reference block in the current block. That is, IBC can use at least one of the inter prediction techniques described in the present disclosure.

[0072] The prediction signal generated by the inter-frame predictor 221 and / or the intra-frame predictor 222 can be used to generate a reconstructed signal or a residual signal. The transformer 232 can generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include discrete cosine transform (DCT), discrete sine transform (DST), graph-based transform (GBT), or conditional nonlinear transform (CNT). Here, GBT means a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process can be applied to square pixel blocks of the same size or can be applied to blocks of variable size instead of square blocks.

[0073] The quantizer 233 quantizes the transform coefficients and transmits them to the entropy encoder 240. The entropy encoder 240 encodes the quantized signals (information about the quantized transform coefficients) and outputs the encoded signals in a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 rearranges the block-quantized transform coefficients into a one-dimensional vector form based on the coefficient scanning order and generates information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoder 240 can implement various encoding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), and others. The entropy encoder 240 can encode information necessary for video / image reconstruction, in addition to the quantized transform coefficients (e.g., syntax element values), together or separately. The encoded information (e.g., encoded video / image information) may be transmitted or stored in a bitstream on a Network Abstraction Layer (NAL) unit basis. The video / image information may also include information about various parameter sets, such as the Adaptation Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS). Furthermore, the video / image information may include general constraint information. In the present disclosure, information and / or syntax elements transmitted / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded using the encoding process described above and included in the bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. Here, the network may include a broadcast network, a communication network, and / or the like, while the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and the like. The transmitter (not shown) that transmits the signal output from the entropy encoder 240 and / or the storage device (not shown) that stores it may be configured as an internal / external component of the encoding device 200, or the transmitter may be included in the entropy encoder 240.

[0074] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, by applying dequantization and inverse transformation to the quantized transform coefficients via the dequantizer 234 and the inverse transformer 235, a residual signal (residual block or residual sample) can be reconstructed. The adder 250 adds the reconstructed residual signal to the prediction signal output from the predictor 220, thereby generating a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample, or reconstructed sample array). When the processing target block has no residual, as in the case of applying skip mode, the prediction block can be used as the reconstructed block. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current block and, as described later, can be used for inter-frame prediction of the next picture through filtering.

[0075] At the same time, luminance mapping and chroma scaling (LMCS) can be applied during picture encoding and / or reconstruction.

[0076] The filter 260 can improve the subjective / objective video quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture can be stored in the memory 270, specifically in the DPB of the memory 270. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. As discussed later in the description of each filtering method, the filter 260 can generate various information related to filtering and send the generated information to the entropy encoder 290. The information about filtering can be encoded in the entropy encoder 290 and output in the form of a bitstream.

[0077] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter-frame predictor 280. This allows the encoding device to avoid prediction mismatches in the encoding device 200 and the decoding device when applying inter-frame prediction, and improves coding efficiency.

[0078] The memory 270 DPB can store the modified reconstructed picture so that it can be used as a reference picture in the inter-frame predictor 221. The memory 270 can store motion information of blocks in the current picture from which motion information has been derived (or encoded), and / or motion information of blocks in reconstructed pictures. The stored motion information can be sent to the inter-frame predictor 221 to be used as motion information for neighboring blocks or motion information for temporally neighboring blocks. The memory 270 can store reconstructed samples of the reconstructed blocks in the current picture and send them to the intra-frame predictor 222.

[0079] Figure 3 This is a diagram schematically illustrating the configuration of a video / image decoding device to which this document can be applied.

[0080] Reference Figure 3 , the video decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 331 and an intra-frame predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. According to an embodiment, the entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 described above may be composed of one or more hardware components (e.g., a decoder chipset or processor). In addition, the memory 360 may include a decoded picture buffer (DPB) and may be composed of a digital storage medium. The hardware components may also include the memory 360 as an internal / external component.

[0081] When a bit stream including video / image information is input, the decoding device 300 can Figure 2 The image is reconstructed correspondingly to the processing of the video / image information in the encoding device. For example, the decoding device 300 can derive the unit / block based on the information related to the block segmentation obtained from the bit stream. The decoding device 300 can perform decoding by using the processing unit applied in the encoding device. Therefore, the decoding processing unit can be, for example, a coding unit, which can be divided into a quadtree structure, a binary tree structure and / or a ternary tree structure using a coding tree unit or a maximum coding unit. One or more transform units can be derived from the coding unit. In addition, the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproducer.

[0082] The decoding device 300 can receive the data from the Figure 2The received signal is output by the encoding device, and the entropy decoder 310 can decode the received signal. For example, the entropy decoder 310 can parse the bitstream to derive information required for image reconstruction (or picture reconstruction) (e.g., video / image information). The video / image information may also include information about various parameter sets such as the Adaptation Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS). Furthermore, the video / image information may include general constraint information. The decoding device can further decode the picture based on the parameter set information and / or general constraint information. In this disclosure, the signaled / received information and / or syntax elements described later can be decoded through a decoding process and obtained from the bitstream. For example, the entropy decoder 310 can decode the information in the bitstream based on coding methods such as Exponential Golomb coding, CAVLC, and CABAC, and can output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of the transform coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, uses information about the decoded target syntax element and decoded information of neighboring and decoding target blocks, or information about symbols / bins decoded in the previous step, to determine a context model. Based on the determined context model, the method predicts bin generation probabilities and performs arithmetic decoding on the bins to generate symbols corresponding to each syntax element value. After determining the context model, the CABAC entropy decoding method updates the context model using information about the symbol / bin decoded from the context model for the next symbol / bin. Information related to prediction, as part of the information decoded in the entropy decoder 310, may be provided to the predictor 330, and information related to the residual, i.e., quantized transform coefficients and associated parameter information, which has been entropy decoded in the entropy decoder 310, may be input to the dequantizer 321. Furthermore, information related to filtering, as part of the information decoded in the entropy decoder 310, may be provided to the filter 350. A receiver (not shown) that receives the signal output from the encoding device may further constitute the decoding device 300 as an internal / external element, and the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding device according to the present disclosure may be referred to as a video / image / picture encoding device, and the decoding device may be classified into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, a predictor 330, an adder 340, a filter 350, and a memory 360.

[0083] The dequantizer 321 may dequantize the quantized transform coefficients to output the transform coefficients. The dequantizer 321 may rearrange the quantized transform coefficients into a two-dimensional block. In this case, the rearrangement may be performed based on the order of coefficient scanning performed in the encoding device. The dequantizer 321 may dequantize the quantized transform coefficients using quantization parameters (e.g., quantization step size information) to obtain the transform coefficients.

[0084] The inverse transformer 322 obtains a residual signal (residual block, residual sample array) by performing inverse transformation on the transformation coefficients.

[0085] The predictor may perform prediction on the current block and generate a prediction block including prediction samples for the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on the information about prediction output from the entropy decoder 310, and specifically may determine the intra / inter prediction mode.

[0086] The predictor can generate a prediction signal based on various prediction methods. For example, the predictor can apply intra prediction or inter prediction to predict a block, and can also apply intra prediction and inter prediction at the same time. This can be called combined inter and intra prediction (CIIP). In addition, the predictor can perform intra block copying (IBC) to predict the block. Intra block copying can be used for content image / video coding such as screen content coding (SCC) for games. Although IBC basically performs prediction in the current block, it can be performed similarly to inter prediction in terms of deriving a reference block in the current block. That is, IBC can use at least one of the inter prediction techniques described in this disclosure.

[0087] The intra-frame predictor 331 can predict the current block by referencing samples in the current picture. Depending on the prediction mode, the referenced samples may be located in the neighborhood of the current block or far away from the current block. In intra-frame prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. The intra-frame predictor 331 may determine the prediction mode to be applied to the current block by using the prediction modes applied to the neighboring blocks.

[0088] The inter-frame predictor 332 can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector in a reference picture. To reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information can include a motion vector and a reference picture index. Motion information can also include information about the inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-frame prediction, neighboring blocks can include spatially neighboring blocks in the current picture and temporally neighboring blocks in the reference picture. For example, the inter-frame predictor 332 can configure a motion information candidate list based on the neighboring blocks and derive the motion vector and / or reference picture index for the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and prediction information can include information indicating the inter-frame prediction mode used for the current block.

[0089] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor 330. When there is no residual when processing the target block as in the case of applying the skip mode, the prediction block can be used as the reconstructed block.

[0090] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, may be output through filtering as described below, or may be used for inter prediction of the next picture.

[0091] In addition, luma mapping and chroma scaling (LMCS) can be applied to the picture decoding process.

[0092] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and store the modified reconstructed image in the memory 360, specifically, in the DPB of the memory 360. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc.

[0093] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter-frame predictor 332. The memory 360 can store motion information of blocks in the current picture from which motion information has been derived (or decoded), and / or motion information of blocks in the reconstructed picture. The stored motion information can be sent to the inter-frame predictor 260 to be used as motion information of neighboring blocks or motion information of temporally neighboring blocks. The memory 360 can store reconstructed samples of the reconstructed blocks in the current picture and send them to the intra-frame predictor 331.

[0094] In this specification, the examples described in the predictor 330, dequantizer 321, inverse transformer 322, and filter 350 of the decoding device 300 may be similarly or correspondingly applied to the predictor 220, dequantizer 234, inverse transformer 235, and filter 260 of the encoding device 200, respectively.

[0095] As described above, when performing video coding, prediction is performed to improve compression efficiency. A prediction block including prediction samples of the current block, i.e., a target coding block, can be generated by prediction. In this case, the prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is derived similarly in the encoding device and the decoding device. The encoding device can improve image coding efficiency by signaling information (residual information) about the residual between the original block, rather than the original sample values ​​of the original block themselves, and the prediction block to the decoding device. The decoding device can derive a residual block including residual samples based on the residual information, can generate a reconstructed block including reconstructed samples by adding the residual block to the prediction block, and can generate a reconstructed picture including the reconstructed block.

[0096] Residual information can be generated through a transformation and quantization process. For example, the encoding device can derive a residual block between the original block and the prediction block, derive transform coefficients by performing a transformation process on the residual samples (residual sample array) included in the residual block, derive quantized transform coefficients by performing a quantization process on the transform coefficients, and can signal the relevant residual information to the decoding device (via a bitstream). In this case, the residual information may include information such as value information, position information, a transformation scheme, a transform kernel, and a quantization parameter for the quantized transform coefficients. The decoding device can perform an inverse quantization / inverse transformation process based on the residual information and derive residual samples (or residual blocks). The decoding device can generate a reconstructed picture based on the prediction block and the residual block. In addition, the encoding device can derive a residual block for inter-frame prediction reference of subsequent pictures by performing inverse quantization / inverse transformation on the quantized transform coefficients, and can generate a reconstructed picture.

[0097] Figure 4 The multi-conversion technology according to the embodiment of the present disclosure is schematically illustrated.

[0098] Reference Figure 4 , the converter can correspond to the above Figure 2 The converter in the encoding device of , and the inverse converter may correspond to the above Figure 2 The inverse transformer in the encoding device, or corresponding to Figure 3 An inverse transformer in a decoding device.

[0099] The transformer may derive (primary) transform coefficients by performing a primary transform based on the residual samples (residual sample array) in the residual block (S410). This primary transform may be referred to as a core transform. Herein, the primary transform may be based on multi-transform selection (MTS), and when a multi-transform is applied as the primary transform, it may be referred to as a multi-core transform.

[0100] Multi-core transform may refer to a transform method that additionally uses discrete cosine transform (DCT) type 2 and discrete sine transform (DST) type 7, DCT type 8, and / or DST type 1. In other words, multi-core transform may refer to a transform method that transforms a spatial domain residual signal (or residual block) into frequency domain transform coefficients (or primary transform coefficients) based on multiple transform kernels selected from DCT type 2, DST type 7, DCT type 8, and DST type 1. Herein, from the perspective of the transformer, the primary transform coefficients may be referred to as time transform coefficients.

[0101] That is, when a conventional transform method is applied, transform coefficients can be generated by applying a transform from the spatial domain to the frequency domain to the residual signal (or residual block) based on DCT type 2. However, when a multi-core transform is applied, transform coefficients (or primary transform coefficients) can be generated by applying a transform from the spatial domain to the frequency domain to the residual signal (or residual block) based on DCT type 2, DST type 7, DCT type 8, and / or DST type 1. Here, DCT type 2, DST type 7, DCT type 8, and DST type 1 may be referred to as transform types, transform kernels, or transform cores. These DCT / DST types may be defined based on basis functions.

[0102] If a multi-core transform is performed, a vertical transform kernel and a horizontal transform kernel for the target block may be selected from the transform kernels. A vertical transform may be performed on the target block based on the vertical transform kernel, and a horizontal transform may be performed on the target block based on the horizontal transform kernel. Here, the horizontal transform may refer to a transform for the horizontal component of the target block, while the vertical transform may refer to a transform for the vertical component of the target block. The vertical transform kernel / horizontal transform kernel may be adaptively determined based on the prediction mode and / or transform index of the target block (CU or sub-block) including the residual block.

[0103] Furthermore, according to one example, if the primary transform is performed by applying MTS, the mapping relationship of the transform kernel can be set by setting a specific basis function to a predetermined value and combining the basis functions to be applied in the vertical transform or the horizontal transform. For example, when the horizontal transform kernel is expressed as trTypeHor and the vertical transform kernel is expressed as trTypeVer, a trTypeHor or trTypeVer value of 0 can be set to DCT2, a trTypeHor or trTypeVer value of 1 can be set to DCT-7, and a trTypeHor or trTypeVer value of 2 can be set to DCT-8.

[0104] In this case, MTS index information may be encoded and signaled to a decoding device to indicate any one of a plurality of transform kernel sets. For example, MTS index 0 may indicate that both trTypeHor and trTypeVer values ​​are 0, MTS index 1 may indicate that both trTypeHor and trTypeVer values ​​are 1, MTS index 2 may indicate that trTypeHor value is 2 and trTypeVer value is 1, MTS index 3 may indicate that trTypeHor value is 1 and trTypeVer value is 2, and MTS index 4 may indicate that both trTypeHor and trTypeVer value is 2.

[0105] In one example, the transformation kernel set according to the MTS index information is illustrated in the following table.

[0106] [Table 1]

[0107]

[0108] The transformer may derive modified (secondary) transform coefficients by performing a secondary transform based on the (primary) transform coefficients (S420). A primary transform is a transform from the spatial domain to the frequency domain, while a secondary transform refers to a transform performed using the correlation between the (primary) transform coefficients in a more compressed expression. The secondary transform may include a non-separable transform. In this case, the secondary transform may be referred to as a non-separable secondary transform (NSST) or a pattern-dependent non-separable secondary transform (MDNSST). A non-separable secondary transform may mean generating modified transform coefficients (or secondary transform coefficients) for the residual signal by performing a secondary transform on the (primary) transform coefficients derived from the primary transform based on a non-separable transform matrix. In this case, vertical and horizontal transforms may not be applied separately to the (primary) transform coefficients (or horizontal and vertical transforms may not be applied independently), but instead the transforms may be applied all at once based on the non-separable transform matrix. In other words, a non-separable secondary transform refers to a transform method in which the vertical and horizontal components of the (primary) transform coefficients are not separated. For example, a two-dimensional signal (transform coefficient) is rearranged into a one-dimensional signal in a certain direction (e.g., row-major or column-major). The modified transform coefficients (or secondary transform coefficients) are then generated based on the non-separable transform matrix. For example, in row-major order, M×N blocks are arranged in a row in the order of the first row, the second row, ..., and the Nth row. In column-major order, M×N blocks are arranged in a row in the order of the first column, the second column, ..., and the Nth column. The non-separable secondary transform can be applied to the top left region of a block containing (primary) transform coefficients (hereinafter referred to as a transform coefficient block). For example, if the width (W) and height (H) of the transform coefficient block are both equal to or greater than 8, an 8×8 non-separable secondary transform can be applied to the top left 8×8 region of the transform coefficient block. In addition, if the width (W) and height (H) of the transform coefficient block are both equal to or greater than 4, and the width (W) or height (H) of the transform coefficient block is less than 8, a 4×4 non-separable sub-transform may be applied to the top left min(8,W)×min(8,H) area of ​​the transform coefficient block. However, this embodiment is not limited thereto, and for example, even if the condition that only the width (W) or height (H) of the transform coefficient block is equal to or greater than 4 is satisfied, a 4×4 non-separable sub-transform may be applied to the top left min(8,W)×min(8,H) area of ​​the transform coefficient block.

[0109] The transformer may perform a non-separable secondary transform based on the selected transform kernel and may obtain modified (secondary) transform coefficients. As described above, the modified transform coefficients may be derived as transform coefficients quantized by a quantizer, encoded and signaled to a decoding device, and transmitted to a dequantizer / inverse transformer in the encoding device.

[0110] At the same time, as described above, if the secondary transform is omitted, the (primary) transform coefficients as the output of the primary (separable) transform can be derived as transform coefficients quantized by the quantizer as described above, and can be encoded and signaled to the decoding device and transmitted to the dequantizer / inverse transformer in the encoding device.

[0111] The inverse transformer may perform the series of processes in the reverse order to the order in which the series of processes have been performed in the above-mentioned transformer. The inverse transformer may receive the (dequantized) transform coefficients and derive (primary) transform coefficients by performing a secondary (inverse) transform (S450), and may obtain a residual block (residual samples) by performing a primary (inverse) transform on the (primary) transform coefficients (S460). In this regard, from the perspective of the inverse transformer, the primary transform coefficients may be referred to as modified transform coefficients. As described above, the encoding device and the decoding device may generate a reconstructed block based on the residual block and the prediction block, and may generate a reconstructed picture based on the reconstructed block.

[0112] The decoding device may further include a secondary inverse transform application determiner (or an element for determining whether to apply a secondary inverse transform) and a secondary inverse transform determiner (or an element for determining a secondary inverse transform). The secondary inverse transform application determiner may determine whether to apply a secondary inverse transform. For example, the secondary inverse transform may be NSST or RST, and the secondary inverse transform application determiner may determine whether to apply a secondary inverse transform based on a secondary transform flag obtained by parsing the bitstream. In another example, the secondary inverse transform application determiner may determine whether to apply a secondary inverse transform based on a transform coefficient of a residual block.

[0113] A secondary inverse transform determiner may determine a secondary inverse transform. In this case, the secondary inverse transform determiner may determine the secondary inverse transform to be applied to the current block based on an NSST (or RST) transform set specified according to an intra-frame prediction mode. In embodiments, the secondary transform determination method may be determined based on the primary transform determination method. Various combinations of the primary transform and the secondary transform may be determined based on the intra-frame prediction mode. In addition, in an example, the secondary inverse transform determiner may determine the area to which the secondary inverse transform is applied based on the size of the current block.

[0114] Meanwhile, as described above, if the secondary (inverse) transform is omitted, the (dequantized) transform coefficients may be received, the primary (separable) inverse transform may be performed, and a residual block (residual samples) may be obtained. As described above, the encoding device and the decoding device may generate a reconstructed block based on the residual block and the prediction block, and may generate a reconstructed picture based on the reconstructed block.

[0115] Meanwhile, in the present disclosure, a reduced secondary transform (RST) in which the size of a transformation matrix (kernel) is reduced may be applied in the concept of NSST in order to reduce the amount of calculation and memory required for the non-separable secondary transform.

[0116] The transform kernel, transform matrix, and coefficients comprising the transform kernel matrix, i.e., kernel coefficients or matrix coefficients, described in this disclosure can be expressed using 8 bits. This can be a requirement for implementation in decoding and encoding devices, and can reduce the amount of memory required to store the transform kernel, as performance degradation can be reasonably accommodated compared to existing 9-bit or 10-bit representations. Furthermore, expressing the kernel matrix using 8 bits allows for the use of smaller multipliers and may be more suitable for single instruction, multiple data (SIMD) instructions for optimal software implementation.

[0117] In this specification, the term "RST" may refer to a transform performed on the residual samples of a target block based on a transform matrix whose size is reduced according to a reduction factor. When performing a downscale transform, the size of the transform matrix is ​​reduced, thereby reducing the amount of computation required for the transform. In other words, RST can be used to address the computational complexity issues that arise when performing non-separable transforms or transforms of large blocks.

[0118] RST may be referred to by various terms such as downscaling transform, downscaling sub-transform, reduced transform, simplified transform, simple transform, etc., and the names that RST may be referred to are not limited to the examples listed. Alternatively, since RST is performed mainly in a low-frequency region including non-zero coefficients in a transform block, it may be referred to as a low-frequency non-separable transform (LFNST).

[0119] Meanwhile, when performing a secondary inverse transform based on an RST, the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include an inverse downscaled secondary transformer for deriving modified transform coefficients based on an inverse RST of the transform coefficients, and an inverse primary transformer for deriving residual samples of the target block based on an inverse primary transform of the modified transform coefficients. The inverse primary transform refers to an inverse transform of the primary transform applied to the residual. In the present disclosure, deriving transform coefficients based on a transform may refer to deriving transform coefficients by applying a transform.

[0120] Hereinafter, a reduced multi-transform technique (Reduced Adaptive Multi-Selection (or Set) (RMTS)) is described.

[0121] As described above, when a combination of multiple transforms (DCT-2, DST-7, DCT-8, DST-1, DCT-5, etc.) is selectively used for primary transform in a multi-transform technique (multi-transform set or adaptive multi-transform), the transform can be applied only to a predefined area to reduce complexity instead of performing the transform in all cases, thereby significantly reducing complexity in the worst case.

[0122] For example, when applying a primary transform to an M×M pixel block based on the aforementioned reduced transform (RT) method, calculations can be performed only on transform blocks of R×R blocks (M>=R) rather than obtaining an M×M transform block. As a result, non-zero significant coefficients exist only in the R×R region, and transform coefficients in other regions can be considered zero and not calculated. The following table illustrates three examples of reduced adaptive multi-transform (RAMT) using predefined reduced transform factor (R) values ​​for the size of the block to which the primary transform is applied.

[0123] [Table 2]

[0124]

[0125] According to one example, when applying the reduced multi-transform illustrated above, a reduced transform factor can be determined based on the primary transform. For example, when the primary transform is DCT2, the computation is simpler than other primary transforms, and therefore, the reduced transform may not be used for small blocks, or a relatively large R value may be used for small blocks, thereby minimizing the degradation of coding performance. For example, different reduced transform factors can be used for DCT2 and other transforms as follows.

[0126] [Table 3]

[0127]

[0128] As shown in Table 3, when the primary transform is DCT2, the size of the transform does not change when the size of the block to be transformed is 8×8 or 16×16, and the reduced size of the transform is limited to 32×32 when the size of the block is 32×32 or larger.

[0129] Alternatively, according to an example, when the flag value indicating whether MTS is applied is 0 (i.e., when DCT2 is applied for both the horizontal and vertical directions), for both (horizontal and vertical) directions, only 32 coefficients from the left and top may be left and the high-frequency components may be zeroed, that is, set to 0 (zeroing implementation 1).

[0130] For example, in a 64x64 transform unit (TU), the transform coefficients are only in the top left 32x32 area, in a 64x16 TU, the transform coefficients are only in the top left 32x16 area, and in an 8x64 TU, the transform coefficients are only in the top left 8x32 area. That is, there are transform coefficients corresponding to only up to a maximum length of 32 in both width and height.

[0131] This zeroing method may be applied only to a residual signal to which intra prediction is applied or may be applied only to a residual signal to which inter prediction is applied. Alternatively, the zeroing method may be applied to both a residual signal to which intra prediction is applied and a residual signal to which inter prediction is applied.

[0132] The change in transform block size, which can be expressed as the aforementioned zeroing or high-frequency zeroing, refers to the process of zeroing out (determining to zero) transform coefficients related to high frequencies of a certain value or greater in a (transform) block having a first width (or length) of W1 and a first height (or length) of H1. When high-frequency zeroing is applied, the transform coefficient values ​​of all transform coefficients in the (transform) block that are outside a low-frequency transform coefficient region configured based on a second width of W2 and a second height of H2 can be determined (set) to zero. The region outside the low-frequency transform coefficient region can be referred to as a high-frequency transform coefficient region. In this example, the low-frequency transform coefficient region can be a rectangular region extending from the top left of the (transform) block.

[0133] That is, high frequency zeroing may be defined as setting all transform coefficients at positions defined by an x-coordinate of w or greater and a y-coordinate of h or greater to 0, where the horizontal x-coordinate value of the top left position of the current transform block (TB) is set to 0 and its vertical y-coordinate value is set to 0 (and where the x-coordinate increases from left to right and the y-coordinate increases downward).

[0134] In this disclosure, specific terms or expressions are used to define specific information or concepts. For example, as described above, in this specification, the process of normalizing transform coefficients corresponding to frequencies of a specific value or greater in a (transform) block having a first width (or length) of W1 and a first height (or length) of H1 is defined as "high-frequency normalization," an area that has been normalized by high-frequency normalization is defined as a "high-frequency transform coefficient area," and an area that has not been normalized is defined as a "low-frequency transform coefficient area." To indicate the size of the low-frequency transform coefficient area, a second width (or length) of W2 and a second height (or length) of H2 are used.

[0135] However, "high-frequency zeroing" can be replaced with various terms such as high-frequency zeroing, high-frequency zeroing-out, high-frequency zeroing-out, high-frequency zero-out, and zero-out, and "high-frequency transform coefficient area" can be replaced with various terms such as high-frequency zeroing application area, high-frequency zeroing area, high-frequency area, high-frequency coefficient area, high-frequency zeroing area, and zeroing area, and "low-frequency transform coefficient area" can be replaced with various terms such as high-frequency zeroing non-application area, low-frequency area, low-frequency coefficient area, and restricted area. Therefore, specific terms or expressions used herein to define specific information or concepts need to be interpreted throughout the specification in view of various operations, functions, and effects according to the content indicated by the term, without limitation to the specification.

[0136] Alternatively, according to an example, the low-frequency transform coefficient region refers to the region remaining after high-frequency zeroing is performed or the region in which valid transform coefficients are left, that is, the region where non-zero transform coefficients may exist, and may be called a zeroing region or a zeroing block.

[0137] According to one example, when the flag value indicating whether MTS is applied is 1, that is, when a different transform (DST-7 or DCT-8) other than DCT2 is applied to the horizontal and vertical directions, the transform coefficient may remain only in the left top area, and the remaining area may be zeroed as follows (zeroing implementation 2).

[0138] - When the width (w) is equal to or greater than 2 n When, only from the left and w / 2 p The transform coefficients corresponding to the length of can be left, and the remaining transform coefficients can be fixed to 0 (returned to zero).

[0139] - When the height (h) is equal to or greater than 2 m When, only from the top with h / 2 q The transform coefficients corresponding to the length of can be left, and the remaining transform coefficients can be fixed to 0 (returned to zero).

[0140] Here, m, n, p, and q may be integers equal to or greater than 0, and may be specifically as follows.

[0141] 1) (m, n, p, q) = (5, 5, 1, 1)

[0142] 2) (m, n, p, q) = (4, 4, 1, 1)

[0143] In configuration 1), the transform coefficients remain only in the top left 16×16 region in a 32×16 TU, and the transform coefficients remain only in the top left 8×16 region in an 8×32 TU.

[0144] This zeroing method may be applied only to the residual signal to which intra prediction is applied or may be applied only to the residual signal to which inter prediction is applied. Alternatively, the zeroing method may be applied to both the residual signal to which intra prediction is applied and the residual signal to which inter prediction is applied.

[0145] Alternatively, according to another example, when the flag value indicating whether MTS is applied is 1, that is, when a different transform (DST-7 or DCT-8) other than DCT2 is applied to the horizontal and vertical directions, the transform coefficients may remain only in the left top area, and the remaining area may be zeroed as follows (zeroing implementation 3).

[0146] - When the height (h) is equal to or greater than the width (w) and equal to or greater than 2 n When only the left top w×(h / 2 p ) area may be left as is, and the remaining transform coefficients may be fixed to 0 (zeroed).

[0147] - When the width (w) is greater than the height (h) and is equal to or greater than 2 m When only the left top (w / 2 q The transform coefficients in the )×h region may be left, and the remaining transform coefficients may be fixed to 0 (zeroed).

[0148] Under the above conditions, when the height (h) and width (w) are the same, the vertical length is reduced (h / 2p), but the horizontal length can also be reduced (w / 2q).

[0149] Here, m, n, p, and q may be integers equal to or greater than 0, and may be specifically as follows.

[0150] 1) (m, n, p, q) = (4, 4, 1, 1)

[0151] 2) (m, n, p, q) = (5, 5, 1, 1)

[0152] In configuration 1), the transform coefficients remain only in the top left 16×16 region in a 32×16 TU, and the transform coefficients remain only in the top left 8×8 region in an 8×16 TU.

[0153] This zeroing method may be applied only to the residual signal to which intra prediction is applied or may be applied only to the residual signal to which inter prediction is applied. Alternatively, the zeroing method may be applied to both the residual signal to which intra prediction is applied and the residual signal to which inter prediction is applied.

[0154] In the foregoing embodiments, the transform coefficient region is limited depending on whether the flag value indicating whether MTS is applied is 0 or the flag value indicating whether MTS is applied is 1. According to one example, a combination of these embodiments is possible.

[0155] 1) Zeroing implementation method 1 + Zeroing implementation method 2

[0156] 2) Zeroing Implementation Method 1 + Zeroing Implementation Method 3

[0157] As mentioned in return-to-zero implementation 2 and return-to-zero implementation 3, the return-to-zero method can be applied only to the residual signal to which intra-frame prediction is applied or can be applied only to the residual signal to which inter-frame prediction is applied. Alternatively, the return-to-zero method can be applied to both the residual signal to which intra-frame prediction is applied and the residual signal to which inter-frame prediction is applied. Therefore, when the MTS flag is 1, the following table can be configured (when the MTS flag is 1, return-to-zero implementation 1 can be applied). Here, the MTS flag can also be configured to indicate the MTS index of the transform kernel used for the MTS. For example, an MTS index of 0 can indicate that return-to-zero implementation 1 is applied.

[0158] [Table 4]

[0159] Configuring Indexes Intra-frame prediction residual signal Inter-frame prediction residual signal 1 Do not use zeroing Do not use zeroing 2 Do not use zeroing Zeroing Implementation Method 2 3 Do not use zeroing Zeroing Implementation Method 3 4 Zeroing Implementation Method 2 Do not use zeroing 5 Zeroing Implementation Method 2 Zeroing Implementation Method 2 6 Zeroing Implementation Method 2 Zeroing Implementation Method 3 7 Zeroing Implementation Method 3 Do not use zeroing 8 Zeroing Implementation Method 3 Zeroing Implementation Method 2 9 Zeroing Implementation Method 3 Zeroing Implementation Method 3

[0160] In return-to-zero embodiment 1, return-to-zero embodiment 2, and return-to-zero embodiment 3, the area that inevitably includes the value 0 in the TU is clearly defined. That is, the area other than the left top area where the transform coefficient is allowed to exist is zeroed. Therefore, according to the embodiment, it can be configured to bypass the area where the transform coefficient has the value 0 as a result of entropy coding of the residual signal, rather than performing residual coding on it. For example, the following configuration is possible.

[0161] 1) In HEVC or VVC, a flag indicating whether a non-zero transform coefficient exists in a coefficient group (CG, which can be a 4x4 or 2x2 block depending on the shape of the sub-block and TU block and the luma component / chroma component) is coded (subblock_flag). Only when subblock_flag is 1 is the inside of the CG scanned and the coefficient level values ​​coded. Therefore, for CGs belonging to areas where zeroing is performed, subblock_flag can be set to a value of 0 by default instead of being coded.

[0162] 2) In HEVC or VVC, the position of the last coefficient in forward scan order (last_coefficient_position_x in the X direction and last_coefficient_position_y in the Y direction) is coded first. Typically, last_coefficient_position_x and last_coefficient_position_y can have maximum values ​​(TU width - 1) and maximum values ​​(TU height - 1), respectively. However, when the region where non-zero coefficients can exist is restricted due to zeroing, the maximum values ​​of last_coefficient_position_x and last_coefficient_position_y are also restricted. Therefore, the maximum values ​​of last_coefficient_position_x and last_coefficient_position_y can be restricted in view of zeroing and then coded. For example, when the binarization method applied to last_coefficient_position_x and last_coefficient_position_y is truncated unary binarization, the maximum length of the truncated unary code (the codeword length that last_coefficient_position_x and last_coefficient_position_y can have) can be reduced based on the adjusted maximum value.

[0163] As described above, when zeroing is applied, particularly when the top left 16x16 region is a low-frequency transform coefficient region (which may be referred to as 32-point reduced MTS or RMT 32 hereinafter), zeroing may be applied both when applying the MTS technique and when applying the 32-point DST-7 or 32-point DCT-8.

[0164] Figure 5 Illustrated is an MTS applied to sub-block transform according to an example of the present disclosure.

[0165] According to an example, a sub-block transform (SBT) can be applied in which a coding unit is divided into sub-blocks and a transform process is performed on the sub-blocks. The sub-block transform is applied to the residual signal generated by inter-frame prediction, and the residual signal block is partitioned into two partitioned sub-blocks. A separable transform is applied to only one of the sub-blocks according to the sub-block transform. The sub-block can be divided horizontally or vertically, and the width or height of the partitioned sub-block can be 1 / 2 or 1 / 4 of the coding unit. When the sub-block transform is applied, since only one of the two partitioned sub-blocks is transformed, residual data exists only in the transformed sub-block, and no residual data exists in the remaining sub-block.

[0166] When the width of a sub-block to which a transform is applied or its height is 64 or more, DCT-2 can be applied in both the horizontal and vertical directions, while when the width of a sub-block to which a transform is applied or its height is 32 or less, DST-7 or DCT-8 can be applied. Therefore, when SBT is applied, zeroing can be performed by applying RMTS32 only when both sides of the sub-block to which a transform is applied have a length of 32 or less. That is, DST-7 or DCT-8 with a length of 32 or less can be applied in each direction (horizontally and vertically), leaving a maximum of 16 transform coefficients for each row or column.

[0167] like Figure 5 As shown, when the block is divided and the transform is applied to the region A, DST-7 or DCT-8 may be applied to each side and the transform applied in the horizontal and vertical directions is not limited to Figure 5 The example shown in the figure. Figure 5 In

[15] , the width and height of the entire block are represented by w and h, respectively, and the width and height of the block to which the separable transform is actually applied are expressed as a (width, height) pair, i.e., (w1, h) or (w, h1). w1 can be 1 / 2 or 1 / 4 of w, and h1 can also be 1 / 2 or 1 / 4 of h.

[0168] like Figure 5 As shown, the block to which the transformation is applied can be positioned on the left or right side of the entire block or on the top or bottom of the entire block. Figure 5 The block may be a residual signal generated by inter-frame prediction. Figure 5 A flag indicating that only one sub-block of the residual signal of the partition shown is transformed, and when the flag is 1, it can also be set by signaling to indicate that the block is Figure 5 The symbol indicates whether the partition is vertical or horizontal.

[0169] It is also possible to signal a flag indicating whether the block A to which the transform is actually applied is positioned on the left or right in the entire block or a flag indicating whether the block A is positioned on the top or bottom.

[0170] like Figure 5 As illustrated, when horizontal transform and vertical transform are determined for a specific block instead of being designated by MTS signaling, the above-mentioned RMTS32 can be applied to each side when the corresponding side in the horizontal direction and the vertical direction has a length of 32. In RMTS32, residual coding can be omitted for a return-to-zero region, or residual coding can be performed by scanning only a non-return-to-zero region.

[0171] Figure 6 Illustrated is 32-point zeroing applied to sub-block transform according to an example of the present disclosure.

[0172] When RMTS32 is applied to Figure 5 When the sub-blocks of the partition shown are Figure 6 There are residual data as shown. That is, Figure 6 It is shown that the RMTS 32 is applied to sub-blocks on which transforms are performed among blocks partitioned according to application of sub-block transforms.

[0173] With respect to the width w and height h of the original transform block, the width and height of the block A to which transform is actually applied may be w / 2 and h / 2 or w / 4 and h / 4, respectively.

[0174] In short, if DST-7 or DCT-8 with a length of 32 can be applied in each of the horizontal and vertical directions, RMTS32 can be applied to any block to which the transform is applied. Whether DST-7 or DCT-8 with a length of 32 is applied can be determined by preset signaling or can be determined without signaling according to a predetermined coding condition.

[0175] In the case where MTS is disabled (e.g., in VVC, MTS can be disabled when “sps_mts_enable_flag” is set to 0 in the sequence parameter set), when SBT is applied, DCT-2 is applied in both horizontal and vertical directions instead of the combination of DCT-7 and DCT-8 presented in Table 3.

[0176] Therefore, when MTS is disabled, it is necessary to prevent RMTS32 from being applied even if the block to be transformed is partitioned into sub-blocks. As described above, when MTS is not applied, DCT-2 may be applied in the primary transform instead of DCT-7 or DCT-8, and the left top block to which non-zero transform coefficients to which DCT-2 may be applied may be subjected to high-frequency zeroing to reduce the width and height to 32. That is, the left top block to which non-zero transform coefficients to which inverse DCT-2 may be applied may be reduced in width and height to 32, but is not zeroed to a width and height less than 32. This is to prevent data loss due to zeroing, and the width or height of the left top block to which inverse DCT-2 is applied is not reduced to 16.

[0177] If MTS is disabled, it is necessary to explicitly check the width or height of the partition sub-block to which DCT-2 is applied so that the width or height of the partition sub-block is not reduced to 16.

[0178] According to an example, by determining "sps_mts_enabled_flag" in the residual coding syntax, it can be configured not to perform zeroing due to RMTS32 when applying SBT.

[0179] Normative texts reflecting the aforementioned embodiments can be shown in the following tables (Tables 5 to 15).

[0180] [Table 5]

[0181]

[0182] [Table 6]

[0183]

[0184] Table 5 and Table 6 include picture information signaled in a sequence parameter set for picture coding and include several pieces of flag information related to transformation.

[0185] sps_transform_skip_enabled_flag specifies whether transform skip is applied, ie, whether the transform skip flag (transform_skip_flag) may be present in the transform unit syntax.

[0186] sps_mts_enabled_flag is flag information that specifies whether MTS (Multiple Transform Selection) can be explicitly used. sps_mts_enabled_flag is equal to 1 and specifies that sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are present in the sequence parameter set syntax.

[0187] When either sps_explicit_mts_intra_enabled_flag or sps_explicit_mts_inter_enabled_flag is 1, it specifies that MTS may be applied to a transform unit, which means that tu_mts_idx may be present in the transform unit syntax. For example, when sps_mts_enabled_flag is equal to 1 and sps_explicit_mts_intra_enabled_flag is equal to 0, implicit MTS may be applied to intra coding units.

[0188] sps_sbt_enabled_flag is flag information specifying whether the previous sub-block transform can be applied to the coding unit for inter prediction.

[0189] When sps_sbt_enabled_flag is equal to 1, sps_sbt_max_size_64_flag for the maximum width and height of a coding unit for which sub-block transformation is allowed may be signaled.

[0190] When sps_sbt_max_size_64_flag is equal to 0, the maximum width and height of a coding unit for allowing block transform is 32, and when sps_sbt_max_size_64_flag is equal to 1, the maximum width and height of a coding unit for allowing sub-block transform is derived to be equal to 64.

[0191] [Table 7]

[0192]

[0193] Table 7 shows transform-related information signaled in the picture parameter set. log2_transform_skip_max_size_minus2 is information for deriving the maximum block size for transform skipping, and the maximum block size for transform skipping is derived as a value of log2_transform_skip_max_size_minus2 plus 2 to the power of 2 (1<<(log2_transform_skip_max_size_minus2 + 2).

[0194] [Table 8]

[0195]

[0196]

[0197]

[0198] [Table 9]

[0199]

[0200]

[0201] Tables 8 and 9 show syntax and semantics of a coding unit to which inter prediction is applied, and a partition shape to which SBT is applied may be determined by the four syntax elements of Table 8.

[0202] cu_sbt_flag specifies whether SBT is applied to the coding unit, and cu_sbt_quad_flag is flag information that specifies whether the block to which the transform is applied is 1 / 4 of the entire block when one coding unit is partitioned into two partition blocks. When cu_sbt_quad_flag is equal to 0, the partition subblock has a size of 1 / 2 of the width or height of the coding unit, and when cu_sbt_quad_flag is equal to 1, the partition subblock has a size of 1 / 4 of the width or height of the coding unit. When the width of a coding unit is w and its height is h, the height of the partition block may be h1 = (1 / 4) xh or its width may be w1 = (1 / 4) xw.

[0203] cu_sbt_horizontal_flag equal to 1 specifies that the coding unit is partitioned horizontally, ie, in the horizontal direction, and cu_sbt_horizontal_flag equal to 0 specifies that the coding unit is partitioned vertically, ie, in the vertical direction.

[0204] When the cu_sbt_pos_flag value is equal to 0, the transform is applied to the upper sub-block among the sub-blocks partitioned in the horizontal direction, and the transform is applied to the left sub-block among the sub-blocks partitioned in the vertical direction. When the cu_sbt_pos_flag value is equal to 1, the transform is applied to the lower sub-block among the sub-blocks partitioned in the horizontal direction, and the transform is applied to the right sub-block among the sub-blocks partitioned in the vertical direction.

[0205] The following table shows trTypeHor and trTypeVer according to cu_sbt_horizontal_flag and cu_sbt_pos_flag.

[0206] [Table 10]

[0207] cu_sbt_horizontal_flag cu_sbt_pos_flag trTypeHor trTypeVer 0 0 2 1 0 1 1 1 1 0 1 2 1 1 1 1

[0208] As described above, when trTypeHor represents a horizontal transform kernel and trTypeVer represents a vertical transform kernel, a trTypeHor or trTypeVer value of 0 may be set for DCT-2, a trTypeHor or trTypeVer value of 1 may be set for DST-7, and a trTypeHor or trTypeVer value of 2 may be set for DCT-8. Therefore, when the length of at least one side of the partition block to which the transform is applied is 64 or greater, DCT-2 may be applied in both the horizontal and vertical directions; otherwise, DST-7 or DCT-8 may be applied. When the current block is partitioned and a transform is performed on the sub-blocks, MTS may be implicitly applied as shown in Table 10.

[0209] When the width and height of a sub-block are both 32 or less and DST-7 or DCT-8 is applied, that is, when MTS is applied to the sub-block, the above RMTS can be applied. For example, if the length of the sub-block in each direction is 32, only 16 transform coefficients can be left by applying DST-7 or DCT-8 with a length of 32.

[0210] However, when the width or height of a subblock is 64 or more, DCT-2 may be applied in both horizontal and vertical directions, and only 32 transform coefficients may be left according to high frequency zeroing, instead of applying RMTS in each direction.

[0211] [Table 11]

[0212]

[0213]

[0214] Table 11 shows part of the syntax and semantics of a transform unit according to an example. tu_mts_idx[ x0 ][ y0 ] specifies the MTS index applied to the transform block, and trTypeHor and trTypeVer may be determined according to the MTS index as shown in Table 1.

[0215] According to another example, mts_idx may be signaled in the coding unit level instead of in the transform unit level.

[0216] [Table 12]

[0217]

[0218]

[0219] Table 12 shows part of the syntax and semantics of residual coding according to an example.

[0220] The syntax elements last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, and last_sig_coeff_y_suffix in Table 12 specify (x, y) position information about the last non-zero transform coefficient in the transform block. Specifically, last_sig_coeff_x_prefix specifies the prefix of the column position of the last significant coefficient in the transform block in the scanning order, last_sig_coeff_y_prefix specifies the prefix of the row position of the last significant coefficient in the transform block in the scanning order, last_sig_coeff_x_suffix specifies the suffix of the column position of the last significant coefficient in the transform block in the scanning order, and last_sig_coeff_y_suffix specifies the suffix of the row position of the last significant coefficient in the transform block in the scanning order. Here, the significant coefficient may refer to a non-zero coefficient. In addition, the scanning order may be a top-right diagonal scanning order. Alternatively, the scanning order may be a horizontal scanning order or a vertical scanning order. The scanning order may be determined based on whether intra / inter prediction is applied to a target block (CB or CB including a TB) and / or a specific intra / inter prediction mode.

[0221] The zeroing area can be configured in the residual coding of Table 12 based on tu_mts_idx[ x0 ][ y0 ] of Table 11.

[0222] In addition, when cu_sbt_flag is equal to 1, the height of the block to which the transform is applied is 32 or less (log2TbHeight < 6), its width is 32 (log2TbWidth < 6 && log2TbWidth > 4), and when sps_mts_enabled_flag is equal to 1, the width of the left top area where non-zero transform coefficients may exist is set to 16 (log2ZoTbWidth = 4). Similarly, when cu_sbt_flag is equal to 1, the width of the block to which the transform is applied is 32 or less (log2TbWidth < 6), its height is 32 (log2TbHeight < 6 && log2TbHeight > 4), and when sps_mts_enabled_flag is equal to 1, the height of the left top area where non-zero transform coefficients may exist is set to 16 (log2ZoTbHeight = 4).

[0223] The block to which the transform is applied may be the original transform block or a partitioned sub-block.

[0224] Here, log2ZoTbWidth and log2ZoTbHeight represent the maximum width and maximum height of the left top area in the transform block where non-zero coefficients may exist, and the left top area may be referred to as a "reduced transform block." That is, log2ZoTbWidth and log2ZoTbHeight may be defined as variables indicating the width and height of the reduced transform block.

[0225] That is, according to an example, when sub-block transform is applied to a coding unit, MTS needs to be applicable so that return to zero (RMTS) is applied, where the width of the left top block where non-zero coefficients may exist is reduced to 16 and the remaining area is 0.

[0226] When MTS is not applied, even if SBT is applied to the coding unit (when cu_sbt_flag is 1), DCT-2 is applied to the sub-block to which the transform is applied instead of MTS. For example, when sps_mts_enabled_flag is 0, DCT-2 is applied instead of DST-7 or DCT-8 even if SBT is applied to the coding block.

[0227] In addition, when SBT is applied to a coding block, DCT-2 is applied only when the width or height of a subblock is 64 or more, and DCT-2 is not applied in other cases.

[0228] That is, in order to ensure that DCT-2 is applied to the sub-block and to prevent RMTS, which may cause data loss, from being applied to the sub-block to which DCT-2 is applied, the encoding device configures image information to check the value of sps_mts_enabled_flag, and the decoding device checks the value of sps_mts_enabled_flag in residual coding according to the configured image information.

[0229] In summary, when only cu_sbt_flag is identified as 1 during image decoding, the value of sps_mts_enabled_flag signaled in the sequence parameter set can be checked when the transform block size is set according to RMTS to prevent RMTS from being applied to sub-blocks to which DCT-2 is applied. When cu_sbt_flag is equal to 1 and sps_mts_enabled_flag is equal to 1, RMTS can be applied, so the width and height of the left top block where non-zero transform coefficients may exist can be set to 16, and when cu_sbt_flag is equal to 1 and sps_mts_enabled_flag is equal to 0, MTS is not applied because DCT-2 is used for transform, and therefore zeroing according to RMTS is not applied. In this case, the width and height of the left top block where non-zero transform coefficients may exist can be set to 32 or less.

[0230] For example, in the case of a 32×32 transform block to which sub-block transform is applied, when sps_mts_enabled_flag is equal to 0, DCT-2 is used as a transform kernel, and thus a width or height of length 32 is not reduced to 16.

[0231] As mentioned above, when MTS is disabled, RMTS with a length of 16 can be prevented from being zeroed by checking sps_mts_enabled_flag.

[0232] In other cases, when cu_sbt_flag is not 1, the height of the transform block is greater than 32, the width of the transform block is not 32, or MTS is not applied, the width of the transform block can be set to the smaller value of the width of the transform block and 32. That is, the maximum width of the transform block can be limited to 32 by high-frequency zeroing. In addition, when cu_sbt_flag is not 1, the width of the transform block is greater than 32, the height of the transform block is not 32, or MTS is not applied, the height of the transform block can be set to the smaller value of the height of the transform block and 32. That is, the maximum height of the transform block can be limited to 32 by zeroing.

[0233] When SBT is applied, if the length of at least one side of the partition block is 64 or greater, DCT-2 can be applied in both the horizontal and vertical directions. Otherwise, DST-7 or DCT-8 can be used as shown in Table 10. Therefore, when SBT is applied, zeroing is performed by applying RMTS32 only when both sides of the partition block to which the transform is applied have a length of 32 or less. That is, when the length of the block in each direction is 32, DST-7 or DCT-8 with a length of 32 can be applied, leaving only 16 transform coefficients.

[0234] As shown in Table 12, when RMTS32 is applied, the width and height of the residual area (low-frequency transform coefficient area) that is not returned to zero can be considered as the width and height of the actual transform block instead of using the width and height of the original transform block used for coding (log2ZoTbWidth = 4 or log2ZoTbHeight = 4) to perform coding.

[0235] For example, when the width x height of the original transform block is 32x16, when RMTS32 is applied, non-zero coefficients exist only in the top left 16x16 region due to zeroing. Therefore, the width and height of the top left region where non-zero transform coefficients may exist are set to 16 and 16, respectively, and then syntax elements (e.g., last_sig_coeff_x_prefix and last_sig_coeff_y_prefix) can be coded.

[0236] In summary, according to the residual coding of Table 12, log2ZoTbWidth and log2ZoTbHeight, which indicate the maximum width and maximum height in which non-zero coefficients can exist, are set before coding last_sig_coeff_x_prefix. The position of the last non-zero coefficient is coded by applying log2ZoTbWidth and log2ZoTbHeight, and then the width and height of the actual transform block are changed to log2ZoTbWidth and log2ZoTbHeight, respectively (log2ZoTbWidth = log2ZoTbWidth and log2ZoTbHeight = log2ZoTbHeight). Subsequently, the syntax elements can be encoded according to the changed values. Therefore, the remaining top left area excluding the zeroed area in the residual coding can be configured as a new transform block, and then the residual samples can be derived.

[0237] When the size of the transform block is reduced to the low-frequency transform coefficient area by zeroing the high-frequency transform coefficients, the values ​​of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix can be restricted to the range from 0 to the value between (log2ZoTbWidth << 1) – 1 and (log2ZoTbHeight << 1) – 1 as shown in the semantics of Table 12.

[0238] [Table 13]

[0239]

[0240]

[0241]

[0242]

[0243]

[0244]

[0245]

[0246]

[0247] Table 13 illustrates the transformation process in which MTS is implicitly applied when SBT is applied to a coding unit (cu_sbt_flag is equal to 1 and Max(nTbW, nTbH) is less than or equal to 32, implicitMtsEnabled is set equal to 1).

[0248] A variable trTypeHor representing a horizontal transformation kernel and a variable trTypeVer representing a vertical transformation kernel may be derived based on Tables 8-14, and Tables 8-14 of Table 13 may correspond to Table 1 of this document.

[0249] In addition, when SBT is applied to the compilation unit, variables trTypeHor and trTypeVer may be derived based on Tables 8-15, and Tables 8-15 of Table 13 may correspond to Table 10 of this document.

[0250] The size of the block to which zeroing is applied, ie, the zeroing block illustrated in Table 12, is expressed as nonZeroW and nonZeroH in Table 13. nonZeroW and nonZeroH may be defined as variables representing the width and height of the top left block where non-zero transform coefficients may exist.

[0251] When LFNST is not applied, nonZeroW may be set to a smaller value (nonZeroW = Min (nTbW, ( trTypeHor > 0 ) ? 16 : 32 )) based on whether trTypeHor is greater than 0 and the width of the transform block (nTbW). When trTypeHor is greater than 0, since MTS is applied, “(trTypeHor > 0 ) ? 16 : 32” is set to 16, and thus nonZeroW is set to the smaller value of the width of the transform block (nTbW) and 16. However, when trTypeHor is not greater than 0, since MTS is not applied, “( trTypeHor > 0 ) ? 16 : 32” is set to 32, and thus nonZeroW is set to the smaller value of the width of the transform block (nTbW) and 32.

[0252] Similarly, when LFNST is not applied, nonZeroH may be set to a smaller value (nonZeroH = Min(nTbH, (trTypeVer>0)?16:32)) based on whether trTypeVer is greater than 0 and the width of the transform block (nTbH). When trTypeVer is greater than 0, since MTS is applied, "(trTypeVer>0)?16:32" is set to 16, and thus nonZeroH is set to the smaller value of the height of the transform block (nTbH) and 16. However, when trTypeVer is not greater than 0, since MTS is not applied, "(trTypeVer>0)?16:32" is set to 32, and thus nonZeroH is set to the smaller value of the height of the transform block (nTbH) and 32.

[0253] That is, the size of the zeroing block is set to 16 or 32 depending on whether MTS is applied, ie, whether trTypeHor and trTypeVer can have a value of 0 or greater.

[0254] The residual sample values ​​may be derived based on nonZeroW and nonZeroH taking into account the zeroing settings (when nTbH is greater than 1, each (vertical) column of scaled transform coefficients d[x][y] with x = 0..nonZeroW − 1, y = 0..nonZeroH − 1 is converted to e[x][y] with x = 0..nonZeroW − 1, y = 0..nTbH − 1 by invoking the one-dimensional transform process specified in clause 8.7.4.4 for each column x = 0..nonZeroW − 1 with as input the height nTbH of the transform block, the non-zero height nonZeroH of the scaled transform coefficients, the list d[x][y] with y = 0..nonZeroH − 1 and the transform type variable trType set equal to trTypeVer, and the output is a list e[x][y] with y = 0..nTbH − 1).

[0255] When the size of the transform block is changed by applying zeroing, the size of the transform block used for context selection of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix can also be changed. Table 14 shows the binarization of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix considering the reduction of the transform block, and Table 15 shows the process of deriving ctxInc (context delta) for deriving last_sig_coeff_x_prefix and lastsig_coeff_y_prefix. Since the context can be selected and distinguished by the context delta, the context model can be derived based on the context delta.

[0256] [Table 14]

[0257]

[0258]

[0259] [Table 15]

[0260]

[0261]

[0262] As illustrated in Table 14, the maximum value (cMax) of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix is ​​set to the low-frequency transform coefficient area (cMax = (cMax = (log2ZoTbWidth << 1) – 1, cMax = (log2ZoTbHeight << 1) – 1) based on log2ZoTbWidth and log2ZoTbHeight corresponding to the width and height of the reduced transform block. 1). When truncated unary is used for binarization of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix, the maximum value (cMax) of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix can be set equal to the maximum value of the codeword used for binarization of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix. Therefore, the maximum length of the prefix codeword representing the last significant coefficient prefix information can be derived based on the size of the zeroed block.

[0263] As illustrated in Table 15, for the CABAC context for two syntax elements, last_sig_coeff_x_prefix and last_sig_coeff_y_prefix, the size of the original transform block (TU) in the low-frequency transform coefficient region is applied instead of the reduced transform block (log2TbSize is set equal to log2TbWidth, log2TbSize is set equal to log2TbHeight).

[0264] In summary, according to an example, the residual sample can be derived based on the last significant coefficient position information, wherein the context model can be derived based on the size of the original transform block whose size has not changed, and the last significant coefficient position can be derived based on the size of the transform block to which zeroing has been applied. Here, the size of the transform block to which zeroing has been applied, i.e., the zeroing block, specifically has a width or height that is smaller than the size, width, or height, of the original transform block.

[0265] The following figures are provided to describe specific examples of the present disclosure. Since the specific names of the devices or the names of the specific signals / messages / fields illustrated in the figures are provided for illustration purposes, the technical features of the present disclosure are not limited to the specific names used in the following figures.

[0266] Figure 7 is a flowchart illustrating the operation of a video decoding apparatus according to an embodiment of the present disclosure.

[0267] Figure 7 Each operation shown in the figure can be performed by Figure 3 Specifically, S700 and S710 may be performed by the decoding device 300 shown in FIG. Figure 3 The entropy decoder 310 shown in FIG is executed, S720 can be performed by Figure 3 The dequantizer 321 shown in FIG is executed, S730 can be performed by Figure 3 The inverse transformer 322 shown in FIG is executed, and S740 can be performed by Figure 3 The adder 340 shown in FIG is executed. The operations according to S700 to S740 are based on the reference Figures 4 to 6 Therefore, some of the previous details of the description will be omitted or will be briefly carried out in conjunction with the above reference Figures 3 to 6 A description of the specific details of those overlaps is provided.

[0268] The decoding apparatus according to the embodiment may receive a bitstream including residual information ( S700 ). Specifically, the entropy decoder 310 of the decoding apparatus may receive a bitstream including residual information.

[0269] The decoding apparatus according to an embodiment may derive a quantized transform coefficient for the current block based on residual information included in a bitstream ( S710 ). Specifically, the entropy decoder 310 of the decoding apparatus may quantize the transform coefficient for the current block based on residual information included in a bitstream.

[0270] The decoding apparatus according to the embodiment may derive transform coefficients from the quantized transform coefficients based on a dequantization process (S720). Specifically, the dequantizer 321 of the decoding apparatus may derive transform coefficients from the quantized transform coefficients based on a dequantization process.

[0271] The decoding apparatus according to the embodiment may derive residual samples for the current block by applying inverse transform to the derived transform coefficients (S730). Specifically, the inverse transform 322 of the decoding apparatus may derive residual samples for the current block by applying inverse transform to the derived transform coefficients.

[0272] The decoding apparatus according to the embodiment may generate a reconstructed picture based on the residual samples for the current block (S740). Specifically, the adder 340 of the decoding apparatus may generate a reconstructed picture based on the residual samples for the current block.

[0273] In an embodiment, the unit of the current block may be a transform block (TB).

[0274] In an embodiment, each transform coefficient for the current block may be associated with a high-frequency transform coefficient region including a transform coefficient of 0 or a low-frequency transform coefficient region including at least one significant transform coefficient.

[0275] In an embodiment, the residual information may include last significant coefficient prefix information and last significant coefficient suffix information regarding a position of a last significant transform coefficient among transform coefficients for the current block.

[0276] In one example, the last significant coefficient prefix information may have a maximum value determined based on the size of the return-to-zero block.

[0277] In an embodiment, the position of the last significant transform coefficient may be determined based on a prefix codeword indicating last significant coefficient prefix information and last significant coefficient suffix information.

[0278] In an embodiment, the maximum length of the prefix codeword may be determined based on the size of the low-frequency transform coefficient region, ie, the return-to-zero block.

[0279] In an embodiment, the size of the zeroing block may be determined based on the width and height of the current block.

[0280] In an embodiment, the last significant coefficient prefix information may include x-axis prefix information and y-axis prefix information, and the prefix codeword may be a codeword for the x-axis prefix information and a codeword for the y-axis prefix information.

[0281] In one example, the x-axis prefix information may be expressed as last_sig_coeff_x_prefix, the y-axis prefix information may be expressed as last_sig_coeff_y_prefix, and the position of the last significant transform coefficient may be expressed as (LastSignificantCoeffX, LastSignificantCoeffY).

[0282] In an embodiment, the residual information may include information about the size of the zeroing block.

[0283] Figure 8 is a flowchart illustrating a process for deriving transform coefficients by a video decoding apparatus according to an embodiment of the present disclosure.

[0284] Figure 8 Each operation shown in the figure can be performed by Figure 3 Specifically, S800 to S840 may be performed by the decoding device 300 shown in FIG. Figure 3 The entropy decoder 310 shown in FIG is executed.

[0285] First, as shown in the figure, a zeroing block for the current block can be derived (S800). As described above, the zeroing block refers to a low-frequency transform coefficient region including non-zero significant transform coefficients, and the width or height of the zeroing block can be derived based on whether MTS using multiple transform kernels is applicable to the current block, whether sub-block transform is applied, and the width or height of the current block.

[0286] According to an example, the decoding device may set the width or height of the return-to-zero block to 16 when MTS is applied, and may set the width or height of the return-to-zero block to 32 or less when MTS is not applied. In this case, whether MTS is applicable may be determined based on sps_mts_enabled_flag indicating whether MTS is applicable.

[0287] Specifically, when DCT-7 or DCT-8 is applied instead of DCT-2 as the transform kernel for the inverse primary transform, the width of the current block is 32, and the height of the current block is 32 or less, the width of the zeroing block may be set to 16. When the above conditions are not satisfied, that is, when the transform kernel is DCT-2, the width of the current block is not 32, or the height of the current block is 64 or more, the width of the zeroing block may be set to the smaller value of the width of the current block and 32.

[0288] Similarly, when DCT-7 or DCT-8 is applied instead of DCT-2 as the transform kernel for the inverse primary transform, the height of the current block is 32, and the width of the current block is 32 or less, the height of the zeroing block may be set to 16. When the above conditions are not satisfied, that is, when the transform kernel is DCT-2, the height of the current block is not 32, or the width of the current block is 64 or more, the height of the zeroing block may be set to the smaller value of the height of the current block and 32.

[0289] Furthermore, according to an example, the width or height of the return-to-zero block may be derived based on flag information (cu_sbt_flag) indicating whether the current block is partitioned into subblocks and transformed. For example, when the value of the flag indicating whether the current block is partitioned into subblocks and transformed is 1, the width of the partitioned subblock is 32, and the height of the subblock is less than 64, the width of the left top region where non-zero transform coefficients may exist in the subblock may be set to 16. Alternatively, when the value of the flag indicating whether the current block is partitioned into subblocks and transformed is 1, the height of the partitioned subblock is 32, and the width of the subblock is less than 64, the height of the left top region where non-zero transform coefficients may exist in the subblock may be set to 16.

[0290] The transform kernel may be derived based on the partition direction of the current block and the position of the sub-block to which the transform is applied as shown in Table 10.

[0291] The width or height of the zeroing block may be derived based on the MTS index of the current block or flag information indicating whether the MTS is applied to the transformation of the current block.

[0292] The size of the return-to-zero block may be smaller than the size of the current block. Specifically, the width of the return-to-zero block may be smaller than the width of the current block, and the height of the return-to-zero block may be smaller than the height of the current block.

[0293] In an embodiment, the size of the zeroing block may be one of 32x16, 16x32, 16x16, or 32x32.

[0294] In an embodiment, the size of the current block may be 64x64, and the size of the zeroing block may be 32x32.

[0295] The decoding apparatus may derive a context model for last significant coefficient position information based on the width or height of the current block ( S810 ).

[0296] According to an example, a context model may be derived based on the size of the original transform block rather than the size of the zeroed block. Specifically, context increments of the x-axis prefix information and the y-axis prefix information corresponding to the last significant coefficient prefix information may be derived based on the size of the original transform block.

[0297] The decoding apparatus may derive a value of last significant coefficient position information based on the derived context model ( S820 ).

[0298] As described above, the last significant coefficient position information may include last significant coefficient prefix information and last significant coefficient suffix information, and the value of the last significant coefficient position may be derived based on a context model.

[0299] The decoding apparatus may derive the last significant coefficient position based on the value of the derived last significant coefficient position information and the width or height of the return-to-zero block ( S830 ).

[0300] In one example, the decoding device may derive the last significant coefficient position within the size of a zeroed block that is smaller than the size of the current block rather than the size of the original current block. That is, the transform coefficient to which the transform is applied may be derived within the size of the zeroed block rather than the size of the current block.

[0301] In one example, the last significant coefficient prefix information may have a maximum value determined based on the size of the return-to-zero block.

[0302] In one example, the last significant coefficient position may be derived based on a prefix codeword indicating last significant coefficient prefix information and last significant coefficient suffix information, and the maximum length of the prefix codeword may be determined based on the size of the return-to-zero block.

[0303] The decoding apparatus may derive a transform coefficient according to the last significant coefficient position derived based on the width or height of the return-to-zero block ( S840 ).

[0304] The transform coefficients can be derived through the residual coding process of Table 13.

[0305] Subsequently, the decoding device may derive residual samples by performing at least one of the foregoing non-separable inverse secondary transform and inverse primary transform based on Table 1 and Table 10.

[0306] The following figures are provided to describe specific examples of the present disclosure. Since the specific names of the devices or the names of the specific signals / messages / fields illustrated in the figures are provided for illustration purposes, the technical features of the present disclosure are not limited to the specific names used in the following figures.

[0307] Figure 9 is a flowchart illustrating the operation of a video encoding apparatus according to an embodiment of the present disclosure.

[0308] Figure 9 Each operation shown in the figure can be performed by Figure 2 Specifically, S900 may be performed by the encoding device 200 shown in FIG. Figure 2 The subtractor 231 shown in FIG is executed, S910 can be performed by Figure 2The converter 232 shown in FIG is executed, S920 can be performed by Figure 2 , and S930 may be performed by the quantizer 233 shown in FIG. Figure 2 The entropy encoder 240 shown in FIG is executed. The operations according to S900 to S930 are based on the reference Figures 4 to 6 Therefore, some of the previous details of the description will be omitted or will be briefly described with reference to the above Figure 2 and Figures 4 to 6 A description of the specific details of those overlaps is provided.

[0309] The encoding apparatus according to an embodiment may derive residual samples for the current block ( S900 ). Specifically, the subtractor 231 of the encoding apparatus may derive residual samples for the current block.

[0310] The encoding apparatus according to an embodiment may transform residual samples for a current block, thereby deriving a transform coefficient for the current block ( S910 ). Specifically, the transformer 232 of the encoding apparatus may transform residual samples for the current block, thereby deriving a transform coefficient for the current block.

[0311] The encoding apparatus according to an embodiment may derive a quantized transform coefficient from the transform coefficient based on quantization ( S920 ). Specifically, the quantizer 233 of the encoding apparatus may derive a quantized transform coefficient from the transform coefficient based on quantization.

[0312] The encoding apparatus according to the embodiment may encode residual information including information about the quantized transform coefficient (S930). Specifically, the entropy encoder 240 of the encoding apparatus may encode residual information including information about the quantized transform coefficient.

[0313] In an embodiment, each transform coefficient for the current block may be associated with a high-frequency transform coefficient region including a transform coefficient of 0 or a low-frequency transform coefficient region including at least one significant transform coefficient, ie, a return-to-zero block.

[0314] In an embodiment, the residual information may include last significant coefficient prefix information and last significant coefficient suffix information regarding a position of a last significant transform coefficient among transform coefficients for the current block.

[0315] In an embodiment, the position of the last significant transform coefficient may be determined based on a prefix codeword indicating last significant coefficient prefix information and last significant coefficient suffix information.

[0316] In one example, the last significant coefficient prefix information may have a maximum value determined based on the size of the return-to-zero block.

[0317] In an embodiment, the maximum length of the prefix codeword may be determined based on the size of the return-to-zero block.

[0318] In an embodiment, the size of the zeroing block may be determined based on the width and height of the current block.

[0319] In an embodiment, the last significant coefficient prefix information may include x-axis prefix information and y-axis prefix information, and the prefix codeword may be a codeword regarding the x-axis prefix information and a codeword for the y-axis prefix information.

[0320] In one example, the x-axis prefix information may be expressed as last_sig_coeff_x_prefix, the y-axis prefix information may be expressed as last_sig_coeff_y_prefix, and the position of the last significant transform coefficient may be expressed as (LastSignificantCoeffX, LastSignificantCoeffY).

[0321] In an embodiment, the residual information may include information about the size of the zeroing block.

[0322] Figure 10 is a flowchart illustrating a process for encoding transform coefficients and information according to an embodiment of the present disclosure.

[0323] Figure 10 Each operation exposed in can be operated by Figure 2 Specifically, S1000 and S1010 may be performed by the converter 232, and S1020 to S1040 may be performed by the converter 232. Figure 2 The entropy encoder 240 disclosed in is executed.

[0324] First, as shown in the figure, a zeroing block for the current block can be derived (S1000). As described above, the zeroing block refers to a low-frequency transform coefficient region including non-zero significant transform coefficients, and the width or height of the zeroing block can be derived based on whether MTS using multiple transform kernels is applicable to the current block, whether sub-block transform is applied, and the width or height of the current block.

[0325] According to an example, the encoding device may set the width or height of the return-to-zero block to 16 when MTS is applied (when DST-7 / DCT-8 is applicable), and may set the width or height of the return-to-zero block to 32 or less when MTS is not applied.

[0326] Specifically, when DCT-7 or DCT-8 is applied instead of DCT-2 as the transform kernel for the primary transform, the width of the current block is 32, and the height of the current block is 32 or less, the width of the zeroing block may be set to 16. When the above conditions are not satisfied, that is, when the transform kernel is DCT-2, the width of the current block is not 32, or the height of the current block is 64 or more, the width of the zeroing block may be set to the smaller value of the width of the current block and 32.

[0327] Similarly, when DCT-7 or DCT-8 is applied instead of DCT-2 as the transform kernel for the primary transform, the height of the current block is 32, and the width of the current block is 32 or less, the height of the zeroing block may be set to 16. When the above conditions are not satisfied, that is, when the transform kernel is DCT-2, the height of the current block is not 32, or the width of the current block is 64 or more, the height of the zeroing block may be set to the smaller value of the height of the current block and 32.

[0328] Furthermore, according to an example, the width or height of the return-to-zero block may be derived based on whether the current block is partitioned into sub-blocks and transformed. For example, when the current block is partitioned into sub-blocks and transformed, the width of the partitioned sub-block is 32, and the height of the sub-block is less than 64, the width of the sub-block may be set to 16. Alternatively, when the current block is partitioned into sub-blocks and transformed, the height of the partitioned sub-block is 32, and the width of the sub-block is less than 64, the height of the sub-block may be set to 16.

[0329] The transform kernel may be derived based on the partition direction of the current block and the position of the sub-block to which the transform is applied as shown in Table 10.

[0330] In an embodiment, the size of the zeroing block may be one of 32x16, 16x32, 16x16, or 32x32.

[0331] In an embodiment, the size of the current block may be 64x64, and the size of the zeroing block may be 32x32.

[0332] The encoding apparatus may derive a transform coefficient based on the return-to-zero block ( S1010 ).

[0333] The encoding apparatus may derive transform coefficients from the residual samples by performing at least one of the foregoing transform processes, ie, primary transform and non-separable inverse secondary transform, based on Table 1 and Table 10.

[0334] The encoding apparatus may derive a last significant coefficient position based on the derived width or height of the return-to-zero block ( S1020 ).

[0335] In one example, the encoding device may derive the last significant coefficient position within the size of a zeroed block that is smaller than or equal to the size of the current block instead of the size of the original current block. That is, the transform coefficient to which the transform is applied may be derived within the size of the zeroed block instead of the size of the current block.

[0336] In one example, the last significant coefficient position may be derived based on a prefix codeword indicating last significant coefficient prefix information and last significant coefficient suffix information, and the maximum length of the prefix codeword may be determined based on the size of the return-to-zero block.

[0337] The encoding apparatus may derive a context model for last significant coefficient position information based on the width or height of the current block ( S1030 ).

[0338] According to an embodiment, the context model can be derived based on the size of the original transform block rather than the size of the zero block. Specifically, the context increments of the x-axis prefix information and the y-axis prefix information corresponding to the last significant coefficient prefix information can be derived based on the size of the original transform block.

[0339] The encoding apparatus may encode position information about a value of a last significant coefficient position based on the derived context model ( S1040 ).

[0340] As described above, the last significant coefficient position information may include last significant coefficient prefix information and last significant coefficient suffix information, and the value of the last significant coefficient position may be encoded based on a context model.

[0341] In the present disclosure, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. When quantization / dequantization is omitted, the quantized transform coefficient may be referred to as a transform coefficient. When transform / inverse transform is omitted, the transform coefficient may be referred to as a coefficient or a residual coefficient, or may still be referred to as a transform coefficient for consistency of expression.

[0342] Furthermore, in this disclosure, quantized transform coefficients and transform coefficients may be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, residual information may include information about the transform coefficients, and this information may be signaled using residual coding syntax. Transform coefficients may be derived based on the residual information (or information about the transform coefficients), and scaled transform coefficients may be derived by inverse transforming (scaling) the transform coefficients. Residual samples may be derived based on the inverse transform (transform) of the scaled transform coefficients. These details may also be applied / expressed in other parts of this disclosure.

[0343] In the above embodiments, the method is described based on a flowchart with the aid of a series of steps or blocks, but the present disclosure is not limited to the order of the steps, and a step may be performed in an order or step different from the above order or step or simultaneously with another step. In addition, it will be understood by those skilled in the art that the steps shown in the flowchart are not exclusive, and another step may be incorporated or one or more steps of the flowchart may be removed without affecting the scope of the present disclosure.

[0344] The above-mentioned method according to the present disclosure may be implemented in a software form, and the encoding device and / or decoding device according to the present disclosure may be included in an image processing device such as a TV, a computer, a smart phone, a set-top box, a display device, etc.

[0345] When the embodiments of the present disclosure are implemented by software, the above methods can be embodied as modules (processes, functions, etc.) for performing the above functions. The modules can be stored in memory and executed by a processor. The memory can be internal or external to the processor and can be connected to the processor in various well-known ways. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. In other words, the embodiments described in the present disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure can be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0346] Furthermore, the decoding device and encoding device to which the present disclosure is applied may be included in multimedia broadcast transceivers, mobile communication terminals, home theater video devices, digital theater video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video on demand (VoD) service providers, over-the-top (OTT) video devices, Internet streaming service providers, three-dimensional (3D) video devices, video phone video devices, and medical video devices, and may be used to process video signals or data signals. For example, over-the-top (OTT) video devices may include game consoles, Blu-ray players, Internet-connected TVs, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), and the like.

[0347] Furthermore, the processing methods applying the present disclosure can be generated in the form of a program executed by a computer and stored in a computer-readable recording medium. Multimedia data having a data structure according to the present disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices in which computer-readable data is stored. Computer-readable recording media may include, for example, Blu-ray Discs (BDs), Universal Serial Buses (USBs), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. Furthermore, computer-readable recording media include media embodied in the form of carrier waves (e.g., transmitted over the Internet). Furthermore, the bitstream generated by the encoding method can be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network. Furthermore, embodiments of the present disclosure can be embodied as a computer program product through program code, and the program code can be executed on a computer through embodiments of the present disclosure. The program code can be stored on a computer-readable carrier.

[0348] Figure 11 The structure of a content streaming system to which the present disclosure is applied is illustrated.

[0349] Furthermore, the content streaming system to which the present disclosure is applied may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0350] The encoding server compresses the content input from a multimedia input device such as a smart phone, camera, or camcorder into digital data to generate a bitstream, and transmits the bitstream to the streaming server. As another example, when a multimedia input device such as a smart phone, camera, or camcorder directly generates a bitstream, the encoding server can be omitted. The bitstream can be generated by applying the encoding method or bitstream generation method of the embodiments of this document. The streaming server can also temporarily store the bitstream during the process of sending or receiving the bitstream.

[0351] The streaming server transmits multimedia data to user devices via a network server based on user requests. The network server serves as a medium for informing users of services. When a user requests a desired service from the network server, the network server transmits the requested service to the streaming server, which then transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server is used to control commands and responses between devices in the content streaming system.

[0352] The streaming server can receive content from a media storage and / or encoding server. For example, when receiving content from an encoding server, the content can be received in real time. In this case, in order to provide a stable streaming service, the streaming server can store the bit stream for a predetermined time.

[0353] For example, user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., watch-type terminals (smart watches), glasses-type terminals (smart glasses), head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc. Each of the servers in the content streaming system may be operated as a distributed server, and in this case, data received by each server may be processed in a distributed manner.

[0354] The claims disclosed herein can be combined in various ways. For example, the technical features of the method claims of the present disclosure can be combined to be implemented or performed in a device, and the technical features of the device claims can be combined to be implemented or performed in a method. Furthermore, the technical features of method claims and device claims that can be combined can be combined to be implemented or performed in a device, and the technical features of method claims and device claims that can be combined can be implemented or performed in a method.

Claims

1. A device comprising: Memory; as well as at least one processor connected to the memory, the at least one processor configured to: receiving a bitstream including residual information, deriving transform coefficients for the current block based on the residual information, and deriving residual samples for the current block based on the transform coefficients, wherein deriving the transform coefficients comprises deriving a zeroed block associated with an area where significant transform coefficients may exist in the current block, The return-to-zero block is derived based on a first flag, wherein the first flag indicates whether a second flag and a third flag are present in a sequence parameter set SPS of the bitstream, The second flag indicates whether the multi-transform selection (MTS) index information can be used for an intra coding unit, and the third flag indicates whether the MTS index information can be used for an inter coding unit. wherein, based on the second flag indicating that the MTS index information can exist for the intra coding unit or the third flag indicating that the MTS index information can exist for the inter coding unit, parsing the MTS index information; and The MTS index information is related to which transform cores are applied to the current block.

2. The device according to claim 1, wherein Based on the first flag indicating the presence of the second flag and the third flag in the SPS of the bitstream, the width or height of the return-to-zero block is set to 16, and Wherein, based on the first flag indicating that the second flag and the third flag do not exist in the SPS of the bitstream, the width or height of the return-to-zero block is set to be less than or equal to 32.

3. The device according to claim 1, wherein Based on a fourth flag being equal to 1, the width or height of the zeroed block is set to 16, the fourth flag being related to whether sub-block transform for performing transform on sub-blocks derived by partitioning coding units is applied to the current block.

4. The device according to claim 3, wherein Based on the height of the sub-block being less than 64 and the width of the sub-block being 32, the width of the return-to-zero block is set to 16.

5. The apparatus according to claim 3, wherein Based on the width of the sub-block being less than 64 and the height of the sub-block being 32, the height of the return-to-zero block is set to 16.

6. The apparatus according to claim 3, wherein A transform kernel for the current block is derived based on a partition direction of the current block and a position of a subblock to which the transform is applied.

7. The apparatus according to claim 1, wherein The residual information includes the last significant coefficient prefix information, and The maximum value of the last significant coefficient prefix information is derived based on the size of the return-to-zero block.

8. The apparatus according to claim 1, wherein The return-to-zero block is derived for a luma component of the current block.

9. A device comprising: Memory; as well as at least one processor connected to the memory, the at least one processor configured to: Derive the residual samples for the current block, deriving transform coefficients based on the residual samples for the current block, and Encoding residual information including information about the transform coefficients and multi-transform select (MTS) index information, wherein deriving the transform coefficients comprises deriving a zeroed block associated with an area where significant transform coefficients may exist in the current block, The return-to-zero block is derived based on a first flag, wherein the first flag indicates whether a second flag and a third flag are present in a sequence parameter set SPS of a bitstream, The second flag indicates whether the MTS index information can exist for an intra coding unit, and the third flag indicates whether the MTS index information can exist for an inter coding unit. wherein, based on the second flag indicating that the MTS index information can exist for the intra coding unit or the third flag indicating that the MTS index information can exist for the inter coding unit, encoding the MTS index information; and The MTS index information is related to which transform cores are applied to the current block.

10. The apparatus according to claim 9, wherein Based on the first flag indicating the presence of the second flag and the third flag in the SPS of the bitstream, the width or height of the return-to-zero block is set to 16, and Wherein, based on the first flag indicating that the second flag and the third flag do not exist in the SPS of the bitstream, the width or height of the return-to-zero block is set to be less than or equal to 32.

11. The apparatus according to claim 9, wherein Based on a fourth flag being equal to 1, the width or height of the zeroed block is set to 16, the fourth flag being related to whether sub-block transform for performing transform on sub-blocks derived by partitioning coding units is applied to the current block.

12. The apparatus according to claim 11, wherein Based on the height of the sub-block being less than 64 and the width of the sub-block being 32, the width of the zeroing block is set to 16, and Wherein, based on the fact that the width of the sub-block is less than 64 and the height of the sub-block is 32, the height of the return-to-zero block is set to 16.

13. The apparatus according to claim 11, wherein A transform kernel for the current block is determined based on a partition direction of the current block and a position of a subblock to which the transform is applied.

14. The apparatus according to claim 9, wherein The residual information includes the last significant coefficient prefix information, and The maximum value of the last significant coefficient prefix information is derived based on the size of the return-to-zero block.

15. A device comprising: at least one processor configured to obtain a bitstream for an image, wherein the bitstream is generated based on: deriving residual samples for a current block, deriving transform coefficients based on the residual samples for the current block, and encoding residual information including information about the transform coefficients and multi-transform select (MTS) index information to generate the bitstream; and a transmitter configured to transmit data comprising the bit stream, wherein deriving the transform coefficients comprises deriving a zeroed block associated with an area where significant transform coefficients may exist in the current block, The return-to-zero block is derived based on a first flag, wherein the first flag indicates whether a second flag and a third flag are present in a sequence parameter set SPS of the bitstream, The second flag indicates whether the MTS index information can exist for an intra coding unit, and the third flag indicates whether the MTS index information can exist for an inter coding unit. wherein, based on the second flag indicating that the MTS index information can exist for the intra coding unit or the third flag indicating that the MTS index information can exist for the inter coding unit, encoding the MTS index information; and The MTS index information is related to which transform cores are applied to the current block.

Images