IMAGE CODING METHOD BASED ON NON-SEPARABLE SECONDARY TRANSFORMATION AND DEVICE THEREOF

Abstract

An image decoding method performed by means of a decoding device in accordance with the present description comprises the steps of: deriving transformation coefficients of a target block from a bitstream; deriving a non-separable secondary transformation index (NSST) with respect to the target block; performing an inverse transformation with respect to the transformation coefficients of the target block on the basis of the NSST index and thereby deriving residual samples of the target block; and generating a reconstructed image on the basis of the residual samples.

Description

Image coding method and device based on non-separable second-order transform

[0001] The present invention relates to image coding technology, and more particularly, to an image decoding method and device according to a non-separable second transform in an image coding system.

[0002] Demand for high-resolution, high-quality video, such as HD (High Definition) and UHD (Ultra High Definition), is growing across various fields. As video data becomes higher in resolution and quality, the amount of information or bits transmitted increases relative to conventional video data. Therefore, transmitting video data using existing media, such as wired or wireless broadband lines, or storing it using existing storage media, increases transmission and storage costs.

[0003] Accordingly, high-efficiency image compression technology is required to effectively transmit, store, and reproduce high-resolution, high-quality image information.

[0004] The technical problem of the present invention is to provide a method and device for increasing image coding efficiency.

[0005] Another technical object of the present invention is to provide an image decoding method and device that applies NSST to a target block.

[0006] Another technical object of the present invention is to provide an image decoding method and device for deriving a range of NSST indexes based on specific conditions of a target block.

[0007] Another technical object of the present invention is to provide a video decoding method and device that determines whether to code an NSST index based on a transform coefficient of a target block.

[0008] According to one embodiment of the present invention, a video decoding method performed by a decoding device is provided. The method is characterized by including the steps of: deriving transform coefficients of a target block from a bitstream; deriving a Non-Separable Secondary Transform (NSST) index for the target block; performing an inversed transform on the transform coefficients of the target block based on the NSST index to derive residual samples of the target block; and generating a restored picture based on the residual samples.

[0009] According to another embodiment of the present invention, a decoding device for performing image decoding is provided. The decoding device is characterized by including an entropy decoding unit for deriving transform coefficients of a target block from a bitstream and deriving a Non-Separable Secondary Transform (NSST) index for the target block, an inverse transform unit for deriving residual samples of the target block by performing an inversed transform on the transform coefficients of the target block based on the NSST index, and an adding unit for generating a restored picture based on the residual samples.

[0010] According to another embodiment of the present invention, a video encoding method performed by an encoding device is provided. The method includes the steps of: deriving residual samples of a target block; deriving transform coefficients of the target block by performing a transform on the residual samples; determining whether to encode an NSST index for the target block; and encoding information on the transform coefficients, wherein the step of determining whether to encode the NSST index includes the steps of: scanning (R+1)-th to (N)-th transform coefficients among the transform coefficients of the target block; and determining not to encode the NSST index if the (R+1)-th to (N)-th transform coefficients include a non-zero transform coefficient, wherein N is the number of samples of an upper-left target area of ​​the target block, R is a reduced coefficient, and R is smaller than N.

[0011] According to another embodiment of the present invention, a video encoding device is provided. The encoding device includes an adding unit for deriving residual samples of a target block, a transform unit for deriving transform coefficients of the target block by performing a transform on the residual samples, and an entropy encoding unit for determining whether to encode an NSST index for the target block and encoding information on the transform coefficients, wherein the entropy encoding unit scans (R+1)-th to (N)-th transform coefficients among the transform coefficients of the target block, and determines not to encode the NSST index when a non-zero transform coefficient is included in the (R+1)-th to (N)-th transform coefficients, wherein N is the number of samples of an upper-left target area of ​​the target block, R is a reduced coefficient, and R is smaller than N.

[0012] According to the present invention, the range of the NSST index can be derived based on specific conditions of the target block, thereby reducing the amount of bits for the NSST index and improving the overall coding efficiency.

[0013] According to the present invention, transmission of a syntax element for an NSST index can be determined based on transform coefficients for a target block, thereby reducing the amount of bits for the NSST index and improving overall coding efficiency.

[0014] Figure 1 is a drawing schematically illustrating the configuration of a video encoding device to which the present invention can be applied.

[0015] Figure 2 illustrates an example of a video encoding method performed by a video encoding device.

[0016] FIG. 3 is a drawing schematically illustrating the configuration of a video decoding device to which the present invention can be applied.

[0017] Figure 4 shows an example of an image decoding method performed by a decoding device.

[0018] Figure 5 schematically illustrates a multiple conversion technique according to the present invention.

[0019] Figure 6 illustrates intra-directional modes of 65 prediction directions.

[0020] FIG. 7a and FIG. 7b are flowcharts illustrating a coding process of a transform coefficient according to one embodiment.

[0021] FIG. 8 is a drawing for explaining an arrangement of transformation coefficients based on a target block according to an embodiment of the present invention.

[0022] Figure 9 shows an example of scanning the transformation coefficients from R+1 to N.

[0023] FIG. 10a and FIG. 10b are flowcharts illustrating a coding process of an NSST index according to one embodiment.

[0024] Figure 11 shows an example of determining whether an NSST index is coded.

[0025] Figure 12 shows an example of scanning the transform coefficients from R+1 to N for all components of the target block.

[0026] Figure 13 schematically illustrates an image encoding method using an encoding device according to the present invention.

[0027] Figure 14 schematically illustrates an encoding device that performs an image encoding method according to the present invention.

[0028] Figure 15 schematically illustrates an image decoding method by a decoding device according to the present invention.

[0029] Fig. 16 schematically illustrates a decoding device that performs an image decoding method according to the present invention.

[0030] The present invention is susceptible to various modifications and embodiments, and thus specific embodiments will be illustrated and described in detail in the drawings. However, this is not intended to limit the present invention to specific embodiments. The terminology used herein is only used to describe specific embodiments and is not intended to limit the technical idea of ​​the present invention. The singular expression includes plural expressions unless the context clearly indicates otherwise. It should be understood that the terms "comprises" or "has" in this specification specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0031] Meanwhile, each component in the drawings described in the present invention is depicted independently for the convenience of explaining different characteristic functions. This does not imply that each component is implemented with separate hardware or software. For example, two or more components may be combined to form a single component, or a single component may be divided into multiple components. Embodiments in which each component is integrated and / or separated are also included within the scope of the present invention, as long as they do not deviate from the essence of the present invention.

[0032] Hereinafter, with reference to the attached drawings, preferred embodiments of the present invention will be described in more detail. Hereinafter, identical components in the drawings will be designated by the same reference numerals, and redundant descriptions of identical components will be omitted.

[0033] Meanwhile, the present invention relates to video / image coding. For example, the methods / embodiments disclosed in the present invention can be applied to methods disclosed in the versatile video coding (VVC) standard or the next-generation video / image coding standard.

[0034] In this specification, a "picture" generally refers to a unit representing a single video image from a specific time period, while a "slice" is a unit that constitutes part of a picture in coding. A single picture may be composed of multiple slices, and pictures and slices may be used interchangeably as needed.

[0035] A pixel or pel can refer to the smallest unit that constitutes a picture (or image). Furthermore, the term "sample" can be used as a counterpart to a pixel. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chroma component.

[0036] A unit represents the basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to that region. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows.

[0037] Figure 1 is a drawing schematically illustrating the configuration of a video encoding device to which the present invention can be applied.

[0038] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (105), a prediction unit (110), a residual processing unit (120), an entropy encoding unit (130), an addition unit (140), a filter unit (150), and a memory (160). The residual processing unit (120) may include a subtraction unit (121), a transformation unit (122), a quantization unit (123), a reordering unit (124), an inverse quantization unit (125), and an inverse transformation unit (126).

[0039] The picture division unit (105) can divide the input picture into at least one processing unit.

[0040] For example, a processing unit may be called a coding unit (CU). In this case, the coding unit may be recursively split from a largest coding unit (LCU) according to a Quad-tree binary-tree (QTBT) structure. For example, one coding unit may be split into multiple coding units of deeper depth based on a quad-tree structure and / or a binary-tree structure. In this case, for example, the quad-tree structure may be applied first and the binary-tree structure may be applied later. Alternatively, the binary-tree structure may be applied first. The coding procedure according to the present invention may be performed based on the final coding unit that is no longer split. In this case, based on coding efficiency according to image characteristics, etc., the largest coding unit may be used directly as the final coding unit, or, if necessary, the coding unit may be recursively split into coding units of lower depth so that the coding unit of the optimal size may be used as the final coding unit. The coding procedure here may include procedures such as prediction, transformation, and restoration, which are described later.

[0041] As another example, a processing unit may include a coding unit (CU), a prediction unit (PU), or a transform unit (TU). A coding unit may be split into coding units of deeper depths from a largest coding unit (LCU) along a quad-tree structure. In this case, based on coding efficiency according to image characteristics, etc., the largest coding unit may be used as the final coding unit, or, if necessary, the coding unit may be recursively split into coding units of lower depths, and the coding unit of the optimal size may be used as the final coding unit. If a smallest coding unit (SCU) is set, the coding unit cannot be split into coding units smaller than the smallest coding unit. Here, the final coding unit refers to a coding unit that is the basis for partitioning or dividing into prediction units or transform units. A prediction unit is a unit partitioned from a coding unit and may be a unit of sample prediction. At this time, the prediction unit may be divided into sub blocks. The transform unit may be divided from the coding unit along a quad tree structure, and may be a unit that derives a transform coefficient and / or a unit that derives a residual signal from the transform coefficient. Hereinafter, the coding unit may be called a coding block (CB), the prediction unit may be called a prediction block (PB), and the transform unit may be called a transform block (TB). A prediction block or prediction unit may mean a specific area in the form of a block within a picture, and may include an array of prediction samples.Additionally, a transform block or transform unit may mean a specific area in the form of a block within a picture and may include an array of transform coefficients or residual samples.

[0042] The prediction unit (110) can perform a prediction on a block to be processed (hereinafter referred to as a current block) and generate a predicted block including prediction samples for the current block. The unit of prediction performed in the prediction unit (110) can be a coding block, a transformation block, or a prediction block.

[0043] The prediction unit (110) can determine whether intra prediction or inter prediction is applied to the current block. For example, the prediction unit (110) can determine whether intra prediction or inter prediction is applied on a CU basis.

[0044] In the case of intra prediction, the prediction unit (110) can derive a prediction sample for the current block based on reference samples outside the current block within the picture to which the current block belongs (hereinafter, the current picture). At this time, the prediction unit (110) can (i) derive the prediction sample based on the average or interpolation of neighboring reference samples of the current block, and (ii) can also derive the prediction sample based on a reference sample existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. In the case of (i), it can be called a non-directional mode or a non-angular mode, and in the case of (ii), it can be called a directional mode or an angular mode. In intra prediction, the prediction mode can have, for example, 33 directional prediction modes and at least two non-directional modes. The non-directional mode can include a DC prediction mode and a planar mode. The prediction unit (110) can also determine the prediction mode to be applied to the current block by using the prediction mode applied to the surrounding blocks.

[0045] In the case of inter prediction, the prediction unit (110) can derive a prediction sample for the current block based on a sample specified by a motion vector on a reference picture. The prediction unit (110) can derive a prediction sample for the current block by applying any one of a skip mode, a merge mode, and an MVP (motion vector prediction) mode. In the case of the skip mode and the merge mode, the prediction unit (110) can use the motion information of the surrounding blocks as the motion information of the current block. In the case of the skip mode, unlike the merge mode, the difference (residual) between the prediction sample and the original sample is not transmitted. In the case of the MVP mode, the motion vector of the current block can be derived by using the motion vector of the surrounding blocks as a motion vector predictor and as a motion vector predictor of the current block.

[0046] In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in a reference picture. The reference picture including the temporal neighboring blocks may be called a collocated picture (colPic). Motion information may include a motion vector and a reference picture index. Information such as prediction mode information and motion information may be (entropy) encoded and output in the form of a bitstream.

[0047] In skip mode and merge mode, when motion information of temporally adjacent blocks is utilized, the top-ranked picture in the reference picture list may be used as a reference picture. The reference pictures included in the reference picture list (Picture Order Count) may be sorted based on the difference in Picture Order Count (POC) between the current picture and the corresponding reference picture. The POC corresponds to the display order of the pictures and can be distinguished from the coding order.

[0048] The subtraction unit (121) generates a residual sample, which is the difference between the original sample and the predicted sample. When skip mode is applied, the residual sample may not be generated as described above.

[0049] The transform unit (122) transforms residual samples in units of transform blocks to generate transform coefficients. The transform unit (122) can perform the transform according to the size of the corresponding transform block and the prediction mode applied to the coding block or prediction block spatially overlapping with the corresponding transform block. For example, if intra prediction is applied to the coding block or the prediction block overlapping with the transform block and the transform block is a 4x4 residual array, the residual sample can be transformed using a DST (Discrete Sine Transform) transform kernel, and in other cases, the residual sample can be transformed using a DCT (Discrete Cosine Transform) transform kernel.

[0050] The quantization unit (123) can quantize the transform coefficients to generate quantized transform coefficients.

[0051] The rearrangement unit (124) rearranges the quantized transform coefficients. The rearrangement unit (124) can rearrange the block-shaped quantized transform coefficients into a one-dimensional vector through a coefficient scanning method. Although the rearrangement unit (124) is described as a separate component here, the rearrangement unit (124) may be a part of the quantization unit (123).

[0052] The entropy encoding unit (130) can perform entropy encoding on quantized transform coefficients. The entropy encoding may include encoding methods such as, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoding unit (130) may encode, together or separately, information necessary for video restoration (e.g., values ​​of syntax elements, etc.) in addition to the quantized transform coefficients. The entropy-encoded information may be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units.

[0053] The inverse quantization unit (125) inversely quantizes the values ​​(quantized transform coefficients) quantized in the quantization unit (123), and the inverse transformation unit (126) inversely transforms the values ​​inversely quantized in the inverse quantization unit (125) to generate residual samples.

[0054] The addition unit (140) restores the picture by combining the residual sample and the prediction sample. The residual sample and the prediction sample can be added block by block to generate a restoration block. Although the addition unit (140) is described as a separate component here, the addition unit (140) can be part of the prediction unit (110). Meanwhile, the addition unit (140) can also be called a restoration unit or restoration block generation unit.

[0055] The filter unit (150) may apply a deblocking filter and / or a sample adaptive offset to the reconstructed picture. Through the deblocking filtering and / or the sample adaptive offset, artifacts at the block boundary within the reconstructed picture or distortion during the quantization process may be corrected. The sample adaptive offset may be applied on a sample basis and may be applied after the deblocking filtering process is completed. The filter unit (150) may also apply an ALF (Adaptive Loop Filter) to the reconstructed picture. The ALF may be applied to the reconstructed picture after the deblocking filter and / or the sample adaptive offset have been applied.

[0056] The memory (160) can store a restored picture (decoded picture) or information required for encoding / decoding. Here, the restored picture may be a restored picture that has completed a filtering process by the filter unit (150). The stored restored picture may be used as a reference picture for (inter) prediction of another picture. For example, the memory (160) may store (reference) pictures used for inter prediction. At this time, the pictures used for inter prediction may be designated by a reference picture set or a reference picture list.

[0057] Fig. 2 illustrates an example of a video encoding method performed by a video encoding device. Referring to Fig. 2, the video encoding method may include intra / inter prediction, transform, quantization, and entropy encoding processes. For example, a prediction block of a current block may be generated through intra / inter prediction, and a residual block of the current block may be generated through subtraction of an input block of the current block and the prediction block. Thereafter, a coefficient block, i.e., transform coefficients of the current block, may be generated through transform on the residual block. The transform coefficients may be quantized and entropy encoded and stored in a bitstream.

[0058] FIG. 3 is a drawing schematically illustrating the configuration of a video decoding device to which the present invention can be applied.

[0059] Referring to FIG. 3, a video decoding device (300) may include an entropy decoding unit (310), a residual processing unit (320), a prediction unit (330), an addition unit (340), a filter unit (350), and a memory (360). Here, the residual processing unit (320) may include a rearrangement unit (321), an inverse quantization unit (322), and an inverse transformation unit (323).

[0060] When a bitstream containing video information is input, the video decoding device (300) can restore the video corresponding to the process in which the video information was processed in the video encoding device.

[0061] For example, the video decoding device (300) can perform video decoding using a processing unit applied in a video encoding device. Accordingly, a processing unit block for video decoding may be, for example, a coding unit, and may be, for example, a coding unit, a prediction unit, or a transformation unit. The coding unit may be divided according to a quad tree structure and / or a binary tree structure from a maximum coding unit.

[0062] Prediction units and transformation units may be further used in some cases, in which case the prediction block may be a block derived or partitioned from a coding unit, and may be a unit of sample prediction. In this case, the prediction unit may be divided into sub-blocks. The transformation unit may be divided from the coding unit along a quad-tree structure, and may be a unit that derives a transform coefficient or a unit that derives a residual signal from a transform coefficient.

[0063] The entropy decoding unit (310) can parse the bitstream to output information necessary for video restoration or picture restoration. For example, the entropy decoding unit (310) can decode information within the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements necessary for video restoration and quantized values ​​of transform coefficients for residuals.

[0064] In more detail, the CABAC entropy decoding method receives a bin corresponding to each syntax element in a bitstream, determines a context model using information of the syntax element to be decoded and decoding information of surrounding and decoding target blocks or information of symbols / bins decoded in a previous step, and predicts the occurrence probability of the bin according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element. At this time, the CABAC entropy decoding method can update the context model using information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model.

[0065] Among the information decoded in the entropy decoding unit (310), information regarding prediction is provided to the prediction unit (330), and the residual value on which entropy decoding is performed in the entropy decoding unit (310), i.e., the quantized transform coefficient, can be input to the rearrangement unit (321).

[0066] The rearrangement unit (321) can rearrange the quantized transform coefficients into a two-dimensional block form. The rearrangement unit (321) can perform rearrangement in response to coefficient scanning performed in the encoding device. Although the rearrangement unit (321) is described as a separate component here, the rearrangement unit (321) may be a part of the dequantization unit (322).

[0067] The inverse quantization unit (322) can output transform coefficients by inversely quantizing quantized transform coefficients based on (inverse) quantization parameters. At this time, information for deriving the quantization parameters can be signaled from the encoding device.

[0068] The inverse transform unit (323) can inversely transform the transform coefficients to derive residual samples.

[0069] The prediction unit (330) can perform a prediction on the current block and generate a predicted block including prediction samples for the current block. The unit of prediction performed in the prediction unit (330) may be a coding block, a transformation block, or a prediction block.

[0070] The prediction unit (330) can determine whether to apply intra prediction or inter prediction based on the information about the prediction. At this time, the unit for determining whether to apply intra prediction or inter prediction and the unit for generating prediction samples may be different. In addition, the unit for generating prediction samples for inter prediction and intra prediction may also be different. For example, whether to apply inter prediction or intra prediction may be determined on a CU basis. In addition, for example, in inter prediction, the prediction mode may be determined on a PU basis and prediction samples may be generated, and in intra prediction, the prediction mode may be determined on a PU basis and prediction samples may be generated on a TU basis.

[0071] In the case of intra prediction, the prediction unit (330) can derive a prediction sample for the current block based on surrounding reference samples within the current picture. The prediction unit (330) can derive a prediction sample for the current block by applying a directional mode or a non-directional mode based on surrounding reference samples of the current block. In this case, the prediction mode to be applied to the current block may be determined using the intra prediction mode of the surrounding block.

[0072] In the case of inter prediction, the prediction unit (330) can derive a prediction sample for the current block based on a sample specified on the reference picture by a motion vector on the reference picture. The prediction unit (330) can derive a prediction sample for the current block by applying any one of a skip mode, a merge mode, and an MVP mode. At this time, information on motion information required for inter prediction of the current block provided from the video encoding device, such as a motion vector, a reference picture index, etc., can be acquired or derived based on the information on the prediction.

[0073] In skip mode and merge mode, motion information of surrounding blocks can be used as motion information of the current block. In this case, the surrounding blocks can include spatial surrounding blocks and temporal surrounding blocks.

[0074] The prediction unit (330) constructs a merge candidate list using motion information of available surrounding blocks, and can use the information indicated by the merge index on the merge candidate list as the motion vector of the current block. The merge index can be signaled from the encoding device. The motion information can include a motion vector and a reference picture. When motion information of temporal surrounding blocks is used in skip mode and merge mode, the top picture on the reference picture list can be used as the reference picture.

[0075] In skip mode, unlike merge mode, the difference (residual) between the predicted sample and the original sample is not transmitted.

[0076] In MVP mode, the motion vector of the current block can be derived using the motion vector of the surrounding blocks as a motion vector predictor. At this time, the surrounding blocks may include spatial and temporal surrounding blocks.

[0077] For example, when the merge mode is applied, a merge candidate list can be generated using the motion vector of the restored spatial neighboring block and / or the motion vector corresponding to the Col block, which is a temporal neighboring block. In the merge mode, the motion vector of the candidate block selected from the merge candidate list is used as the motion vector of the current block. The information regarding the prediction can include a merge index indicating a candidate block having an optimal motion vector selected from among the candidate blocks included in the merge candidate list. At this time, the prediction unit (330) can derive the motion vector of the current block using the merge index.

[0078] As another example, when the MVP (Motion Vector Prediction) mode is applied, a motion vector predictor candidate list can be generated using the motion vectors of the reconstructed spatial neighboring blocks and / or the motion vectors corresponding to the Col block, which is a temporal neighboring block. That is, the motion vectors of the reconstructed spatial neighboring blocks and / or the motion vectors corresponding to the Col block, which is a temporal neighboring block, can be used as motion vector candidates. The information regarding the prediction can include a predicted motion vector index indicating an optimal motion vector selected from among the motion vector candidates included in the list. At this time, the prediction unit (330) can select the predicted motion vector of the current block from among the motion vector candidates included in the motion vector candidate list using the motion vector index. The prediction unit of the encoding device can obtain a motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, and can encode and output the same in the form of a bitstream. That is, the MVD can be obtained as a value obtained by subtracting the motion vector predictor from the motion vector of the current block. At this time, the prediction unit (330) can obtain the motion vector difference included in the information regarding the prediction, and derive the motion vector of the current block through the addition of the motion vector difference and the motion vector predictor. The prediction unit can also obtain or derive a reference picture index indicating a reference picture, etc., from the information regarding the prediction.

[0079] The addition unit (340) can restore the current block or current picture by adding the residual sample and the prediction sample. The addition unit (340) can also restore the current picture by adding the residual sample and the prediction sample in block units. When skip mode is applied, the residual is not transmitted, so the prediction sample can become the restored sample. Although the addition unit (340) is described as a separate component here, the addition unit (340) can also be a part of the prediction unit (330). Meanwhile, the addition unit (340) can also be called a restoration unit or a restoration block generation unit.

[0080] The filter unit (350) may apply deblocking filtering, sample adaptive offset, and / or ALF to the restored picture. At this time, the sample adaptive offset may be applied on a sample basis and may be applied after deblocking filtering. The ALF may be applied after deblocking filtering and / or sample adaptive offset.

[0081] The memory (360) can store restored pictures (decoded pictures) or information required for decoding. Here, the restored picture may be a restored picture that has completed a filtering process by the filter unit (350). For example, the memory (360) can store pictures used for inter prediction. At this time, the pictures used for inter prediction may be designated by a reference picture set or a reference picture list. The restored picture may be used as a reference picture for another picture. In addition, the memory (360) may output the restored pictures according to the output order.

[0082] Fig. 4 illustrates an example of an image decoding method performed by a decoding device. Referring to Fig. 4, the image decoding method may include entropy decoding, inverse quantization, inverse transform, and intra / inter prediction processes. For example, the decoding device may perform an inverse process of the encoding method. Specifically, quantized transform coefficients may be obtained through entropy decoding on a bitstream, and a coefficient block of a current block, i.e., transform coefficients, may be obtained through an inverse quantization process on the quantized transform coefficients. A residual block of the current block may be derived through inverse transform on the transform coefficients, and a reconstructed block of the current block may be derived through addition of the prediction block of the current block derived through intra / inter prediction and the residual block.

[0083] Meanwhile, through the above-described transformation, lower frequency transformation coefficients for the residual block of the current block can be derived, and a zero tail can be derived at the end of the residual block.

[0084] Specifically, the above transformation may consist of two main processes, which may include a core transform and a secondary transform. A transformation including the core transform and the secondary transform may be referred to as a multi-transform technique.

[0085] Figure 5 schematically illustrates a multiple conversion technique according to the present invention.

[0086] Referring to FIG. 5, the conversion unit may correspond to the conversion unit in the encoding device of FIG. 1 described above, and the inverse conversion unit may correspond to the inverse conversion unit in the encoding device of FIG. 1 described above or the inverse conversion unit in the decoding device of FIG. 3.

[0087] The transformation unit can derive (first) transformation coefficients by performing a first transformation based on residual samples (residual sample array) within the residual block (S510). Here, the first transformation may include an adaptive multiple core transform (AMT). The adaptive multiple core transform may also be expressed as a multiple transform set (MTS).

[0088] The above adaptive multi-core transform may indicate a method of transforming by additionally using Discrete Cosine Transform (DCT) Type 2 and Discrete Sine Transform (DST) Type 7, DCT Type 8, and / or DST Type 1. That is, the above adaptive multi-core transform may indicate a transform method of transforming a residual signal (or residual block) of a spatial domain into transform coefficients (or primary transform coefficients) of a frequency domain based on a plurality of transform kernels selected from among the DCT Type 2, the DST Type 7, the DCT Type 8, and the DST Type 1. Here, the primary transform coefficients may be called temporary transform coefficients from the perspective of a transform unit.

[0089] In other words, when the conventional transform method is applied, a transformation from the spatial domain to the frequency domain can be applied to the residual signal (or residual block) based on DCT type 2, so that transform coefficients can be generated. In contrast, when the adaptive multi-core transform is applied, a transformation from the spatial domain to the frequency domain can be applied to the residual signal (or residual block) based on DCT type 2, DST type 7, DCT type 8, and / or DST type 1, so that transform coefficients (or first-order transform coefficients) can be generated. Here, DCT type 2, DST type 7, DCT type 8, and DST type 1, etc. may be called a transform type, a transform kernel, or a transform core.

[0090] For reference, the above DCT / DST transformation types can be defined based on basis functions, and the basis functions can be represented as shown in the following table.

[0091]

[0092] When the above adaptive multi-core transform is performed, a vertical transform kernel and a horizontal transform kernel for a target block may be selected from among the transform kernels, and a vertical transform may be performed for the target block based on the vertical transform kernel, and a horizontal transform may be performed for the target block based on the horizontal transform kernel. Here, the horizontal transform may represent a transform for horizontal components of the target block, and the vertical transform may represent a transform for vertical components of the target block. The vertical transform kernel / horizontal transform kernel may be adaptively determined based on a transform index indicating a prediction mode and / or a transform subset of a target block (CU or sub-block) encompassing a residual block.

[0093] For example, the adaptive multi-core transform can be applied when both the width and the height of the target block are less than or equal to 64, and whether the adaptive multi-core transform of the target block is applied can be determined based on a CU level flag. Specifically, when the CU level flag is 0, the existing transform method described above can be applied. That is, when the CU level flag is 0, a transformation from a spatial domain to a frequency domain based on the DCT type 2 can be applied to a residual signal (or a residual block) to generate transform coefficients, and the transform coefficients can be encoded. Meanwhile, the target block here can be a CU. When the CU level flag is 0, the adaptive multi-core transform can be applied to the target block.

[0094] In addition, for a luma block of a target block to which the adaptive multi-core transform is applied, two additional flags may be signaled, and a vertical transform kernel and a horizontal transform kernel may be selected based on the flags. The flag for the vertical transform kernel may be represented as an AMT vertical flag, and AMT_TU_vertical_flag (or EMT_TU_vertical_flag) may represent a syntax element of the AMT vertical flag. The flag for the horizontal transform kernel may be represented as an AMT horizontal flag, and AMT_TU_horizontal_flag (or EMT_TU_horizontal_flag) may represent a syntax element of the AMT horizontal flag. The AMT vertical flag may indicate one transform kernel candidate among transform kernel candidates included in a transform subset for the vertical transform kernel, and the transform kernel candidate indicated by the AMT vertical flag may be derived as a vertical transform kernel for the target block. In addition, the AMT horizontal flag may indicate one of the transformation kernel candidates included in the transformation subset for the horizontal transformation kernel, and the transformation kernel candidate indicated by the AMT horizontal flag may be derived as the horizontal transformation kernel for the target block. Meanwhile, the AMT vertical flag may be indicated as an MTS vertical flag, and the AMT horizontal flag may be indicated as an MTS horizontal flag.

[0095] Meanwhile, three transformation subsets may be preset, and one of the transformation subsets may be derived as a transformation subset for the vertical transformation kernel based on the intra prediction mode applied to the target block. In addition, one of the transformation subsets may be derived as a transformation subset for the horizontal transformation kernel based on the intra prediction mode applied to the target block. For example, the preset transformation subsets may be derived as shown in the following table.

[0096]

[0097] Referring to Table 2, a transform subset having an index value of 0 may represent a transform subset including DST type 7 and DCT type 8 as transform kernel candidates, a transform subset having an index value of 1 may represent a transform subset including DST type 7 and DST type 1 as transform kernel candidates, and a transform subset having an index value of 2 may represent a transform subset including DST type 7 and DCT type 8 as transform kernel candidates.

[0098] The transformation subset for the vertical transformation kernel and the transformation subset for the horizontal transformation kernel derived based on the intra prediction mode applied to the target block can be derived as shown in the following table.

[0099]

[0100] Here, V represents a transform subset for the vertical transform kernel, and H represents a transform subset for the horizontal transform kernel.

[0101] When the value of the AMT flag (or EMT_CU_flag) for the target block is 1, a transform subset for the vertical transform kernel and a transform subset for the horizontal transform kernel can be derived based on the intra prediction mode of the target block as shown in Table 3. Thereafter, among the transform kernel candidates included in the transform subset for the vertical transform kernel, a transform kernel candidate indicated by the AMT vertical flag of the target block can be derived as the vertical transform kernel of the target block, and among the transform kernel candidates included in the transform subset for the horizontal transform kernel, a transform kernel candidate indicated by the AMT horizontal flag of the target block can be derived as the horizontal transform kernel of the target block. Meanwhile, the AMT flag may also be expressed as an MTS flag.

[0102] For example, an intra prediction mode may include two non-directional (or non-angular) intra prediction modes and 65 directional (or angular) intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode numbered 0 and a DC intra prediction mode numbered 1, and the directional intra prediction modes may include 65 intra prediction modes numbered 2 to 66. However, this is merely an example, and the present invention may also be applied to cases where the number of intra prediction modes is different. Meanwhile, in some cases, an intra prediction mode numbered 67 may be further used, and the 67 intra prediction mode may represent a linear model (LM) mode.

[0103] Figure 6 illustrates intra-directional modes of 65 prediction directions.

[0104] Referring to Fig. 6, intra prediction modes with horizontal directionality and intra prediction modes with vertical directionality can be distinguished centered on intra prediction mode number 34 having an upward left diagonal prediction direction. H and V in Fig. 6 represent horizontal and vertical directionality, respectively, and numbers -32 to 32 represent displacements in units of 1 / 32 on the sample grid position. Intra prediction modes numbered 2 to 33 have horizontal directionality, and intra prediction modes numbered 34 to 66 have vertical directionality. The 18th intra prediction mode and the 50th intra prediction mode represent the horizontal intra prediction mode and the vertical intra prediction mode, respectively. The 2nd intra prediction mode can be called the left-downward diagonal intra prediction mode, the 34th intra prediction mode can be called the left-upward diagonal intra prediction mode, and the 66th intra prediction mode can be called the right-upward diagonal intra prediction mode.

[0105] The transform unit can derive (secondary) transform coefficients by performing a secondary transform based on the (first) transform coefficients (S520). If the first transform was a transform from a spatial domain to a frequency domain, the second transform can be viewed as a transform from a frequency domain to a frequency domain. The second transform can include a non-separable transform. In this case, the second transform can be called a non-separable secondary transform (NSST) or a mode-dependent non-separable secondary transform (MDNSST). The non-separable secondary transform can represent a transform that generates transform coefficients (or secondary transform coefficients) for a residual signal by performing a secondary transform on the (first) transform coefficients derived through the first transform based on a non-separable transform matrix. Here, based on the non-separable transformation matrix, the vertical transformation and the horizontal transformation (or the horizontal-vertical transformation independently) can be applied to the (primary) transformation coefficients at once without being applied separately. In other words, the non-separable secondary transformation can refer to a transformation method of generating transformation coefficients (or secondary transformation coefficients) by transforming the vertical and horizontal components of the (primary) transformation coefficients together without being separated based on the non-separable transformation matrix. The non-separable secondary transformation can be applied to the top-left region of a block composed of (primary) transformation coefficients (hereinafter, referred to as a transformation coefficient block or a target block). For example, when the width (W) and the height (H) of the transformation coefficient block are both 8 or more, an 8×8 non-separable secondary transformation can be applied to the top-left 8×8 region of the transformation coefficient block (hereinafter, referred to as the top-left target region).In addition, when the width (W) and height (H) of the transform coefficient block are both 4 or greater, and the width (W) or height (H) of the transform coefficient block is less than 8, a 4×4 non-separable second-order transform can be applied to the upper left min(8,W)×min(8,H) region of the transform coefficient block.

[0106] For example, when a 4×4 input block is used, the non-separable second-order transform can be performed as follows.

[0107] The above 4×4 input block X can be represented as follows.

[0108]

[0109] When the above X is expressed in vector form, the vector can be expressed as follows.

[0110]

[0111] In this case, the second-order non-separable transformation can be calculated as follows.

[0112]

[0113] Here, represents the transformation coefficient vector, and T represents a 16×16 (non-separable) transformation matrix.

[0114] Through the above mathematical expression 3, a 16×1 transformation coefficient vector can be derived, and the above can be reorganized into 4×4 blocks through the scan order (horizontal, vertical, diagonal, etc.). However, the above-described calculation is just an example, and in order to reduce the computational complexity of the non-separable second-order transform, the Hypercube-Givens Transform (HyGT) or the like can be used to calculate the non-separable second-order transform.

[0115] Meanwhile, the non-separable second-order transform may have a transform kernel (or transform core, transform type) selected mode-dependently. Here, the mode may include an intra-prediction mode and / or an inter-prediction mode.

[0116] As described above, the non-separable secondary transform can be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. That is, the non-separable secondary transform can be performed based on an 8×8 sub-block size or a 4×4 sub-block size. For example, for the mode-based transform kernel selection, 35 sets of non-separable secondary transform kernels, 3 each for the non-separable secondary transform, can be configured for both the 8×8 sub-block size and the 4×4 sub-block size. That is, 35 transform sets can be configured for the 8×8 sub-block size, and 35 transform sets can be configured for the 4×4 sub-block size. In this case, each of the 35 transform sets for the 8×8 sub-block size may contain three 8×8 transform kernels, and each of the 35 transform sets for the 4×4 sub-block size may contain three 4×4 transform kernels. However, the transform sub-block size, the number of sets, and the number of transform kernels in the set may be sizes other than 8×8 or 4×4, as an example, or n sets may be configured, and k transform kernels may be included in each set.

[0117] The above transformation set may be referred to as an NSST set, and the transformation kernel within the NSST set may be referred to as an NSST kernel. Selection of a particular set among the above transformation sets may be performed based on, for example, the intra prediction mode of the target block (CU or sub-block).

[0118] In this case, the mapping between the 35 transformation sets and the intra prediction modes can be represented, for example, as shown in the following table. Note that if the LM mode is applied to the target block, the secondary transformation may not be applied to the target block.

[0119]

[0120] Meanwhile, if it is determined that a specific set is to be used, one of the k transform kernels within the specific set may be selected through a non-separable secondary transform index. The encoding device may derive a non-separable secondary transform index indicating a specific transform kernel based on a rate-distortion (RD) check, and signal the non-separable secondary transform index to a decoding device. The decoding device may select one of the k transform kernels within the specific set based on the non-separable secondary transform index. For example, an NSST index value of 0 may indicate a first non-separable secondary transform kernel, an NSST index value of 1 may indicate a second non-separable secondary transform kernel, and an NSST index value of 2 may indicate a third non-separable secondary transform kernel. Alternatively, an NSST index value of 0 may indicate that the first non-separable secondary transform is not applied to the target block, and NSST index values ​​1 to 3 may indicate the three transform kernels.

[0121] Referring back to FIG. 5, the transform unit can perform the non-separable second-order transform based on the selected transform kernels and obtain (second-order) transform coefficients. The transform coefficients can be derived as quantized transform coefficients through the quantization unit as described above, and can be encoded and transmitted to the decoding device and the inverse quantization / inverse transform unit within the signaling and encoding device.

[0122] Meanwhile, if the secondary transformation is omitted, the (primary) transformation coefficients, which are the output of the primary (separation) transformation, can be derived as quantized transformation coefficients through a quantization unit as described above, and can be encoded and transmitted to a decoding device and an inverse quantization / inverse transformation unit within the signaling and encoding device.

[0123] The inverse transform unit can perform a series of procedures in the reverse order of the procedures performed in the above-described transform unit. The inverse transform unit can receive (inverse quantized) transform coefficients, perform a secondary (inverse) transform to derive (primary) transform coefficients (S550), and perform a primary (inverse) transform on the (primary) transform coefficients to obtain a residual block (residual samples). Here, the primary transform coefficients may be referred to as modified transform coefficients from the inverse transform unit's perspective. As described above, the encoding device and the decoding device can generate a reconstructed block based on the residual block and the predicted block, and generate a reconstructed picture based on the reconstructed block.

[0124] Meanwhile, as described above, when the second (inverse) transformation is omitted, the (inverse quantized) transformation coefficients can be received and the first (separate) transformation can be performed to obtain a residual block (residual samples). As described above, the encoding device and the decoding device can generate a restoration block based on the residual block and the predicted block, and can generate a restoration picture based on the same.

[0125] Meanwhile, the non-separable secondary transform described above may not be applied to a block coded in the transform skip mode. For example, if the NSST index for the target CU is signaled and the value of the NSST index is not 0, the non-separable secondary transform may not be applied to a block coded in the transform skip mode within the target CU. In addition, if the target CU including blocks of all components (luma component, chroma component, etc.) is coded in the transform skip mode or if the number of non-zero transform coefficients among the transform coefficients for the target CU is less than 2, the NSST index may not be signaled. A specific coding process of the transform coefficients is as follows.

[0126] FIG. 7a and FIG. 7b are flowcharts illustrating a coding process of a transform coefficient according to one embodiment.

[0127] Each step disclosed in FIGS. 7a and 7b may be performed by the encoding device (100) or decoding device (300) disclosed in FIGS. 1 and 3, and more specifically, may be performed by the entropy encoding unit (130) disclosed in FIG. 1 and the entropy decoding unit (310) disclosed in FIG. 3. Accordingly, specific details overlapping with those described above in FIG. 1 or FIG. 3 will be omitted or simplified for brevity.

[0128] This specification uses terms or phrases to define specific information or concepts. For example, this specification refers to "a flag indicating whether at least one non-zero transform coefficient exists among the transform coefficients for the target block" as cbf. However, "cbf" can be replaced with various terms such as coded_block_flag. Therefore, when interpreting terms or phrases used to define specific information or concepts in this specification as a whole, the interpretation should not be limited to the name, but should be interpreted with attention to the various operations, functions, and effects that the term implies.

[0129] Figure 7a illustrates the encoding process of the transform coefficients.

[0130] An encoding device (100) according to one embodiment may determine whether a flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block indicates 1 (S700). If the flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block indicates 1, there may be at least one non-zero transform coefficient among the transform coefficients for the target block. Conversely, if the flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block indicates 0, all transform coefficients for the target block may indicate 0.

[0131] A flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block can be expressed as, for example, a cbf flag. The cbf flag can include cbf_luma[x0][y0][trafoDepth] for a luma block and cbf_cb[x0][y0][trafoDepth] and cbf_cr[x0][y0][trafoDepth] flags for chroma blocks. Here, the array indices x0 and y0 indicate the position of the top-left luma / chroma sample of the target block with respect to the top-left luma / chroma sample of the current picture, and the array index trafoDepth can indicate a level into which a coding block is divided for the purpose of transform coding. Blocks where trafoDepth indicates 0 correspond to coding blocks, and trafoDepth can be regarded as 0 when the coding block and the transform block are defined identically.

[0132] An encoding device (100) according to one embodiment can encode information about transform coefficients for a target block (S710) when a flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block in S700 indicates 1.

[0133] Information about transform coefficients for the target block may include, for example, at least one of information about the position of the last non-zero transform coefficient, group flag information indicating whether a subgroup of the target block includes a non-zero transform coefficient, and information about a simplification coefficient. A detailed description of each piece of information will be provided below.

[0134] According to one embodiment, the encoding device (100) can determine whether conditions for performing NSST are met (S720). More specifically, the encoding device (100) can determine whether conditions for encoding an NSST index are met. At this time, the NSST index may be referred to as, for example, a transform index.

[0135] According to one embodiment, the encoding device (100) may encode the NSST index if it is determined that the conditions for performing NSST are met in S720 (S730). More specifically, the encoding device (100) may encode the NSST index if it is determined that the conditions for encoding the NSST index are met.

[0136] An encoding device (100) according to one embodiment may omit operations according to S710, S720, and S730 when a flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block in S700 indicates 0.

[0137] In addition, the encoding device (100) according to one embodiment may omit the operation according to S730 if it is determined that the conditions for performing NSST in S720 are not met.

[0138] Figure 7b illustrates the decoding process of the transform coefficients.

[0139] A decoding device (300) according to one embodiment may determine whether a flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block indicates 1 (S740). If the flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block indicates 1, there may be at least one non-zero transform coefficient among the transform coefficients for the target block. Conversely, if the flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block indicates 0, all transform coefficients for the target block may indicate 0.

[0140] A decoding device (300) according to one embodiment can decode information about transform coefficients for a target block (S750) if a flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block in S740 indicates 1.

[0141] According to one embodiment, the decoding device (300) can determine whether conditions for performing NSST are met (S760). More specifically, the decoding device (300) can determine whether conditions for decoding an NSST index from a bitstream are met.

[0142] A decoding device (300) according to one embodiment can decode an NSST index if it is determined that the conditions for performing NSST are met in S760 (S770).

[0143] According to one embodiment, the decoding device (300) may omit operations according to S750, S760, and S770 if a flag indicating whether there is at least one non-zero transform coefficient among the transform coefficients for the target block in S740 indicates 0.

[0144] In addition, the decoding device (300) according to one embodiment may omit the operation according to S770 if it is determined that the conditions for performing NSST in S760 are not met.

[0145] As described above, signaling the NSST index when NSST is not performed can degrade coding efficiency. Furthermore, varying the NSST index coding method depending on specific conditions can improve overall video coding efficiency. Accordingly, the present invention proposes various NSST index coding methods.

[0146] For example, the NSST index range may be determined based on a specific condition. In other words, the range of values ​​of the NSST index may be determined based on a specific condition. Specifically, the maximum value of the NSST index may be determined based on the specific condition.

[0147] For example, the range of values ​​of the NSST index can be determined based on the block size. Here, the block size can be defined as a minimum (W, H). W can represent width, and H can represent height. In this case, the range of values ​​of the NSST index can be determined by comparing the width of the target block with W, and comparing the height of the target block with the minimum H.

[0148] Alternatively, the block size may be defined as (W*H), which is the number of samples in the block. In this case, the range of values ​​of the NSST index may be determined by comparing W*H, which is the number of samples in the target block, with a specific value.

[0149] Additionally, for example, the range of values ​​of the NSST index can be determined based on the shape of the block, i.e., the block type. Here, the block type can be defined as a square block or a non-square block. In this case, the range of values ​​of the NSST index can be determined based on whether the target block is a square block or a non-square block.

[0150] Alternatively, the block type may be defined by the ratio of the long side (the longer side among the width and height) and the short side of the block. In this case, the range of values ​​of the NSST index may be determined by comparing the ratio of the long side and the short side of the target block with a preset threshold value (e.g., 2 or 3). Here, the ratio may represent a value obtained by dividing the long side by the short side. For example, when the width of the target block is longer than the height, the range of values ​​of the NSST index may be determined by comparing a value obtained by dividing the width by the height with the preset threshold value. In addition, when the height of the target block is longer than the width, the range of values ​​of the NSST index may be determined by comparing a value obtained by dividing the height by the width with the preset threshold value.

[0151] Additionally, the range of values ​​of the NSST index may be determined based on, for example, the intra prediction mode applied to the block. For example, the range of values ​​of the NSST index may be determined based on whether the intra prediction mode applied to the target block is a non-directional intra prediction mode or a directional intra prediction mode.

[0152] Alternatively, as another example, the range of values ​​of the NSST index may be determined based on whether the intra prediction mode applied to the target block is an intra prediction mode included in Category A or Category B. Here, as an example, Category A may include intra prediction mode 2, intra prediction mode 10, intra prediction mode 18, intra prediction mode 26, intra prediction mode 34, intra prediction mode 42, intra prediction mode 50, intra prediction mode 58, and intra prediction mode 66, and Category B may include intra prediction modes other than the intra prediction modes included in Category A. The intra prediction modes included in Category A may be preset, and Category A and Category B may be preset to include intra prediction modes different from the above-described examples.

[0153] Alternatively, as another example, the range of values ​​of the NSST index may be determined based on the AMT factor of the block. The AMT factor may also be referred to as the MTS factor.

[0154] For example, the AMT factor may be defined by the AMT flag described above. In this case, the range of values ​​of the NSST index may be determined based on the value of the AMT flag of the target block.

[0155] Alternatively, the AMT factor may be defined by the AMT vertical flag and / or the AMT horizontal flag described above. In this case, the range of values ​​of the NSST index may be determined based on the values ​​of the AMT vertical flag and / or the AMT horizontal flag of the target block.

[0156] Alternatively, the AMT factor may be defined as a transformation kernel applied in a multi-core transformation. In this case, the range of values ​​of the NSST index may be determined based on the transformation kernel applied in the multi-core transformation of the target block.

[0157] Alternatively, as another example, the range of values ​​of the NSST index may be determined based on the components of the block. For example, the range of values ​​of the NSST index for the luma block of the target block and the range of values ​​of the NSST index for the chroma block of the target block may be applied differently.

[0158] Meanwhile, the range of values ​​of the NSST index may be determined by combining the specific conditions described above.

[0159] The range of values ​​of the NSST index determined based on the above specific conditions, i.e., the maximum value of the NSST index, can be set in various ways.

[0160] For example, based on the specific condition, the maximum value of the NSST index can be determined as R1, R2, or R3. Specifically, when the specific condition corresponds to category A, the maximum value of the NSST index can be derived as R1, when the specific condition corresponds to category B, the maximum value of the NSST index can be derived as R2, and when the specific condition corresponds to category C, the maximum value of the NSST index can be derived as R3.

[0161] R1 for the above category A, R2 for the above category B, and R3 for the above category C can be derived as shown in the following table.

[0162]

[0163] The above R1, R2, and R3 can be preset. For example, the relationship between the above R1, R2, and R3 can be derived as shown in the following mathematical formula.

[0164]

[0165] Referring to mathematical expression 4, R1 may be greater than or equal to 0, R2 may be greater than R1, and R3 may be greater than R2. Meanwhile, if the maximum value of the NSST index for the target block is determined as R1 when R1 is 0, the NSST index may not be signaled, and the value of the NSST index may be inferred as 0.

[0166] Additionally, the present invention proposes an implicit NSST index coding method.

[0167] In general, when NSST is applied, the distribution of non-zero transform coefficients among the transform coefficients may change. In particular, when the reduced secondary transform (RST) is used as a secondary transform under certain conditions, the NSST index may not be coded.

[0168] Here, the RST may represent a second transformation using a simplified transformation matrix as a non-separable transformation matrix, and the simplified transformation matrix may be determined by mapping an N-dimensional vector to an R-dimensional vector located in another space, where R is smaller than N. The N may mean the square of the length of one side of a block to which the transformation is applied or the total number of transformation coefficients corresponding to the block to which the transformation is applied, and the simplified factor may mean an R / N value. The simplified factor may be referred to by various terms such as reduced factor, reduced factor, reduction factor, simplified factor, simple factor, etc. Meanwhile, R may be referred to as a reduced coefficient, but in some cases, the simplified factor may mean R. In addition, in some cases, the simplified factor may mean an N / R value.

[0169] The size of the simplified transformation matrix according to one embodiment is RxN, which is smaller than the size NxN of a normal transformation matrix, and can be defined as in mathematical expression 5 below.

[0170]

[0171] Simplified transformation matrix T for the transformation coefficients to which the first transformation of the target block is applied. RxN When multiplied, (second) transform coefficients for the target block can be derived.

[0172] When the above RST is applied, since a simplified transformation matrix of size RxN is applied to the secondary transformation, the transformation coefficients from R+1 to N can be implicitly 0. In other words, when the transformation coefficients of the target block are derived by applying the above RST, the values ​​of the transformation coefficients from R+1 to N can be 0. Here, the transformation coefficients from R+1 to N can represent the R+1-th transformation coefficient to the N-th transformation coefficient among the transformation coefficients. Specifically, the arrangement of the transformation coefficients of the target block can be described as follows.

[0173] FIG. 8 is a diagram for explaining an arrangement of transform coefficients based on a target block according to an embodiment of the present invention. The descriptions regarding the transform described below in FIG. 8 can also be applied to the inverse transform. For a target block (or residual block, 800), NSST (an example of a secondary transform) based on a primary transform and a simplification transform can be performed. In one example, the 16x16 block illustrated in FIG. 8 represents a target block (800), and the 4x4 blocks indicated as A to P can represent subgroups of the target block (800). The primary transform can be performed on the entire range of the target block (800), and after the primary transform is performed, NSST can be applied to an 8x8 block composed of subgroups A, B, E, and F (hereinafter, the upper left target area). At this time, since only R NSST transform coefficients (where R represents a simplification coefficient and R is smaller than N) are derived when NSST based on simplification transform is performed, the NSST transform coefficients in the range from R+1 to N can be determined as 0. For example, when R is 16, the 16 transform coefficients derived by performing NSST based on simplification transform can be assigned to each block included in subgroup A, which is the upper left 4x4 block included in the upper left target area of ​​the target block (900), and a transform coefficient of 0 can be assigned to each of NR blocks included in subgroups B, E, and F, i.e., 64-16=48 blocks. The primary transform coefficients on which NSST based on simplification transform is not performed can be assigned to each block included in subgroups C, D, G, H, I, J, K, L, M, N, O, and P.

[0174] Therefore, if at least one non-zero transform coefficient is derived by scanning the transform coefficients from R+1 to N, the RST may be determined not to be applied, and the value of the NSST index may be implicitly set to 0 without separate signaling. That is, if at least one non-zero transform coefficient is derived by scanning the transform coefficients from R+1 to N, the RST may not be applied, and the value of the NSST index may be determined to be 0 without separate signaling.

[0175] Figure 9 shows an example of scanning the transformation coefficients from R+1 to N.

[0176] Referring to Fig. 9, the size of the target block to which the transformation is applied may be 64x64, and R=16 (i.e., R / N=16 / 64=1 / 4). That is, Fig. 9 may represent the upper left target area of ​​the target block. A simplified transformation matrix having a size of 16x64 may be applied to the secondary transformation for 64 samples of the upper left target area of ​​the target block. In this case, when the RST is applied to the upper left target area, the values ​​of the transformation coefficients from 17 to 64(N) must be 0. In other words, when at least one non-zero transformation coefficient is derived among the transformation coefficients from 17 to 64 of the target block, the RST is not applied, and the value of the NSST index may be derived as 0 without separate signaling. Accordingly, the decoding device can decode the transform coefficients of the target block, scan the transform coefficients from 17 to 64 among the decoded transform coefficients, and if a non-zero transform coefficient is derived, derive the value of the NSST index as 0 without signaling a separate syntax element for the NSST index. Meanwhile, if there is no non-zero transform coefficient among the transform coefficients from 17 to 64, the decoding device can receive and decode the NSST index.

[0177] FIG. 10a and FIG. 10b are flowcharts illustrating a coding process of an NSST index according to one embodiment.

[0178] Figure 10a illustrates the encoding process of the NSST index.

[0179] The encoding device can encode transform coefficients for a target block (S1000). The encoding device can perform entropy encoding on the quantized transform coefficients. The entropy encoding can include encoding methods such as, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC).

[0180] The encoding device can determine whether an (explicit) NSST index for a target block is coded (S1010). Here, the explicit NSST index may indicate an NSST index transmitted to a decoding device. That is, the encoding device can determine whether to generate a signaled NSST index. In other words, the encoding device can determine whether to allocate bits for a syntax element for the NSST index. If the decoding device can derive the value of the NSST index even if the NSST index is not signaled, as in the above-described embodiment, the encoding device may not code the NSST index. A specific process for determining whether the NSST index is coded will be described later.

[0181] If it is determined that the above (explicit) NSST index is coded, the encoding device can encode the NSST index (S1020).

[0182] Figure 10b illustrates the decoding process of the NSST index.

[0183] The decoding device can decode transform coefficients for the target block (S1030).

[0184] The decoding device can determine whether an (explicit) NSST index for the target block is coded (S1040). Here, the explicit NSST index may represent an NSST index signaled from an encoding device. If, as in the above-described embodiment, the decoding device can derive the value of the NSST index even if the NSST index is not signaled, the NSST index may not be signaled from the encoding device. The specific process for determining whether the NSST index is coded will be described later.

[0185] If it is determined that the above (explicit) NSST index is coded, the encoding device can decode the NSST index (S1040).

[0186] Figure 11 shows an example of determining whether an NSST index is coded.

[0187] The encoding device / decoding device can determine whether a condition for coding an NSST index for a target block is met (S1100). For example, if the cbf flag for the target block indicates 0, the encoding device / decoding device can determine not to code the NSST index for the target block. Alternatively, if the target block is coded in a transform skip mode or the number of non-zero transform coefficients among the transform coefficients for the target block is less than a preset threshold, the encoding device / decoding device can determine not to code the NSST index for the target block. For example, the preset threshold may be 2.

[0188] If the condition for coding the NSST index for the above target block is met, the encoding device / decoding device can scan the transform coefficients from R+1 to N (S1110). The transform coefficients from R+1 to N can represent the transform coefficients from R+1 to N in the scan order among the transform coefficients.

[0189] The encoding device / decoding device can determine whether a non-zero transform coefficient is derived from among the transform coefficients from R+1 to N (S1120). If a non-zero transform coefficient is derived from among the transform coefficients from R+1 to N, the encoding device / decoding device can determine not to code the NSST index for the target block. In this case, the encoding device / decoding device can derive the value of the NSST index for the target block as 0. In other words, for example, if the NSST index having a value of 0 indicates that NSST is not applied, the encoding device / decoding device may not perform NSST for the upper left target area of ​​the target block.

[0190] Meanwhile, if a non-zero transform coefficient is not derived among the transform coefficients from R+1 to N, the encoding device can encode the NSST index for the target block, and the decoding device can decode the NSST index for the target block.

[0191] Meanwhile, a method may be proposed in which the components of the target block (luma component, chroma Cb component, chroma Cr component) use the common NSST index.

[0192] For example, the same NSST index may be used for the chroma Cb block of the target block and the chroma Cr block of the target block. Also, as another example, the same NSST index may be used for the luma block of the target block, the chroma Cb block of the target block, and the chroma Cr block of the target block.

[0193] When two or three components of the target block use the same NSST index, the encoding device can scan transform coefficients from R+1 to N of all components (luma block, chroma Cb block, chroma Cr block of the target block), and when at least one non-zero transform coefficient is derived, the encoding device can derive the value of the NSST index as 0 without encoding the NSST index. In addition, the decoding device can scan transform coefficients from R+1 to N of all components (luma block, chroma Cb block, chroma Cr block of the target block), and when at least one non-zero transform coefficient is derived, the decoding device can derive the value of the NSST index as 0 without decoding the NSST index.

[0194] Figure 12 shows an example of scanning the transform coefficients from R+1 to N for all components of the target block.

[0195] Referring to Fig. 12, the sizes of the luma block, chroma Cb block, and chroma Cr block of the target block to which the transformation is applied may be 64x64, and R=16 (i.e., R / N=16 / 64=1 / 4). That is, Fig. 12 illustrates the upper left target area of ​​the luma block, the upper left target area of ​​the chroma Cb block, and the upper left target area of ​​the chroma Cr block. Accordingly, a simplified transformation matrix having a size of 16x64 may be applied to the secondary transformation for 64 samples of each of the upper left target area of ​​the luma block, the upper left target area of ​​the chroma Cb block, and the upper left target area of ​​the chroma Cr block. In this case, when the RST is applied to the upper left target area of ​​the luma block, the upper left target area of ​​the chroma Cb block, and the upper left target area of ​​the chroma Cr block, the values ​​of the transform coefficients from 17 to 64 (N) of each block must be 0. In other words, when at least one non-zero transform coefficient is derived from the transform coefficients from 17 to 64 of each block, the RST is not applied, and the value of the NSST index can be derived as 0 without separate signaling. Accordingly, the decoding device can decode transform coefficients for all components of the target block, and scan transform coefficients from 17 to 64 of the luma block, the chroma Cb block, and the chroma Cr block among the decoded transform coefficients, and when a non-zero transform coefficient is derived, the value of the NSST index can be derived as 0 without signaling a separate syntax element for the NSST index. Meanwhile, if there is no non-zero transform coefficient among the transform coefficients from 17 to 64, the decoding device can receive and decode the NSST index. The NSST index can be used as an index for the luma block, the chroma Cb block, and the chroma Cr block.

[0196] In addition, the present invention may propose a method for signaling an NSST index indicator at a higher level. NSST_Idx_indicator may represent a syntax element for the NSST index indicator. For example, the NSST index indicator may be coded at a CTU (Coding Tree Unit) level, and the NSST index indicator may indicate whether NSST is applied to a target CTU. That is, the NSST index indicator may indicate whether NSST is available for the target CTU. Specifically, when the NSST index indicator for the target CTU is enabled (when NSST is available for the target CTU), that is, when the value of the NSST index indicator is 1, an NSST index for a CU or TU included in the target CTU may be coded. If the NSST index indicator for the target CTU is not activated (NSST is not available for the target CTU), i.e., if the value of the NSST index indicator is 0, the NSST index for the CU or TU included in the target CTU may not be coded. Meanwhile, the NSST index indicator may be coded at the CTU level as described above, or may be coded at the sample group level of any other arbitrary size. For example, the NSST index indicator may be coded at the CU (Coding Unit) level.

[0197] Fig. 13 schematically illustrates a video encoding method using an encoding device according to the present invention. The method disclosed in Fig. 13 can be performed by the encoding device disclosed in Fig. 1. Specifically, for example, S1300 of Fig. 13 can be performed by a subtraction unit of the encoding device, S1310 can be performed by a conversion unit of the encoding device, and S1320 to S1330 can be performed by an entropy encoding unit of the encoding device. In addition, although not illustrated, a process of deriving a prediction sample can be performed by a prediction unit of the encoding device.

[0198] An encoding device derives residual samples of a target block (S1300). For example, the encoding device may determine whether to perform inter prediction or intra prediction on the target block, and may determine a specific inter prediction mode or a specific intra prediction mode based on RD cost. Depending on the determined mode, the encoding device may derive prediction samples for the target block, and may derive the residual samples by adding the original samples of the target block and the prediction samples.

[0199] The encoding device performs a transformation on the residual samples to derive the transformation coefficients of the target block (S1310). The encoding device can determine whether to apply NSST to the target block.

[0200] When the NSST is applied to the target block, the encoding device can perform a core transform on the residual samples to derive modified transform coefficients, and perform NSST on modified transform coefficients located in the upper left target area of ​​the target block based on a simplified transform matrix to derive the transform coefficients of the target block. Modified transform coefficients other than the modified transform coefficients located in the upper left area of ​​the target block can be derived as the transform coefficients of the target block. The size of the simplified transform matrix can be RxN, where N can be the number of samples of the upper left target area, R can be a reduced coefficient, and R can be smaller than N.

[0201] Specifically, the core transform for the residual samples can be performed as follows. The encoding device can determine whether to apply an adaptive multiple core transform (AMT) to the target block. In this case, an AMT flag indicating whether the adaptive multiple core transform of the target block is applied can be generated. If the AMT is not applied to the target block, the encoding device can derive DCT type 2 as a transform kernel for the target block, and perform a transform on the residual samples based on the DCT type 2 to derive the modified transform coefficients.

[0202] When the AMT is applied to the target block, the encoding device may configure a transform subset for a horizontal transform kernel and a transform subset for a vertical transform kernel, derive a horizontal transform kernel and a vertical transform kernel based on the transform subsets, and perform a transform on the residual samples based on the horizontal transform kernel and the vertical transform kernel to derive modified transform coefficients. Here, the transform subset for the horizontal transform kernel and the transform subset for the vertical transform kernel may include DCT type 2, DST type 7, DCT type 8, and / or DST type 1 as candidates. In addition, transform index information may be generated, and the transform index information may include an AMT horizontal flag indicating the horizontal transform kernel and an AMT vertical flag indicating the vertical transform kernel. Meanwhile, the transform kernel may be referred to as a transform type or a transform core.

[0203] Meanwhile, if the NSST is not applied to the target block, the encoding device can perform a core transform on the residual samples to derive the transform coefficients of the target block.

[0204] Specifically, the core transform for the residual samples can be performed as follows. The encoding device can determine whether to apply an adaptive multiple core transform (AMT) to the target block. In this case, an AMT flag indicating whether the adaptive multiple core transform of the target block is applied can be generated. If the AMT is not applied to the target block, the encoding device can derive DCT type 2 as a transform kernel for the target block, and perform a transform on the residual samples based on the DCT type 2 to derive the transform coefficients.

[0205] When the AMT is applied to the target block, the encoding device may configure a transform subset for a horizontal transform kernel and a transform subset for a vertical transform kernel, derive a horizontal transform kernel and a vertical transform kernel based on the transform subsets, and perform a transform on the residual samples based on the horizontal transform kernel and the vertical transform kernel to derive transform coefficients. Here, the transform subset for the horizontal transform kernel and the transform subset for the vertical transform kernel may include DCT type 2, DST type 7, DCT type 8, and / or DST type 1 as candidates. In addition, transform index information may be generated, and the transform index information may include an AMT horizontal flag indicating the horizontal transform kernel and an AMT vertical flag indicating the vertical transform kernel. Meanwhile, the transform kernel may be called a transform type or a transform core.

[0206] The encoding device determines whether to encode the NSST index (S1320).

[0207] For example, the encoding device may scan the (R+1)th to (N)th transform coefficients among the transform coefficients of the target block, and if the (R+1)th to (N)th transform coefficients include a non-zero transform coefficient, the encoding device may determine not to encode the NSST index. Here, N is the number of samples of the upper left target area, R is a reduced coefficient, and R may be smaller than N. N may be derived as the product of the width and the height of the upper left target area.

[0208] In addition, if the R+1th to Nth transform coefficients do not include a non-zero transform coefficient, the encoding device may determine to encode the NSST index. In this case, information about the transform coefficients may include a syntax element for the NSST index. That is, the syntax element for the NSST index may be encoded. In other words, a bit for a syntax element for the NSST index may be allocated.

[0209] Meanwhile, the encoding device may determine whether a condition for performing the NSST is met, and if the NSST can be performed, may determine to encode the NSST index for the target block. For example, an NSST index indicator for a target CTU including the target block may be generated from a bitstream, and the NSST index indicator may indicate whether NSST is applied to the target CTU. If the value of the NSST index indicator is 1, the encoding device may determine to encode the NSST index for the target block, and if the value of the NSST index indicator is 0, the decoding device may determine not to encode the NSST index for the target block. As in the example described above, the NSST index indicator may be signaled at a CTU level, or the NSST index indicator may be signaled at a CU level or another higher level.

[0210] Additionally, the NSST index can be used for multiple components of the target block.

[0211] For example, the NSST index may be used for inverse transformation for transform coefficients of a luma block of the target block, transform coefficients of a chroma Cb block, and transform coefficients of a chroma Cr block. In this case, the (R+1)-th to (N)-th transform coefficients of the luma block, the (R+1)-th to (N)-th transform coefficients of the chroma Cb block, and the (R+1)-th to (N)-th transform coefficients of the chroma Cr block may be scanned, and if a non-zero transform coefficient is included in the scanned transform coefficients, the NSST index may be determined not to be encoded. If a non-zero transform coefficient is not included in the scanned transform coefficients, the NSST index may be determined to be encoded. In this case, information about the transform coefficients may include a syntax element for the NSST index. That is, the syntax element for the NSST index may be encoded. In other words, a bit may be allocated for a syntax element for the NSST index.

[0212] As another example, the NSST index may be used for inverse transformation for transform coefficients of a luma block and transform coefficients of a chroma Cb block of the target block. In this case, the (R+1)-th to (N)-th transform coefficients of the luma block and the (R+1)-th to (N)-th transform coefficients of the chroma Cb block may be scanned, and if a non-zero transform coefficient is included in the scanned transform coefficients, the NSST index may be determined not to be encoded. If a non-zero transform coefficient is not included in the scanned transform coefficients, the NSST index may be determined to be encoded. In this case, information about the transform coefficients may include a syntax element for the NSST index. That is, the syntax element for the NSST index may be encoded. In other words, a bit may be allocated for a syntax element for the NSST index.

[0213] As another example, the NSST index may be used for inverse transformation for transform coefficients of a luma block and transform coefficients of a chroma Cr block of the target block. In this case, the (R+1)-th to (N)-th transform coefficients of the luma block and the (R+1)-th to (N)-th transform coefficients of the chroma Cr block may be scanned, and if a non-zero transform coefficient is included in the scanned transform coefficients, the NSST index may be determined not to be encoded. If a non-zero transform coefficient is not included in the scanned transform coefficients, the NSST index may be determined to be encoded. In this case, information about the transform coefficients may include a syntax element for the NSST index. That is, the syntax element for the NSST index may be encoded. In other words, a bit may be allocated for a syntax element for the NSST index.

[0214] Meanwhile, the range of the NSST index can be derived based on specific conditions. For example, the maximum value of the NSST index can be derived based on the specific conditions, and the range can be derived from 0 to the derived maximum value. The value of the derived NSST index can be included within the range.

[0215] For example, the range of the NSST index can be derived based on the size of the target block. Specifically, a minimum width and a minimum height can be preset, and the range of the NSST index can be derived based on the width and the minimum width of the target block, and the height and the minimum height of the target block. In addition, the range of the NSST index can be derived based on the number of samples of the target block and a specific value. The number of samples can be a value obtained by multiplying the width and the height of the target block, and the specific value can be preset.

[0216] In addition, as another example, the range of the NSST index can be derived based on the type of the target block. Specifically, the range of the NSST index can be derived based on whether the target block is a non-square block. In addition, the range of the NSST index can be derived based on the ratio and specific value between the width and height of the target block. The ratio between the width and height of the target block can be a value obtained by dividing the longer side by the shorter side among the width and height of the target block, and the specific value can be preset.

[0217] In addition, as another example, the range of the NSST index can be derived based on the intra prediction mode of the target block. Specifically, the range of the NSST index can be derived based on whether the intra prediction mode of the target block is a non-directional intra prediction mode or a directional intra prediction mode. In addition, the range of the NSST index can be derived based on whether the intra prediction mode of the target block is an intra prediction mode included in Category A or Category B. Here, the intra prediction mode included in Category A and the intra prediction mode included in Category B can be preset. For example, the above category A may include intra prediction mode 2, intra prediction mode 10, intra prediction mode 18, intra prediction mode 26, intra prediction mode 34, intra prediction mode 42, intra prediction mode 50, intra prediction mode 58, and intra prediction mode 66, and the above category B may include intra prediction modes other than the intra prediction modes included in the above category A.

[0218] Additionally, as another example, the range of the NSST index can be derived based on information about the core transform of the target block. For example, the range of the NSST index can be derived based on an Adaptive Multiple Core Transform (AMT) flag indicating whether an AMT is applied. Additionally, the range of the NSST index can be derived based on an AMT Horizontal flag indicating a horizontal transform kernel and an AMT Vertical flag indicating a vertical transform kernel.

[0219] Meanwhile, if the value of the NSST index is 0, the NSST index may indicate that NSST is not applied to the target block.

[0220] The encoding device encodes information on transform coefficients (S1330). The information on the transform coefficients may include information on the size, position, etc. of the transform coefficients. In addition, as described above, the information on the transform coefficients may further include the NSST index, the transform index information, and / or the AMT flag. Image information including the information on the transform coefficients may be output in the form of a bitstream. In addition, the image information may further include the NSST index indicator and / or prediction information. The prediction information may include information related to the prediction procedure, such as prediction mode information and information on motion information (e.g., when inter prediction is applied).

[0221] The output bitstream can be transmitted to a decoding device via a storage medium or network.

[0222] Fig. 14 schematically illustrates an encoding device that performs an image encoding method according to the present invention. The method disclosed in Fig. 13 can be performed by the encoding device disclosed in Fig. 14. Specifically, for example, the addition unit of the encoding device of Fig. 14 can perform S1300 of Fig. 13, the conversion unit of the encoding device can perform S1310, and the entropy encoding unit of the encoding device can perform S1320 to S1330 of Fig. 13. In addition, although not illustrated, the process of deriving a prediction sample can be performed by the prediction unit of the encoding device.

[0223] Fig. 15 schematically illustrates an image decoding method by a decoding device according to the present invention. The method disclosed in Fig. 15 can be performed by the decoding device disclosed in Fig. 3. Specifically, for example, S15400 to S1510 of Fig. 15 can be performed by the entropy decoding unit of the decoding device, S1520 by the inverse transform unit of the decoding device, and S1530 by the adding unit of the decoding device. In addition, although not illustrated, the process of deriving a prediction sample can be performed by the prediction unit of the decoding device.

[0224] The decoding device derives the transform coefficients of the target block from the bitstream (S1500). The decoding device can derive the transform coefficients of the target block by decoding information about the transform coefficients of the target block received through the bitstream. The received information about the transform coefficients of the target block can be expressed as residual information.

[0225] Meanwhile, the transform coefficients of the target block may include transform coefficients of a luma block of the target block, transform coefficients of a chroma Cb block of the target block, and transform coefficients of a chroma Cr block of the target block.

[0226] The decoding device derives a Non-Separable Secondary Transform (NSST) index for the target block (S1510).

[0227] For example, the decoding device can scan the (R+1)th to (N)th transform coefficients among the transform coefficients of the target block, and if the (R+1)th to (N)th transform coefficients include a non-zero transform coefficient, the value of the NSST index can be derived as 0. Here, N is the number of samples of the upper left target area of ​​the target block, R is a reduced coefficient, and R can be smaller than N. N can be derived as the product of the width and the height of the upper left target area.

[0228] In addition, if the R+1-th to N-th transform coefficients do not include a non-zero transform coefficient, the decoding device can derive the value of the NSST index by parsing a syntax element for the NSST index included in the bitstream. That is, if the R+1-th to N-th transform coefficients do not include a non-zero transform coefficient, the bitstream can include a syntax element for the NSST index, and the decoding device can derive the value of the NSST index by parsing a syntax element for the NSST index received through the bitstream.

[0229] Meanwhile, the decoding device can determine whether a condition for performing the NSST is met, and if the NSST can be performed, the decoding device can derive an NSST index for the target block. For example, an NSST index indicator for a target CTU including the target block can be signaled from a bitstream, and the NSST index indicator can indicate whether NSST is enabled for the target CTU. When the value of the NSST index indicator is 1, the decoding device can derive the NSST index for the target block, and when the value of the NSST index indicator is 0, the decoding device may not derive the NSST index for the target block. As in the example described above, the NSST index indicator can be signaled at a CTU level, or the NSST index indicator can be signaled at a CU level or another higher level.

[0230] Additionally, the NSST index can be used for multiple components of the target block.

[0231] For example, the NSST index may be used for inverse transformation for transform coefficients of a luma block of the target block, transform coefficients of a chroma Cb block, and transform coefficients of a chroma Cr block. In this case, the (R+1)-th to (N)-th transform coefficients of the luma block, the (R+1)-th to (N)-th transform coefficients of the chroma Cb block, and the (R+1)-th to (N)-th transform coefficients of the chroma Cr block may be scanned, and if a non-zero transform coefficient is included in the scanned transform coefficients, the value of the NSST index may be derived as 0. If a non-zero transform coefficient is not included in the scanned transform coefficients, the bitstream may include a syntax element for the NSST index, and the value of the NSST index may be derived by parsing a syntax element for the NSST index received through the bitstream.

[0232] As another example, the NSST index may be used for inverse transformation for transform coefficients of a luma block and transform coefficients of a chroma Cb block of the target block. In this case, the (R+1)-th to (N)-th transform coefficients of the luma block and the (R+1)-th to (N)-th transform coefficients of the chroma Cb block may be scanned, and if a non-zero transform coefficient is included in the scanned transform coefficients, the value of the NSST index may be derived as 0. If a non-zero transform coefficient is not included in the scanned transform coefficients, the bitstream may include a syntax element for the NSST index, and the value of the NSST index may be derived by parsing a syntax element for the NSST index received through the bitstream.

[0233] As another example, the NSST index may be used for inverse transformation for transform coefficients of a luma block and transform coefficients of a chroma Cr block of the target block. In this case, the (R+1)-th to (N)-th transform coefficients of the luma block and the (R+1)-th to (N)-th transform coefficients of the chroma Cr block may be scanned, and if a non-zero transform coefficient is included in the scanned transform coefficients, the value of the NSST index may be derived as 0. If a non-zero transform coefficient is not included in the scanned transform coefficients, the bitstream may include a syntax element for the NSST index, and the value of the NSST index may be derived by parsing a syntax element for the NSST index received through the bitstream.

[0234] Meanwhile, the range of the NSST index can be derived based on specific conditions. For example, the maximum value of the NSST index can be derived based on the specific conditions, and the range can be derived from 0 to the derived maximum value. The value of the derived NSST index can be included within the range.

[0235] For example, the range of the NSST index can be derived based on the size of the target block. Specifically, a minimum width and a minimum height can be preset, and the range of the NSST index can be derived based on the width and the minimum width of the target block, and the height and the minimum height of the target block. In addition, the range of the NSST index can be derived based on the number of samples of the target block and a specific value. The number of samples can be a value obtained by multiplying the width and the height of the target block, and the specific value can be preset.

[0236] In addition, as another example, the range of the NSST index can be derived based on the type of the target block. Specifically, the range of the NSST index can be derived based on whether the target block is a non-square block. In addition, the range of the NSST index can be derived based on the ratio and specific value between the width and height of the target block. The ratio between the width and height of the target block can be a value obtained by dividing the longer side by the shorter side among the width and height of the target block, and the specific value can be preset.

[0237] In addition, as another example, the range of the NSST index can be derived based on the intra prediction mode of the target block. Specifically, the range of the NSST index can be derived based on whether the intra prediction mode of the target block is a non-directional intra prediction mode or a directional intra prediction mode. In addition, the range of the NSST index can be derived based on whether the intra prediction mode of the target block is an intra prediction mode included in Category A or Category B. Here, the intra prediction mode included in Category A and the intra prediction mode included in Category B can be preset. For example, the above category A may include intra prediction mode 2, intra prediction mode 10, intra prediction mode 18, intra prediction mode 26, intra prediction mode 34, intra prediction mode 42, intra prediction mode 50, intra prediction mode 58, and intra prediction mode 66, and the above category B may include intra prediction modes other than the intra prediction modes included in the above category A.

[0238] Additionally, as another example, the range of the NSST index can be derived based on information about the core transform of the target block. For example, the range of the NSST index can be derived based on an Adaptive Multiple Core Transform (AMT) flag indicating whether an AMT is applied. Additionally, the range of the NSST index can be derived based on an AMT Horizontal flag indicating a horizontal transform kernel and an AMT Vertical flag indicating a vertical transform kernel.

[0239] Meanwhile, if the value of the NSST index is 0, the NSST index may indicate that NSST is not applied to the target block.

[0240] The decoding device performs an inversed transform on the transform coefficients of the target block based on the NSST index to derive residual samples of the target block (S1520).

[0241] For example, if the value of the NSST index is 0, the decoding device can perform a core transform on the transform coefficients of the target block to derive the residual samples.

[0242] Specifically, the decoding device can obtain an Adaptive Multiple Core Transform (AMT) flag from the bitstream indicating whether an AMT is applied.

[0243] When the value of the above AMT flag is 0, the decoding device can derive DCT type 2 as a transform kernel for the target block, and perform an inverse transform on the transform coefficients based on the DCT type 2 to derive the residual samples.

[0244] When the value of the AMT flag is 1, the decoding device may configure a transform subset for a horizontal transform kernel and a transform subset for a vertical transform kernel, derive a horizontal transform kernel and a vertical transform kernel based on transform index information obtained from the bitstream and the transform subsets, and perform an inverse transform on the transform coefficients based on the horizontal transform kernel and the vertical transform kernel to derive the residual samples. Here, the transform subset for the horizontal transform kernel and the transform subset for the vertical transform kernel may include DCT type 2, DST type 7, DCT type 8, and / or DST type 1 as candidates. In addition, the transform index information may include an AMT horizontal flag indicating one of the candidates included in the transform subset for the horizontal transform kernel, and an AMT vertical flag indicating one of the candidates included in the transform subset for the vertical transform kernel. Meanwhile, the transform kernel may be referred to as a transform type or a transform core.

[0245] As another example, when the value of the NSST index is not 0, the decoding device may perform NSST on transform coefficients located in the upper left target area of ​​the target block based on a reduced transform matrix indicated by the NSST index to derive modified transform coefficients, and may perform a core transform on the target block including the modified transform coefficients to derive the residual samples. The size of the reduced transform matrix may be RxN, where N may be the number of samples in the upper left target area, R may be a reduced coefficient, and R may be smaller than N.

[0246] The core transform for the target block can be performed as follows. The decoding device can obtain an AMT flag indicating whether an adaptive multiple core transform (AMT) is applied from a bitstream, and when the value of the AMT flag is 0, the decoding device can derive DCT type 2 as a transform kernel for the target block, and perform an inverse transform for the target block including the modified transform coefficients based on the DCT type 2 to derive the residual samples.

[0247] When the value of the AMT flag is 1, the decoding device may configure a transform subset for a horizontal transform kernel and a transform subset for a vertical transform kernel, derive a horizontal transform kernel and a vertical transform kernel based on transform index information obtained from the bitstream and the transform subsets, and perform an inverse transform on the target block including the modified transform coefficients based on the horizontal transform kernel and the vertical transform kernel to derive the residual samples. Here, the transform subset for the horizontal transform kernel and the transform subset for the vertical transform kernel may include DCT type 2, DST type 7, DCT type 8, and / or DST type 1 as candidates. In addition, the transform index information may include an AMT horizontal flag indicating one of the candidates included in the transform subset for the horizontal transform kernel and an AMT vertical flag indicating one of the candidates included in the transform subset for the vertical transform kernel. Meanwhile, the transform kernel may be referred to as a transform type or a transform core.

[0248] The decoding device generates a reconstructed picture based on the residual samples (S1530). The decoding device can generate the reconstructed picture based on the residual samples. For example, the decoding device can perform inter-prediction or intra-prediction on a target block based on prediction information received through a bitstream and derive prediction samples, and can generate the reconstructed picture by adding the prediction samples and the residual samples. As described above, an in-loop filtering procedure, such as deblocking filtering, SAO, and / or ALF procedure, can be applied to the reconstructed picture to improve subjective / objective image quality as needed.

[0249] Fig. 16 schematically illustrates a decoding device that performs a video decoding method according to the present invention. The method disclosed in Fig. 15 can be performed by the decoding device disclosed in Fig. 16. Specifically, for example, the entropy decoding unit of the decoding device of Fig. 16 can perform S1500 to S1510 of Fig. 15, the inverse transform unit of the decoding device of Fig. 16 can perform S1520 of Fig. 15, and the adding unit of the decoding device of Fig. 16 can perform S1530 of Fig. 15. In addition, although not illustrated, a process of deriving a prediction sample can be performed by the prediction unit of the decoding device of Fig. 16.

[0250] According to the present invention described above, the range of the NSST index can be derived based on specific conditions of the target block, thereby reducing the amount of bits for the NSST index and improving the overall coding efficiency.

[0251] In addition, according to the present invention, transmission of a syntax element for an NSST index can be determined based on transform coefficients for a target block, thereby reducing the amount of bits for the NSST index and improving overall coding efficiency.

[0252] While the methods described in the above-described embodiments are described based on a flowchart as a series of steps or blocks, the present invention is not limited to the order of the steps, and certain steps may occur in a different order or simultaneously with other steps described above. Furthermore, those skilled in the art will appreciate that the steps depicted in the flowchart are not exclusive, and that other steps may be included, or one or more steps in the flowchart may be deleted, without affecting the scope of the present invention.

[0253] The method according to the present invention described above can be implemented in the form of software, and the encoding device and / or decoding device according to the present invention can be included in a device that performs image processing, such as a TV, a computer, a smartphone, a set-top box, a display device, etc.

[0254] When the embodiments of the present invention are implemented in software, the above-described method can be implemented as a module (process, function, etc.) that performs the above-described function. The module can be stored in memory and executed by a processor. The memory can be internal or external to the processor and can be connected to the processor by various well-known means. The processor can include an application-specific integrated circuit (ASIC), another chipset, logic circuit, and / or data processing device. The memory can include a read-only memory (ROM), a random access memory (RAM), flash memory, a memory card, a storage medium, and / or other storage devices. That is, the embodiments described in the present invention can be implemented and performed on a processor, a microprocessor, a controller, or a chip. For example, the functional units illustrated in each drawing can be implemented and performed on a computer, a processor, a microprocessor, a controller, or a chip.

[0255] In addition, the decoding device and encoding device to which the present invention is applied may be included in a multimedia broadcasting transmitting and receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video conversation device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a video-on-demand (VoD) service providing device, an OTT video (Over the top video) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a video phone video device, and a medical video device, and may be used to process a video signal or a data signal. For example, the OTT video (Over the top video) device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, a DVR (Digital Video Recorder), and the like.

[0256] In addition, the processing method to which the present invention is applied can be produced in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present invention can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium may include, for example, Blu-ray Disc (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer-readable recording medium includes media implemented in the form of a carrier wave (e.g., transmission via the Internet). In addition, a bitstream generated by an encoding method can be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network. In addition, embodiments of the present invention can be implemented as a computer program product by program code, and the program code can be executed on a computer according to embodiments of the present invention. The above program code can be stored on a computer-readable carrier.

[0257] In addition, the content streaming system to which the present invention is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0258] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, camcorders, etc. into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices such as smartphones, cameras, camcorders, etc. directly generate a bitstream, the encoding server may be omitted. The bitstream may be generated by an encoding method or a bitstream generation method to which the present invention is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0259] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits the multimedia data to the user. At this time, the content streaming system may include a separate control server, in which case the control server controls commands / responses between each device within the content streaming system.

[0260] The streaming server can receive content from a media repository and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0261] Examples of the user devices may include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc. Each server in the content streaming system may be operated as a distributed server, in which case data received from each server may be distributedly processed.

Claims

1. In a video decoding method performed by a decoding device, A step of deriving transform coefficients of a target block from a bitstream; A step of deriving a Non-Separable Secondary Transform (NSST) index for the target block; A step of deriving residual samples of the target block by performing an inversed transform on the transform coefficients of the target block based on the NSST index; and An image decoding method, characterized by comprising a step of generating a restored picture based on the residual samples.

2. In paragraph 1, The step of deriving the residual samples of the target block by performing an inversed transform on the transform coefficients of the target block based on the NSST index is as follows: When the value of the NSST index is 0, a step of performing a core transform on the transform coefficients of the target block to derive the residual samples; and An image decoding method, characterized in that it comprises a step of performing NSST on transform coefficients located in the upper left target area of ​​the target block based on a reduced transform matrix indicated by the NSST index when the value of the NSST index is not 0, to derive modified transform coefficients, and performing a core transform on the target block including the modified transform coefficients to derive the residual samples.

3. In paragraph 2, The size of the above simplified transformation matrix is ​​RxN, An image decoding method, wherein N is the number of samples in the upper left target area, R is a reduced coefficient, and R is smaller than N.

4. In paragraph 3, The step of deriving the NSST index for the target block is: A step of scanning the R+1th to Nth transform coefficients among the transform coefficients of the target block; and An image decoding method comprising a step of deriving the value of the NSST index as 0 when the R+1th to Nth transform coefficients include a non-zero transform coefficient.

5. In paragraph 3, The step of deriving the NSST index for the target block is: A video decoding method characterized by further comprising a step of parsing a syntax element for the NSST index included in the bitstream to derive a value of the NSST index when the R+1th to Nth transform coefficients do not include a non-zero transform coefficient.

6. In the above paragraph 1, The range of the NSST index is derived based on the width and minimum width of the target block, and the height and minimum height of the target block. An image decoding method characterized in that the minimum width and the minimum height are preset.

7. In paragraph 1, The range of the NSST index is derived based on the number of samples and specific values ​​of the target block, An image decoding method characterized in that the number of samples is a value obtained by multiplying the width and height of the target block, and the specific value is preset.

8. In paragraph 1, An image decoding method characterized in that the range of the NSST index is derived based on whether the target block is a non-square block.

9. In paragraph 3, When the above NSST index is used for inverse transformation for the transform coefficients of the luma block of the target block, the transform coefficients of the chroma Cb block, and the transform coefficients of the chroma Cr block, the R+1 to N-th transform coefficients of the luma block, the R+1 to N-th transform coefficients of the chroma Cb block, and the R+1 to N-th transform coefficients of the chroma Cr block are scanned, An image decoding method, characterized in that if the scanned transform coefficients include a non-zero transform coefficient, the value of the NSST index is derived as 0.

10. In paragraph 1, An NSST index indicator for a target CTU containing the target block is signaled from the bitstream, A video decoding method, characterized in that the above NSST index indicator indicates whether NSST is available (enabled) in the target CTU.

11. In the video decoding device, An entropy decoding unit that derives transform coefficients of a target block from a bitstream and derives an NSST (Non-Separable Secondary Transform) index for the target block; An inverse transform unit that performs an inverse transform on the transform coefficients of the target block based on the NSST index to derive residual samples of the target block; and An image decoding device characterized by including an addition unit that generates a restored picture based on the residual samples.

12. In a video encoding method performed by an encoding device, A step of deriving residual samples of a target block; A step of performing a transform on the residual samples to derive transform coefficients of the target block; A step of determining whether to encode the NSST index for the target block; and Including a step of encoding information about the above transformation coefficients, The step of determining whether to encode the above NSST index is: A step of scanning the R+1th to Nth transform coefficients among the transform coefficients of the target block; and Including a step of determining not to encode the NSST index if the R+1th to Nth transform coefficients include a non-zero transform coefficient, An image encoding method characterized in that the above N is the number of samples in the upper left target area of ​​the above target block, the above R is a reduced coefficient, and the above R is smaller than the above N.

13. In paragraph 12, The step of determining whether to encode the above NSST index is: If the R+1th to Nth transform coefficients do not include a non-zero transform coefficient, the step of determining to encode the NSST index is further included. A video encoding method, characterized in that when the R+1th to Nth transform coefficients do not include a non-zero transform coefficient, information about the transform coefficients includes a syntax element for the NSST index.

14. In paragraph 12, The step of performing a transformation on the residual samples to derive the transformation coefficients of the target block is as follows: A step of determining whether to apply NSST to the above target block; When the NSST is applied to the target block, a step of performing a core transformation on the residual samples to derive modified transform coefficients, and performing NSST on the modified transform coefficients located in the upper left target area based on a simplified transform matrix to derive the transform coefficients of the target block; and An image encoding method, characterized in that it comprises a step of performing a core transform on the residual samples to derive the transform coefficients of the target block when the NSST is not applied to the target block.

15. In paragraph 12, The range of the NSST index is derived based on the width and minimum width of the target block, and the height and minimum height of the target block. An image encoding method characterized in that the minimum width and the minimum height are preset.

Images