Method and device for processing video signal by using reduced secondary transform

Abstract

To provide a method and device of processing a video signal for significantly reducing the computational complexity, enhancing the performance of a transform index, and enhancing the coding efficiency.SOLUTION: A method includes: obtaining a secondary transform index from a video signal; deriving a secondary transform corresponding to the secondary transform index, where the secondary transform represents a reduced secondary transform, and the reduced secondary transform represents a transform outputting L (L<N) transform coefficient data (L×1 transform coefficient vectors) based on inputted N residual data (N×1 residual vectors); obtaining a transform coefficient block by performing entropy decoding and dequantization for a current block (N×N) in parallel; performing an inverse secondary transform for the transform coefficient block using the reduced secondary transform; performing an inverse primary transform for a block which the inverse secondary transform is applied to; and reconstructing the current block using a block which the inverse primary transform is applied to.SELECTED DRAWING: Figure 21

Description

[Technical field]

[0001] The present invention relates to a method and apparatus for processing a video signal, and more particularly to a design of a Reduced Secondary Transform (RST) that can be applied to a 4x4 block, an arrangement and scanning order of transform coefficients generated after applying the 4x4 RST, and a transform index coding method for specifying the 4x4 RST to be applied. [Background technology]

[0002] Next-generation video content is characterized by high spatial resolution, high frame rate, and high dimensionality of scene representation. Processing such content will bring about a significant increase in memory storage, memory access rate, and processing power.

[0003] Therefore, there is a need to design coding tools to process next-generation video content more efficiently. In particular, when applying transforms, there is a need to design transforms that are much more efficient in terms of coding efficiency and complexity. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention proposes an encoder / decoder structure to reflect the new transform design.

[0005] The present invention proposes a design of a RST that can be applied to a 4x4 block, a transform index coding method and structure for specifying the arrangement and scanning order of transform coefficients generated after applying the 4x4 RST, and the 4x4 RST to be applied. [Means for solving the problem]

[0006] The present invention provides a method to reduce complexity and improve coding efficiency through a novel transform design.

[0007] The present invention provides a method for designing a RST that can be applied to a 4x4 block.

[0008] The present invention provides a configuration of an area to which a 4x4 RST is applied, a method for arranging transform coefficients generated after applying a 4x4 RST, a scan order of the arranged transform coefficients, and a method for aligning and matching transform coefficients generated for each block.

[0009] The present invention provides a method for coding a transform index that specifies a 4x4 RST.

[0010] The present invention provides a method for determining whether a non-zero transform coefficient exists in an impermissible region when 4x4 RST is applied, and for conditionally coding the corresponding transform index.

[0011] The present invention provides a method for coding the position of the last non-zero transform coefficient, then conditionally coding its transform index, and then omitting the associated residual coding for impermissible positions.

[0012] The present invention provides a method for applying different transform index coding and residual coding to luma blocks and chroma blocks, respectively, when applying 4x4 RST. Effect of the Invention

[0013] The present invention applies 4x4 RST when encoding still images or moving images, and can significantly reduce the amount of calculations compared to when other Nsst (non-separable secondary transform) is applied.

[0014] In addition, taking into account the fact that valid transform coefficients do not exist in a particular region when 4x4 RST is applied, it is possible to improve performance by conditionally coding a transform index that specifies the 4x4 RST and applying associated residual coding optimization.

[0015] In this way, the computational complexity can be reduced through the new low-complexity computation algorithm, and coding efficiency can be improved. [Brief description of the drawings]

[0016] [Figure 1] 1 shows a schematic block diagram of an encoder for encoding a video signal as an embodiment to which the present invention is applied. [Diagram 2] 1 shows a schematic block diagram of a decoder in which a video signal is decoded as an embodiment to which the present invention is applied. [Figure 3A] 1 is a diagram for explaining a block division structure using QT (QuadTree, hereinafter referred to as "QT") as an embodiment to which the present invention can be applied. FIG. [Figure 3B] 1 is a diagram for explaining a block division structure using a Binary Tree (hereinafter referred to as "BT") as an embodiment to which the present invention can be applied. FIG. [Figure 3C] 1 is a diagram for explaining a block division structure using a TT (Ternary Tree, hereinafter referred to as "TT") as an embodiment to which the present invention can be applied. FIG. [Figure 3D] 1 is a diagram for explaining a block division structure using an Asymmetric Tree (hereinafter referred to as "AT") as an embodiment to which the present invention can be applied. [Figure 4] As an embodiment to which the present invention is applied, a schematic block diagram of a transform and quantization unit 120 / 130 and an inverse quantization and inverse transform unit 140 / 150 in an encoder is shown. [Diagram 5] As an embodiment to which the present invention is applied, a schematic block diagram of an inverse quantization and inverse transform unit 220 / 230 in a decoder is shown. [Figure 6] FIG. 1 is a diagram showing a table of transform configuration groups to which MTS (Multiple Transform Selection) is applied, as an embodiment to which the present invention is applied. [Figure 7] 2 is a flowchart illustrating an encoding process in which multiple transform selection (MTS) is performed as an embodiment to which the present invention is applied. [Figure 8] 1 is a flowchart illustrating a decoding process in which multiple transform selection (MTS) is performed as an embodiment to which the present invention is applied. [Figure 9] 1 is a flowchart illustrating a process of encoding an MTS flag and an MTS index as an embodiment to which the present invention is applied. [Figure 10] 1 is a flowchart illustrating a decoding process for applying a horizontal transform or a vertical transform to a row or a column based on an MTS flag and an MTS index, as an embodiment to which the present invention is applied. [Figure 11] As an embodiment to which the present invention is applied, a flow chart for performing an inverse transformation based on transformation-related parameters is shown. [Figure 12] 13 is a table showing allocation of a transform set for each intra prediction mode in Nsst as an embodiment to which the present invention is applied. [Figure 13] As an embodiment to which the present invention is applied, a calculation flow diagram for Givens rotation is shown. [Figure 14] As an embodiment to which the present invention is applied, a one-round configuration in 4x4 Nsst composed of a Givens rotation layer and permutation is shown. [Figure 15]FIG. 2 is a block diagram illustrating the operations of a forward reduced transform and a forward reduced transform as an embodiment to which the present invention is applied. [Figure 16] 13 is a diagram illustrating a process of performing a reverse scan from the 64th to the 17th blocks based on a reverse scan order, according to an embodiment of the present invention. [Figure 17] As an embodiment to which the present invention is applied, three forward scan orders of transform coefficient blocks (transform blocks) are shown. [Figure 18] In an embodiment to which the present invention is applied, when a diagonal scan is applied to the upper left 4x8 block and a 4x4 RST is applied, the positions of valid transform coefficients and the order of forward scan for each 4x4 block are shown. [Figure 19] As an embodiment to which the present invention is applied, a case will be described in which a diagonal scan is applied to the upper left 4x8 block, and valid transform coefficients of two 4x4 blocks are combined into one 4x4 block when a 4x4 RST is applied. [Figure 20] 1 shows a flow chart for encoding a video signal based on a reduced quadratic transform as an embodiment to which the present invention can be applied. [Figure 21] 1 shows a flow chart for decoding a video signal based on a reduced quadratic transform as an embodiment to which the present invention is applied. [Figure 22] 1 shows a structural diagram of a content streaming system as an embodiment to which the present invention is applied.

[0017] [Best Mode for Carrying Out the Invention] The present invention provides a method for restoring a video signal based on a reduced secondary transform, comprising the steps of: obtaining a secondary transform index from the video signal; deriving a secondary transform corresponding to the secondary transform index, where the secondary transform means a reduced secondary transform, and the reduced secondary transform is a transform in which N residual data (Nx1 residual vector) is input and L transform coefficient data (Lx1 transform coefficient vector) (L < N) is output; performing entropy decoding and inverse quantization on the current block (NxN) to obtain a transform coefficient block; performing an inverse secondary transform on the transform coefficient block using the reduced secondary transform; performing an inverse primary transform on the block on which the inverse secondary transform has been performed; and restoring the current block using the block on which the inverse primary transform has been performed.

[0018] In the present invention, the reduced secondary transform is applied to a specific region of the current block, and the specific region is the upper left MxM (M ≦ N) region within the current block.

[0019] In the present invention, when the inverse secondary transform is performed, a 4x4 reduced secondary transform is applied to each of the divided 4x4 blocks within the current block.

[0020] In the present invention, whether to obtain the secondary transform index is determined based on the position of the last non-zero transform coefficient within the transform coefficient block.

[0021] In the present invention, when the last non-zero transform coefficient is not located in the specific region, the secondary transform index is obtained, and the specific region indicates the remaining region excluding the positions where non-zero transform coefficients can exist when the transform coefficients are arranged in the scan order when the reduced secondary transform is applied.

[0022] In the present invention, the method includes the step of obtaining a primary conversion index of the current block from the video signal, where the primary conversion index corresponds to any one of a plurality of combinations of conversions constituted by a combination of DST7 and / or DCT8, and further includes a step of deriving a combination of conversions corresponding to the primary conversion index. The combination of conversions is constituted by a horizontal conversion and a vertical conversion, the horizontal conversion and the vertical conversion correspond to any one of the DST7 or the DCT8, and the inverse primary conversion is performed using the combination of conversions.

[0023] The present invention provides an apparatus for restoring a video signal based on a reduced secondary conversion, including a parsing unit that obtains a secondary conversion index from the video signal, a conversion unit that derives a secondary conversion corresponding to the secondary conversion index, where the secondary conversion means a reduced secondary conversion, and the reduced secondary conversion indicates a conversion in which N residual data (N×1 residual vector) is input and L conversion coefficient data (L×1 conversion coefficient vector) is output (L < N). The apparatus further includes an entropy decoding unit that performs entropy decoding on the current block (N×N), an inverse quantization unit that performs inverse quantization on the current block on which the entropy decoding has been performed to obtain a conversion coefficient block, the conversion unit that performs an inverse secondary conversion on the conversion coefficient block using the reduced secondary conversion and performs an inverse primary conversion on the block on which the inverse secondary conversion has been performed, and a restoration unit that restores the current block using the block on which the inverse primary conversion has been performed.

Embodiments for Carrying Out the Invention

[0024] Hereinafter, the configuration and operation of embodiments of the present invention will be described with reference to the accompanying drawings. The configuration and operation of the present invention described with reference to the drawings are described as one embodiment, and thereby the technical idea, core configuration, and operation of the present invention are not limited.

[0025] In addition, the terms used in the present invention are generally used as widely as possible, but in specific cases, terms arbitrarily selected by the applicant are used for explanation. In such cases, the meanings are clearly described in the detailed description of the relevant part, so it is clarified that the terms used in the description of the present invention should not be analyzed simply based on the names of the terms, but should be analyzed by understanding the meanings of the relevant terms.

[0026] In addition, the terms used in the present invention are general terms selected to describe the invention, but if there are other terms with similar meanings, they can be substituted for more appropriate analysis. For example, in the case of signal, data, sample, picture, frame, block, etc., they can be appropriately substituted and analyzed in each coding process. In addition, in the case of partitioning, decomposition, splitting, division, etc., they can be appropriately substituted and analyzed in each coding process.

[0027] In this document, MTS (Multiple Transform Selection, hereinafter referred to as "MTS") may refer to a method of performing a transform using at least two or more transform types. This may also be expressed as AMT (Adaptive Multiple Transform) or EMT (Explicit Multiple Transform), and similarly, mts_idx may also be expressed as AMT_idx, EMT_idx, tu_mts_idx, AMT_TU_idx, EMT_TU_idx, a transform index, or a transform combination index, and the present invention is not limited to such expressions.

[0028] FIG. 1 is a schematic block diagram of an encoder for encoding a video signal as an embodiment to which the present invention is applied.

[0029] As shown in FIG. 1, the encoder 100 includes an image division unit 110, a transformation unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transformation unit 150, a filtering unit 160, a decoded picture buffer (DPB) 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190.

[0030] The image division unit 110 divides an input image (or a picture or a frame) input to the encoder 100 into one or more processing units. For example, the processing units may be a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0031] However, the above terms are used merely for the convenience of explanation of the present invention, and the present invention is not limited to the definition of the terms. In addition, for the convenience of explanation, the term coding unit is used in this specification as a unit used in the process of encoding or decoding a video signal, but the present invention is not limited thereto, and can be appropriately analyzed according to the contents of the invention.

[0032] The encoder 100 subtracts a prediction signal output from the inter prediction unit 180 or the intra prediction unit 185 from the input image signal to generate a residual signal, and the generated residual signal is transmitted to the conversion unit 120.

[0033] The transform unit 120 applies a transform technique to the residual signal to generate transform coefficients. The transform process can be applied to a square block of a quadtree structure, a block (square or rectangular) divided by a binary tree structure, a ternary tree structure, or an asymmetric tree structure.

[0034] The transform unit 120 may perform a transform based on a plurality of transforms (or a combination of transforms), and such a transform method may be called Multiple Transform Selection (MTS). The MTS may also be called Adaptive Multiple Transform (AMT) or Enhanced Multiple Transform (EMT).

[0035] The above (MTS, AMT, EMT) may refer to a transformation method that is performed based on a transformation (or a combination of transformations) adaptively selected from a plurality of transformations (or a combination of transformations).

[0036] The plurality of transforms (or combinations of transforms) may include the transforms (or combinations of transforms) described herein in Figure 6. In this specification, the transforms or transform types may be expressed as, for example, DCT-Type 2, DCT-II, DCT2, DCT2, etc.

[0037] The converter 120 may implement the following embodiments.

[0038] The present invention provides a method for designing a RST that can be applied to a 4x4 block.

[0039] The present invention provides a configuration of an area to which a 4x4 RST is applied, a method for arranging transform coefficients generated after applying a 4x4 RST, a scan order of the arranged transform coefficients, and a method for aligning and matching transform coefficients generated for each block.

[0040] The present invention provides a method for coding a transform index that specifies a 4x4 RST.

[0041] The present invention provides a method for determining whether a non-zero transform coefficient exists in an area that is not allowed when 4x4 RST is applied, and for conditionally coding the corresponding transform index.

[0042] The present invention provides a method for coding the position of the last non-zero transform coefficient, then conditionally coding its transform index, and then omitting the associated residual coding for impermissible positions.

[0043] The present invention provides a method for applying different transform index coding and residual coding to luma blocks and chroma blocks, respectively, when applying 4x4 RST.

[0044] Specific embodiments for this are described in more detail herein.

[0045] The quantization unit 130 quantizes the transform coefficients and transmits the quantized transform coefficients to the entropy encoding unit 190, and the entropy encoding unit 190 may entropy code the quantized signal and output it as a bitstream.

[0046] Although the transform unit 120 and the quantization unit 130 are described as separate functional units, the present invention is not limited thereto and may be combined into one functional unit. The inverse quantization unit 140 and the inverse transform unit 150 may also be combined into one functional unit.

[0047] The quantized signal output from the quantizer 130 can be used to generate a prediction signal. For example, the quantized signal can be restored to a residual signal by applying inverse quantization and inverse transformation via the inverse quantizer 140 and the inverse transformer 150 in a loop. A reconstructed signal can be generated by adding the restored residual signal to a prediction signal output from the inter prediction unit 180 or the intra prediction unit 185.

[0048] Meanwhile, quantization errors occurring during the compression process may cause block boundaries to become visible. This phenomenon is called blocking artifacts, and is one of the important factors for evaluating image quality. A filtering process can be performed to reduce this artifact. Through this filtering process, blocking artifacts can be removed and errors in the current picture can be reduced, thereby improving image quality.

[0049] The filtering unit 160 applies filtering to the reconstructed signal, and outputs the signal to a playback device or transmits the signal to a decoded picture buffer 170. The filtered signal transmitted to the decoded picture buffer 170 can be used as a reference picture in the inter prediction unit 180. In this way, by using the filtered picture as a reference picture in the inter prediction mode, not only the image quality but also the coding efficiency can be improved.

[0050] The decoded picture buffer 170 may store the filtered pictures for use as reference pictures from the inter predictor 180.

[0051] The inter prediction unit 180 performs temporal prediction and / or spatial prediction to remove temporal redundancy and / or spatial redundancy by referring to a reconstructed picture. Here, the reference picture used to perform the prediction is a transformed signal that has undergone quantization and inverse quantization in block units during previous encoding / decoding, so blocking artifacts and ringing artifacts may exist.

[0052] Therefore, in order to solve the performance degradation due to the signal discontinuity and quantization, the inter prediction unit 180 may apply a low pass filter to interpolate signals between pixels in sub-pixel units. Here, a sub-pixel refers to a virtual pixel generated by applying an interpolation filter, and an integer pixel refers to an actual pixel present in a restored picture. As an interpolation method, linear interpolation, bi-linear interpolation, a Wiener filter, etc. may be applied.

[0053] The interpolation filter may be applied to a reconstructed picture to improve prediction accuracy. For example, the inter prediction unit 180 may apply an interpolation filter to integer pixels to generate interpolated pixels, and perform prediction using an interpolated block composed of the interpolated pixels as a prediction block.

[0054] Meanwhile, the intra prediction unit 185 may predict a current block by referring to samples in the vicinity of a block to be currently encoded. The intra prediction unit 185 may perform the following process to perform intra prediction. First, reference samples required for generating a prediction signal may be prepared. Then, a prediction signal may be generated using the prepared samples. Then, a prediction mode is encoded. In this case, the reference samples may be prepared through reference sample padding and / or reference sample filtering. Since the reference samples have undergone prediction and reconstruction processes, quantization errors may exist. Therefore, in order to reduce such errors, a reference sample filtering process may be performed for each prediction mode used in intra prediction.

[0055] A prediction signal generated via the inter prediction unit 180 or the intra prediction unit 185 can be used to generate a reconstructed signal or can be used to generate a residual signal.

[0056] FIG. 2 is a schematic block diagram of a decoder according to an embodiment of the present invention, in which a video signal is decoded.

[0057] Referring to FIG. 2, the decoder 200 may include an analysis unit (not shown), an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, a filtering unit 240, a decoded picture buffer (DPB) 250, an inter prediction unit 260, and an intra prediction unit 265.

[0058] The reconstructed video signal outputted through the decoder 200 can be reproduced through a reproduction device.

[0059] The decoder 200 can receive a signal output from the encoder 100 of FIG. 1, and the received signal can be entropy decoded via an entropy decoding unit 210 .

[0060] The inverse quantization unit 220 obtains transform coefficients from the entropy decoded signal using the quantization step size information.

[0061] The inverse transform unit 230 inversely transforms the transform coefficients to obtain a residual signal.

[0062] Here, the present invention provides a method for configuring a transform combination for each transform configuration group classified by at least one of a prediction mode, a block size, or a block shape, and the inverse transform unit 230 can perform an inverse transform based on the transform combination configured according to the present invention. Also, the embodiments described in this specification can be applied.

[0063] The inverse transform unit 230 can execute the following embodiments.

[0064] The present invention provides a method for restoring a video signal based on a reduced secondary transform.

[0065] The inverse transform unit 230 can induce a secondary transform corresponding to a secondary transform index, perform an inverse secondary transform on a transform coefficient block using the secondary transform, and perform an inverse primary transform on the block on which the inverse secondary transform has been performed. Here, the secondary transform means a reduced secondary transform, and the reduced secondary transform indicates a transform in which N residual data (Nx1 residual vector) is input and L transform coefficient data (Lx1 transform coefficient vector) is output (L < N).

[0066] In the present invention, the reduced secondary transform is applied to a specific region of the current block, and the specific region is the upper left MxM (M ≦ N) region within the current block.

[0067] In the present invention, when the inverse secondary transform is executed, a 4x4 reduced secondary transform is applied to each of the divided 4x4 blocks within the current block.

[0068] In the present invention, whether or not to obtain the secondary transform index is determined based on the position of the last non-zero transform coefficient in the transform coefficient block.

[0069] In the present invention, the secondary transform index is obtained when the last non-zero transform coefficient is not located in a specific area, and the specific area indicates a remaining area excluding a position where a non-zero transform coefficient can exist when the transform coefficients are arranged according to a scan order when the reduced secondary transform is applied.

[0070] The inverse transform unit 230 may derive a transform combination corresponding to a primary transform index, and perform an inverse primary transform using the transform combination. Here, the primary transform index corresponds to any one of a plurality of transform combinations constituted by a combination of DST7 and / or DCT8, and the transform combination is constituted by a horizontal transform and a vertical transform. At this time, the horizontal transform and the vertical transform correspond to any one of the DST7 or the DCT8.

[0071] Although the inverse quantization unit 220 and the inverse transform unit 230 are described as separate functional units, the present invention is not limited thereto and may be combined into one functional unit.

[0072] A reconstructed signal is generated by adding the obtained residual signal to a prediction signal output from the inter prediction unit 260 or the intra prediction unit 265.

[0073] The filtering unit 240 applies filtering to the reconstructed signal and outputs the signal to a playback device or transmits the signal to the decoded picture buffer unit 250. The filtered signal transmitted to the decoded picture buffer unit 250 can be used as a reference picture in the inter prediction unit 260.

[0074] In this specification, the embodiments described for the transform unit 120 and each functional unit of the encoder 100 may be similarly applied to the inverse transform unit 230 and corresponding functional units of the decoder.

[0075] FIG. 3 is a diagram for explaining block division structures according to an embodiment to which the present invention can be applied, in which FIG. 3A is a QT (QuadTree, hereinafter referred to as "QT"), FIG. 3B is a BT (Binary Tree, hereinafter referred to as "BT"), FIG. 3C is a TT (Ternary Tree, hereinafter referred to as "TT"), and FIG. 3D is an AT (Asymmetric Tree, hereinafter referred to as "AT").

[0076] In video coding, a block can be divided on a quadtree (QT) basis. A subblock divided by QT can be further divided recursively using QT. A leaf block that is not further divided by QT can be divided by at least one of binary tree (BT), ternary tree (TT) or asymmetric tree (AT). BT has two types of division, horizontal BT (2N×N, 2N×N) and vertical BT (N×2N, N×2N). TT has two types of division, horizontal TT (2N×1 / 2N, 2N×N, 2N×1 / 2N) and vertical TT (1 / 2N×2N, N×2N, 1 / 2N×2N). AT has four forms of partitioning: horizontal-up AT (2Nx1 / 2N, 2Nx3 / 2N), horizontal-down AT (2Nx3 / 2N, 2Nx1 / 2N), vertical-left AT (1 / 2Nx2N, 3 / 2Nx2N), vertical-right AT (3 / 2Nx2N, 1 / 2Nx2N). Each BT, TT, and AT can be further partitioned recursively using BT, TT, and AT.

[0077] 3A shows an example of QT division. Block A is divided into four sub-blocks (A0, A1, A2, A3) by QT. Sub-block A1 is further divided into four sub-blocks (B0, B1, B2, B3) by QT.

[0078] 3B shows an example of BT division. Block B3, which is not further divided by QT, is divided by vertical BT (C0, C1) or horizontal BT (D0, D1). Each subblock, such as block C0, can be further divided recursively, such as in the form of horizontal BT (E0, E1) or vertical BT (F0, F1).

[0079] 3C shows an example of TT division. Block B3, which is not further divided by QT, is divided by vertical TT (C0, C1, C2) or horizontal TT (D0, D1, D2). Each sub-block, such as block C1, can be further divided recursively, such as in the form of horizontal TT (E0, E1, E2) or vertical TT (F0, F1, F2).

[0080] 3D shows an example of AT division. Block B3, which is not further divided by QT, is divided by vertical AT(C0, C1) or horizontal AT(D0, D1). Each subblock, such as block C1, can be further divided recursively, such as in the form of horizontal AT(E0, E1) or vertical TT(F0, F1).

[0081] Meanwhile, BT, TT, and AT division can be used together for division. For example, subblocks divided by BT can be divided by TT or AT. Subblocks divided by TT can be divided by BT or AT. Subblocks divided by AT can be divided by BT or TT. For example, after horizontal BT division, each subblock can be divided by vertical BT, or after vertical BT division, each subblock can be divided by horizontal BT. The above two division methods have different division orders, but the final divided shapes are the same.

[0082] In addition, when a block is divided, the order of searching the block can be defined in various ways. In general, searching a block from left to right and from top to bottom means an order of determining whether to perform additional block division of each divided sub-block, means a coding order of each sub-block when the block is not further divided, or means a search order when referring to information of other neighboring blocks in a sub-block.

[0083] 4 and 5 show an embodiment to which the present invention is applied. FIG. 4 shows a schematic block diagram of a transform and quantization unit 120 / 130 and an inverse quantization and inverse transform unit 140 / 150 in an encoder, and FIG. 5 shows a schematic block diagram of an inverse quantization and inverse transform unit 220 / 230 in a decoder.

[0084] 4, the transform and quantization unit 120 / 130 includes a primary transform unit 121, a secondary transform unit 122, and a quantization unit 130. The inverse quantization and inverse transform unit 140 / 150 includes an inverse quantization unit 140, an inverse secondary transform unit 151, and an inverse primary transform unit 152.

[0085] As shown in FIG. 5, the inverse quantization and inverse transform unit 220 / 230 includes an inverse quantization unit 220, an inverse secondary transform unit 231 and an inverse primary transform unit 232.

[0086] In the present invention, the transformation can be performed through multiple stages. For example, as shown in Fig. 4, two stages of a primary transform and a secondary transform can be applied, or more transformation stages can be used depending on the algorithm. Here, the primary transform can be called a core transform.

[0087] The primary transform unit 121 applies a primary transform to the residual signal, where the primary transform may already be defined as a table in the encoder and / or decoder.

[0088] For the primary transform, Discrete Cosine Transform type 2 (hereinafter, referred to as DCT2) may be applied. Alternatively, Discrete Sine Transform type 7 (hereinafter, referred to as DST7) may be applied in a specific case. For example, DST7 may be applied to a 4×4 block in an intra prediction mode.

[0089] Also, in the case of the primary transform, a combination of multiple transforms (DST 7, DCT 8, DST 1, DCT 5) of Multiple Transform Selection (MTS) can be applied. For example, FIG. 6 can be applied.

[0090] The secondary transform unit 122 may apply a secondary transform to the primary transformed signal, where the secondary transform may be predefined in a table from the encoder and / or decoder.

[0091] In one embodiment, the secondary transform may be a Non-Separable Secondary Transform (hereinafter, referred to as "Nsst") that may be conditionally applied. For example, the Nsst may be applied only to an intra-prediction block, and may have a set of applicable transforms for each prediction mode group.

[0092] Here, the prediction mode group may be set based on the symmetry of the prediction direction. For example, since the prediction mode 52 and the prediction mode 16 are symmetric based on the prediction mode 34 (diagonal direction), they form one group and the same transform set may be applied. In this case, when applying the transform of the prediction mode 52, the input data is transposed before being applied, because the transform set is the same as that of the prediction mode 16.

[0093] On the other hand, Planar mode and DC mode have their own transformation sets, each consisting of two transformations, since there is no directional symmetry, while the remaining directional modes have three transformations per transformation set.

[0094] In another embodiment, for the secondary transform, a combination of various transforms (DST 7, DCT 8, DST 1, DCT 5) of Multiple Transform Selection (MTS) may be applied. For example, Figure 6 can be applied. In another embodiment, DST7 can be applied as a secondary transform.

[0095] In another embodiment, the Nsst is not applied to the entire block that has been subjected to the primary transformation, but may be applied only to the top-left 8x8 region. For example, if the size of the block is 8x8 or more, 8x8 Nsst is applied, and if the size is less than 8x8, 4x4 Nsst is applied. In this case, the block is divided into 4x4 blocks, and then 4x4 Nsst is applied to each of them.

[0096] As another embodiment, 4x4 Nsst can be applied even when 4xN / Nx4 (N>=16).

[0097] The Nsst, 4x4 Nsst and 8x8 Nsst will be described in more detail below with reference to FIGS. 12 to 15 and other embodiments of the specification.

[0098] The quantization unit 130 may quantize the secondary transformed signal. The inverse quantization and inverse transformation unit (140 / 150) performs the above-described processes inversely, and therefore a duplicated description will be omitted.

[0099] FIG. 5 shows a schematic block diagram of the inverse quantization and inverse transform unit (220 / 230) within the decoder.

[0100] 5, the inverse quantization and inverse transform unit (220 / 230) may include an inverse quantization unit 220, an inverse secondary transform unit 231, and an inverse primary transform unit 232.

[0101] The inverse quantization unit 220 obtains transform coefficients from the entropy decoded signal using quantization step size information.

[0102] The inverse secondary transform unit 231 performs an inverse secondary transform on the transform coefficients. Here, the inverse secondary transform refers to an inverse transform of the secondary transform described with reference to FIG.

[0103] In another embodiment, in the case of the secondary transformation, a combination of various transformations (DST7, DCT8, DST1, DCT5) of Multiple Transform Selection (MTS) can be applied. For example, FIG. 6 can be applied.

[0104] The inverse primary transform unit 232 performs an inverse primary transform on the inverse secondary transformed signal (or block) to obtain a residual signal. Here, the inverse primary transform refers to the inverse transform of the primary transform described in FIG. 4.

[0105] In one embodiment, for the primary transform, a combination of various transforms (DST 7, DCT 8, DST 1, DCT 5) of Multiple Transform Selection (MTS) may be applied, for example, as shown in FIG.

[0106] In one embodiment of the present invention, DST7 may be applied as a primary transform.

[0107] In one embodiment of the present invention, a DCT8 may be applied as the primary transform.

[0108] The present invention provides a method for configuring a transform combination for each transform configuration group classified according to at least one of a prediction mode, a block size, and a block shape, and the inverse primary transform unit 232 may perform an inverse transform based on the transform combination configured according to the present invention. Also, the embodiments described in this specification may be applied.

[0109] FIG. 6 is a table showing transform configuration groups to which MTS (Multiple Transform Selection) is applied as an embodiment to which the present invention is applied.

[0110] Transformation settings group to which MTS (Multiple Transform Selection) is applied

[0111] In this specification, a candidate for the j-th transformation combination of the transformation setting group Gi is expressed as a pair as shown in the following formula (1).

[0112] [Number 1] (H(Gi, j), V(Gi, j))

[0113] Here, H(Gi,j) refers to the horizontal transform of the j-th candidate, and V(Gi,j) refers to the vertical transform of the j-th candidate. For example, in FIG. 6, it can be expressed as H(G3,2)=DST7, V(G3,2)=DCT8. Depending on the context, the value assigned to H(Gi,j) or V(Gi,j) may be a nominal value for distinguishing the transform as in the above example, an index value for the transform, or a 2D matrix of the transform.

[0114] In addition, in this specification, the 2D matrix values ​​of DCT and DST can be expressed as in the following Equations 2 and 3.

[0115] [Number 2] DCT type 2:, DCT type 8:

[0116] [Number 3] DST type 7:, DST type 4:

[0117] Here, DST or DCT is indicated by S or C, the type number is written in Roman numerals as a superscript, and the subscript N indicates that it is an NxN transform. Also, for 2D matrices such as the above, we assume that the column vector forms the transform basis.

[0118] 6, a transform configuration group is determined based on a prediction mode, and there can be a total of six groups (G0 to G5). G0 to G4 correspond to cases where intra prediction is applied, and G5 indicates a combination (or a transform set, or a transform combination set) of transforms applied to a residual block generated by inter prediction.

[0119] A combination of transformations can consist of a horizontal transform (or row transform) applied to the rows of the 2D block and a vertical transform (or column transform) applied to the columns.

[0120] Here, each transform setting group may have four transform combination candidates. The four transform combination candidates may be selected or determined via transform combination indexes of 0 to 3, and the transform combination indexes may be encoded and transmitted from the encoder to the decoder.

[0121] In one embodiment, residual data (or residual signals) obtained through intra prediction may have different statistical characteristics depending on the intra prediction mode, and therefore, as shown in FIG 6, a different transform other than the general cosine transform may be applied for each intra prediction mode.

[0122] Looking carefully at FIG. 6, there are shown a case where 35 intra prediction modes are used and a case where 67 intra prediction modes are used. A plurality of transform combinations can be applied to each transform setting group in each intra prediction mode column. For example, the plurality of transform combinations can be composed of four (row-direction transform, column-direction transform) combinations. As a specific example, in group 0, DST-7 and DCT-5 can be applied to both the row (horizontal) direction and the column (vertical) direction, making a total of four combinations possible.

[0123] Since a total of four transform kernel combinations can be applied to each intra prediction mode, a transform combination index for selecting one of them can be transmitted for each transform unit. In this specification, the transform combination index can be referred to as an MTS index and can be represented by mts_idx.

[0124] In addition to the transform kernels shown in FIG. 6, there may be cases where DCT2 is optimal for all row and column directions due to the characteristics of a residual signal. Therefore, by defining an MTS flag for each coding unit, transforms can be performed adaptively. Here, when the MTS flag is 0, DCT2 is applied to all row and column directions, and when the MTS flag is 1, one of the four combinations can be selected or determined via the MTS index.

[0125] In one embodiment, when the MTS flag is 1, if the number of non-zero transform coefficients for one transform unit is not greater than a threshold, DST-7 can be applied in all row and column directions without applying the transform kernel of Fig. 6. For example, the threshold can be set to 2, which can be set differently based on the block size or transform unit size. This is also applicable to other embodiments of the specification.

[0126] In one embodiment, the values ​​of the transform coefficients are first analyzed, and if the number of non-zero transform coefficients is not greater than a threshold, the amount of additional information transmitted can be reduced by applying DST-7 without analyzing the MTS index.

[0127] In one embodiment, when the MTS flag is 1, if the number of non-zero transform coefficients for a transform unit is greater than a threshold, the MTS index can be analyzed, and the horizontal transform and vertical transform can be determined based on the MTS index.

[0128] In one embodiment, MTS can be applied only if both the width and height of the transform unit are less than or equal to 32.

[0129] In one embodiment, FIG. 6 can be preset through off-line training.

[0130] In one embodiment, the MTS index can be defined as one index that can simultaneously indicate a combination of horizontal and vertical transforms, or the MTS index can define a horizontal transform index and a vertical transform index separately.

[0131] In one embodiment, the MTS flag or the MTS index may be defined at at least one level of a sequence, a picture, a slice, a block, a coding unit, a transform unit, or a prediction unit. For example, the MTS flag or the MTS index may be defined at at least one level of a sequence parameter set (sps) or a transform unit.

[0132] In another embodiment, the transform combination (horizontal transform, vertical transform) corresponding to the transform index can be configured independent of the MTS flag, the prediction mode, and / or the block shape. For example, the transform combination can be at least one of DCT2, DST7, and / or DCT8. As a specific example, if the transform index is 0, 1, 2, 3, or 4, the transform combination can be (DCT2, DCT2), (DST7, DST7), (DCT8, DST7), (DST7, DCT8), or (DCT8, DCT8), respectively.

[0133] FIG. 7 is a flowchart showing an encoding process in which MTS (Multiple Transform Selection) is performed as an embodiment to which the present invention is applied.

[0134] Although the present specification describes an embodiment in which transformations are applied separately for the horizontal and vertical directions, the combination of transformations can also be configured as non-separable transforms.

[0135] Alternatively, it can be configured with a mixture of separable and non-separable transforms. In this case, when a non-separable transform is used, it is not necessary to select a transform by row / column or by horizontal / vertical direction, and the combination of transforms in Figure 6 is used only when a separable transform is selected.

[0136] In addition, the method proposed in this specification can be applied regardless of a primary transform or a secondary transform. That is, there is no restriction that it must be applied to only one of the two, and both can be applied. Here, the primary transform may refer to a transform for initially transforming a residual block, and the secondary transform may refer to a transform for applying a transform to a block generated as a result of the primary transform.

[0137] First, the encoder may determine a transform setting group corresponding to a current block (S710). Here, the transform setting group may refer to the transform setting group of FIG. 6, but the present invention is not limited thereto and may be configured with other transform combinations.

[0138] The encoder may perform a transform for a combination of available candidate transforms within the group of transforms (S720).

[0139] As a result of performing the transforms, the encoder may determine or select a combination of transforms that has a minimum rate distortion (RD) cost (S730).

[0140] The encoder may encode an index of a transform combination corresponding to the selected transform combination (S740).

[0141] FIG. 8 is a flowchart showing a decoding process performed by MTS (Multiple Transform Selection) as an embodiment to which the present invention is applied.

[0142] First, the decoder may determine a transform setting group for a current block (S810).

[0143] The decoder may analyze (or obtain) a transform combination index from the video signal, where the transform combination index may correspond to any one of a plurality of transform combinations in the transform setting group (S820). For example, the transform setting group may include DST7 (Discrete Sine Transform type 7) and DCT8 (Discrete Cosine Transform type 8). The transform combination index may be referred to as an MTS index.

[0144] In one embodiment, the transform setting group may be set based on at least one of a prediction mode, a block size, or a block shape of the current block.

[0145] The decoder may derive a transform combination corresponding to the transform combination index (S830), where the transform combination is composed of a horizontal transform and a vertical transform and may include at least one of the DST-7 or DCT-8.

[0146] In addition, the combination of transformations may refer to the combination of transformations described in Fig. 6, but the present invention is not limited thereto, and may be configured with other combinations of transformations according to other embodiments of the present specification.

[0147] The decoder may perform an inverse transform of the current block based on the combination of transforms (S840). If the combination of transforms is composed of a row (horizontal) transform and a column (vertical) transform, the row (horizontal) transform may be applied first, and then the column (vertical) transform may be applied. However, the present invention is not limited thereto, and if the combination of transforms is composed of a row (horizontal) transform and a column (vertical) transform, the non-separable transform may be applied immediately.

[0148] In one embodiment, when the vertical transform or the horizontal transform is the DST-7 or DCT-8, the inverse transform of the DST-7 or the inverse transform of the DCT-8 may be applied column by column and then applied row by row.

[0149] In one embodiment, the vertical transformation or the horizontal transformation may be applied separately for each row and / or each column.

[0150] In one embodiment, the index of the transformation combination may be obtained based on an MTS flag indicating whether MTS is performed, i.e., the index of the transformation combination may be obtained if MTS is performed based on the MTS flag.

[0151] In one embodiment, the decoder may check whether the number of non-zero transform coefficients is greater than a threshold, and the index of the transform combination may be obtained if the number of non-zero transform coefficients is greater than a threshold.

[0152] In one embodiment, the MTS flag or the MTS index may be defined at at least one level of a sequence, a picture, a slice, a block, a coding unit, a transform unit, or a prediction unit.

[0153] In one embodiment, the inverse transform can be applied only if the width and height of the transform unit are both less than or equal to 32.

[0154] Meanwhile, in another embodiment, the process of determining the transform setting group and the process of analyzing the index of the transform combination may be performed simultaneously, or the step S810 may be preset in the encoder and / or decoder and may be omitted.

[0155] FIG. 9 is a flowchart illustrating a process of encoding an MTS flag and an MTS index according to an embodiment of the present invention.

[0156] The encoder may determine whether a multiple transform selection (MTS) is applied to the current block (S910).

[0157] If multiple transform selection (MTS) is applied, the encoder may encode with MTS flag=1 (S920).

[0158] The encoder may determine an MTS index based on at least one of a prediction mode, a horizontal transform, and a vertical transform of the current block (S930). Here, the MTS index refers to an index indicating one of a plurality of transform combinations for each intra prediction mode, and the MTS index may be transmitted for each transform unit.

[0159] Once the MTS index is determined, the encoder may encode the MTS index (S940).

[0160] On the other hand, if the (MTS Multiple Transform Selection) is not applied, the encoder may encode the MTS flag=0 (S950).

[0161] FIG. 10 is a flowchart illustrating a decoding process for applying a horizontal transform or a vertical transform to a row or a column based on an MTS flag and an MTS index, as an embodiment to which the present invention is applied.

[0162] The decoder may parse an MTS flag from the bitstream (S1010), where the MTS flag may indicate whether a multiple transform selection (MTS) is applied to the current block.

[0163] The decoder may determine whether a multiple transform selection (MTS) of the current block is applied based on the MTS flag (S1020). For example, the decoder may determine whether the MTS flag is 1.

[0164] If the MTS flag is 1, the decoder may check whether the number of non-zero transform coefficients is greater than (or equal to) a threshold (S1030). For example, the threshold may be set to 2, which may be set differently based on the block size or transform unit size.

[0165] If the number of non-zero transform coefficients is greater than a threshold, the decoder may analyze an MTS index (S1040). Here, the MTS index refers to an index indicating one of a plurality of transform combinations for each intra prediction mode or inter prediction mode, and the MTS index may be transmitted for each transform unit. Alternatively, the MTS index may refer to an index indicating one of transform combinations defined in a pre-configured transform combination table, and the pre-configured transform combination table may refer to FIG. 6, but the present invention is not limited thereto.

[0166] The decoder may derive or determine horizontal and vertical transforms based on at least one of the MTS index or the prediction mode (S1050).

[0167] Alternatively, the decoder may derive a combination of transforms corresponding to the MTS index. For example, the decoder may derive or determine a horizontal transform and a vertical transform corresponding to the MTS index.

[0168] On the other hand, if the number of non-zero transform coefficients is not greater than the threshold, the decoder may apply a preset vertical inverse transform to each column (S1060). For example, the vertical inverse transform may be an inverse transform of DST7.

[0169] Then, the decoder may apply a preset horizontal inverse transform to each row (S1070). For example, the horizontal inverse transform may be an inverse transform of DST7. That is, if the number of non-zero transform coefficients is not greater than a threshold, a transform kernel preset in the encoder or decoder may be used. For example, a commonly used transform kernel may be used instead of one defined in the transform combination table of FIG. 6.

[0170] On the other hand, if the MTS flag is 0, the decoder may apply a preset vertical inverse transform to each column (S1080). For example, the vertical inverse transform may be an inverse transform of DCT2.

[0171] Then, the decoder may apply a preset horizontal inverse transform to each row (S1090). For example, the horizontal inverse transform may be an inverse transform of DCT2. That is, if the MTS flag is 0, a transform kernel preset in the encoder or decoder may be used. For example, a commonly used transform kernel may be used instead of one defined in the transform combination table shown in FIG. 6.

[0172] FIG. 11 shows a flowchart of an embodiment of the present invention for performing an inverse transformation based on transformation-related parameters.

[0173] A decoder to which the present invention is applied can acquire sps_mts_intra_enabled_flag or sps_mts_inter_enabled_flag (S1110). Here, sps_mts_intra_enabled_flag indicates whether tu_mts_flag exists in the residual coding syntax of an intra coding unit. For example, if sps_mts_intra_enabled_flag=0, tu_mts_flag does not exist in the residual coding syntax of an intra coding unit, and if sps_mts_intra_enabled_flag=0, tu_mts_flag exists in the residual coding syntax of an intra coding unit. And sps_mts_inter_enabled_flag indicates whether tu_mts_flag exists in the residual coding syntax of an inter coding unit. For example, if sps_mts_inter_enabled_flag=0, then tu_mts_flag is not present in the residual coding syntax of the inter coding unit, and if sps_mts_inter_enabled_flag=0, then tu_mts_flag is present in the residual coding syntax of the inter coding unit.

[0174] The decoder may acquire tu_mts_flag based on sps_mts_intra_enabled_flag or sps_mts_inter_enabled_flag (S1120). For example, when sps_mts_intra_enabled_flag=1 or sps_mts_inter_enabled_flag=1, the decoder may acquire tu_mts_flag. Here, tu_mts_flag indicates whether multiple transform selection (hereinafter referred to as "MTS") is applied to the residual samples of the luma transform block. For example, when tu_mts_flag=0, MTS is not applied to the residual samples of the luma transform block, and when tu_mts_flag=1, MTS is applied to the residual samples of the luma transform block.

[0175] As another example, at least one of the embodiments of this document may be applied to the tu_mts_flag.

[0176] The decoder may obtain mts_idx based on tu_mts_flag (S1130). For example, when tu_mts_flag=1, the decoder may obtain mts_idx, where mts_idx indicates which transform kernel is applied to luma residual samples along the horizontal and / or vertical directions of the current transform block.

[0177] For example, for mts_idx, at least one of the embodiments in this document can be applied. As a specific example, at least one of the embodiments in FIG. 6 can be applied.

[0178] The decoder may derive a transform kernel corresponding to mts_idx (S1140). For example, the transform kernel corresponding to mts_idx may be defined as being divided into a horizontal transform and a vertical transform.

[0179] As another example, different transformation kernels may be applied to the horizontal transformation and the vertical transformation, although the present invention is not limited thereto, and the same transformation kernel may be applied to the horizontal transformation and the vertical transformation.

[0180] In one embodiment, mts_idx may be defined as shown in Table 1 below.

[0181] [Table 1]

[0182] The decoder can then perform an inverse transform based on the transform kernel (S1150).

[0183] A decoding process for performing a conversion process according to another embodiment of the present invention will now be described.

[0184] The decoder may ascertain a transform size (nTbS) (S10), where the transform size (nTbS) may be a variable indicating a horizontal sample size of the scaled transform coefficients.

[0185] The decoder may check a transform kernel type (trType) (S20). Here, the transform kernel type (trType) may be a variable indicating a transform kernel type, and various embodiments of this document may be applied. The transform kernel type (trType) may include a horizontal transform kernel type (trTypeHor) and a vertical transform kernel type (trTypeVer).

[0186] Referring to Table 1, the transform kernel type (trType) may indicate DCT2 when it is 0, DCT7 when it is 1, and DCT8 when it is 2.

[0187] The decoder may perform transform matrix multiplication based on at least one of a transform size (nTbS) or a transform kernel type (S30).

[0188] As another example, if the transformation kernel type is 1 and the transformation size is 4, a predetermined transformation matrix (1) may be applied when performing the transformation matrix multiplication.

[0189] As another example, if the transformation kernel type is 1 and the transformation size is 8, a transformation matrix (2) determined in advance may be applied when performing the transformation matrix multiplication.

[0190] As another example, if the transformation kernel type is 1 and the transformation size is 16, the previously determined transformation matrix (3) may be applied when performing the transformation matrix multiplication.

[0191] As another example, if the transformation kernel type is 1 and the transformation size is 32, the predefined transformation matrix (4) can be applied.

[0192] Similarly, if the transformation kernel type is 2 and the transformation size is 4, 8, 16, or 32, predefined transformation matrices (5), (6), (7), and (8) can be applied, respectively.

[0193] Here, each of the predefined transformation matrices (1) to (8) may correspond to one of various types of transformation matrices. For example, the transformation matrices of the type illustrated in FIG. 6 may be applied.

[0194] The decoder can derive transformed samples based on multiplication of a transform matrix (S40).

[0195] The above embodiments may be used individually, but the invention is not limited thereto and may be used in combination with the above embodiments and other embodiments herein.

[0196] FIG. 12 is a table showing allocation of a transform set for each intra prediction mode in NSST as an embodiment to which the present invention is applied.

[0197] Non-Separable Secondary Transform(NSST) A secondary transform unit may apply a secondary transform to the primary transformed signal, where the secondary transform may be predefined in a table in the encoder and / or decoder.

[0198] In one embodiment, the secondary transform may be a Non-Separable Secondary Transform (hereinafter, referred to as "Nsst") that may be conditionally applied. For example, the Nsst may be applied only to an intra-prediction block, and may have a set of applicable transforms for each prediction mode group.

[0199] Here, the prediction mode group may be set based on the symmetry of the prediction direction. For example, since the prediction mode 52 and the prediction mode 16 are symmetric based on the prediction mode 34 (diagonal direction), they form one group and the same transform set may be applied. In this case, when applying the transform of the prediction mode 52, the input data is transposed before being applied, because the transform set is the same as that of the prediction mode 16.

[0200] Meanwhile, in the case of the planar mode and the DC mode, since there is no directional symmetry, each of them has a transformation set, and the transformation set may be composed of two transformations. For the remaining directional modes, each transformation set may be composed of three transformations. However, the present invention is not limited thereto, and each transformation set may be composed of multiple transformations.

[0201] FIG. 13 shows a calculation flow diagram for the Givens rotation as an embodiment to which the present invention is applied.

[0202] In another embodiment, the Nsst is not applied to the entire block that has been subjected to the primary transformation, but is applied only to the top-left 8x8 region. For example, if the size of the block is 8x8 or more, 8x8 Nsst is applied, and if the size is less than 8x8, 4x4 Nsst is applied. In this case, the block is divided into 4x4 blocks, and then 4x4 Nsst is applied to each of them.

[0203] As another embodiment, 4x4 Nsst can be applied even when 4xN / Nx4 (N>=16).

[0204] Both 8x8 Nsst and 4x4 Nsst are non-separable transforms according to the transform combination configuration described in this document, so 8x8 Nsst receives 64 data inputs and outputs 64 data, while 4x4 Nsst has 16 inputs and 16 outputs.

[0205] Both 8x8 Nsst and 4x4 Nsst are composed of a hierarchical combination of Givens rotations. The matrix corresponding to one Gibbons rotation is shown in Equation 4, and the matrix product is shown in Equation 5.

[0206]

number

[0207]

number

[0208] As shown in Figure 13, one Gibbons rotation rotates two data, so a total of 32 or 8 Gibbons rotations are required to process 64 (in the case of 8x8 NSST) or 16 (in the case of 4x4 NSST) data, respectively.

[0209] Therefore, a Gibbons rotation layer is formed by bundling 32 or 8 elements. The output data of one Gibbons rotation layer is transmitted as input data to the next Gibbons rotation layer with a fixed permutation.

[0210] FIG. 14 shows a one-round configuration of 4x4 Nsst composed of a Gibbons rotation layer and permutation as an embodiment to which the present invention is applied.

[0211] If we look carefully at Figure 14, we can see that in the case of 4x4 Nsst, four Gibbons rotation layers are processed sequentially. The output data of the Gibbons rotation layer shown in Figure 14 goes through a fixed permutation (i.e. shuffling) and is transmitted as the input data of the next Gibbons rotation layer.

[0212] As shown in FIG. 14, the permutation pattern is determined regularly. In the case of 4x4 Nsst, four Gibbons rotation layers and their permutations form one round.

[0213] In the case of 8x8 Nsst, six Gibbons rotation layers and their permutations form one round. 4x4 Nsst goes through the second round, and 8x8 Nsst goes through the fourth round. Different rounds use the same permutation pattern, but the Gibbons rotation angles applied are different. Therefore, the angle data of all Gibbons rotations that make up each transformation must be stored.

[0214] In the last step, one more final substitution is performed on the data output through the Gibbons rotation layer, and the substitution information is saved separately for each conversion. The substitution is finally performed by the forward NSST, and conversely, in the inverse NSST, the inverse substitution is applied first.

[0215] In the case of the inverse Nsst, it is advisable to execute the Gibbons rotation layer and substitution applied in the forward Nsst in reverse order, and also rotate each Gibbons rotation angle by taking a (-) value.

[0216] FIG. 15 is a block diagram for explaining the operations of a forward reduced transform and a reverse reduced transform as an embodiment to which the present invention is applied.

[0217] Reduced Secondary Transform (RST)

[0218] Assuming that an orthogonal matrix representing one transformation has the form of NxN, the reduced transform (hereinafter referred to as "RT") is to leave only R out of N transformation basis vectors (R < N). The matrix of the forward RT for generating transformation coefficients is given as in Equation 6 below.

[0219]

Equation

[0220] Since the matrix of the inverse RT is the transpose matrix of the forward RT matrix, when illustrating the application of the forward RT and the inverse RT, it is as shown in FIG. 15 above.

[0221] Assuming the case of applying the RT to the upper left 8x8 block of the transformed block that has undergone a primary transformation, the RT can be named an 8x8 reduced secondary transform (8x8 RST).

[0222] When the R value in Equation 6 is 16, the forward 8x8 RST has a 16x64 matrix shape, and the reverse 8x8 RST has a 64x16 matrix shape.

[0223] Also, the same configuration of the transformation set as in Fig. 12 can be applied to the 8x8 RST. That is, a corresponding 8x8 RST can be applied based on the transformation set in Fig. 12.

[0224] In one embodiment, when one transform set in Fig. 12 is composed of two or three transforms depending on the intra prediction mode, one of up to four transforms may be selected, including a case where no secondary transform is applied, where one transform can be viewed as an identity matrix.

[0225] When the four transforms are assigned indices of 0, 1, 2, and 3, respectively, a syntax element called an Nsst index can be signaled for each transform block to specify the corresponding transform. That is, for an 8x8 top left block, an 8x8 Nsst can be specified in the case of an Nsst, and an 8x8 RST can be specified in the configuration of an RST through the Nsst index. In addition, the 0th index can be assigned to an identity matrix, i.e., when no quadratic transform is applied.

[0226] When the forward 8x8 RST as in Equation 6 is applied, 16 valid transform coefficients are generated, so it can be said that 64 input data constituting an 8x8 area is reduced to 16 output data. From the perspective of a two-dimensional area, only about 1 / 4 of the area is filled with valid transform coefficients. Therefore, the 16 output data obtained by applying the forward 8x8 RST is filled in the upper left corner area of ​​4x4 in FIG. 16.

[0227] FIG. 16 is a diagram illustrating a process of performing a reverse scan from the 64th to the 17th blocks based on a reverse scan order, as an embodiment to which the present invention is applied.

[0228] FIG. 16 shows a scan of the 17th coefficient to the 64th coefficient (in the forward scan order) when the forward scan order starts from 1. However, FIG. 16 shows a reverse scan, which indicates a reverse scan from the 64th coefficient to the 17th coefficient.

[0229] 16, the upper left 4x4 region is a Region of Interest (ROI) to which valid transform coefficients are assigned, and the remaining regions are empty, i.e., the remaining regions may be assigned a value of 0 by default.

[0230] If there is a valid non-zero transform coefficient outside the ROI region of FIG. 16, it means that the 8x8 RST is not applied. In this case, the corresponding Nsst index coding can be omitted.

[0231] Conversely, if there are no non-zero transform coefficients outside the ROI region in Figure 16 (when 8x8 RST is applied, areas outside the ROI are assigned 0), it is possible that 8x8 RST has been applied, so the Nsst index can be coded.

[0232] In this manner, the conditional Nsst index coding can be performed after the residual coding process since it is necessary to check the presence or absence of a transform coefficient that is not 0 (zero).

[0233] The present invention deals with the design of a RST and related optimization methods that can be applied to 4x4 blocks from the RST structure. The embodiments described herein can be applied not only to 4x4 RSTs, but also to 8x8 RSTs or other forms of transformation.

[0234] FIG. 17 shows three forward scan sequences of transform coefficient blocks (transform blocks) as an embodiment to which the present invention is applied.

[0235] Embodiment 1: RST applicable to 4x4 blocks

[0236] The only non-separable transform that can be applied to a 4x4 block is a 16x16 transform. That is, if the data elements that make up a 4x4 block are arranged in row-first or column-first order, they become a 16x1 vector, to which a non-separable transform can be applied.

[0237] The forward 16x16 transform is composed of 16 row-wise transform basis vectors, and the transform coefficients of the transform basis vectors are obtained by applying the inner product between the 16x1 vector and each transform basis vector. The process of obtaining the transform coefficients corresponding to all 16 transform basis vectors is the same as multiplying the input 16x1 vector by a 16x16 non-separable transform matrix.

[0238] The transform coefficients obtained by the matrix multiplication have the form of a 16x1 vector, but each transform coefficient may have different statistical characteristics. For example, if a 16x1 transform coefficient vector is composed of the 0th element to the 15th element, the variance of the 0th element may be greater than the variance of the 15th element. That is, the earlier an element is located, the greater its variance value and the greater its energy value.

[0239] By applying an inverse 16x16 non-separable transform from the 16x1 transform coefficients, the original 4x4 block signal can be restored. If the forward 16x16 non-separable transform is an orthonormal transform, its inverse 16x16 transform can be obtained via the transpose matrix of the forward 16x16 transform.

[0240] By multiplying the inverse 16x16 non-separable transform matrix with the 16x1 transform coefficient vector, we obtain 16x1 vector data, and by arranging it in row-first or column-first order, we can restore the 4x4 block signal.

[0241] As previously mentioned, the elements of a 16x1 transform coefficient vector may have different statistical properties.

[0242] If the transform coefficients located forward (close to the 0th element) have a larger energy, a signal close to the original signal can be restored even if an inverse transform is applied to some of the transform coefficients that appear first without using all the transform coefficients. For example, if an inverse 16x16 non-separable transform is composed of 16 column basis vectors, a 16xL matrix can be constructed by leaving only L column basis vectors. In addition, after leaving only the important L transform coefficients among the transform coefficients (Lx1 vector), a 16xL matrix and an Lx1 vector can be multiplied to restore a 16x1 vector with a small error from the original input 16x1 vector data.

[0243] As a result, since only L coefficients are used to restore data, there is no 16x1 transform coefficient vector when obtaining transform coefficients, but an Lx1 transform coefficient vector is obtained. That is, by selecting L row-direction transform vectors from a forward 16x16 non-separable transform matrix to construct an Lx16 transform, and then multiplying it by a 16x1 input vector, the important L transform coefficients can be obtained.

[0244] The L value has a range of 1≦L<16. In general, L of the 16 transformation basis vectors can be selected in any manner. However, from the viewpoint of encoding and decoding, it may be advantageous in terms of coding efficiency to select a transformation basis vector that is important in terms of the energy of the signal.

[0245] Embodiment 2: Setting the application area of ​​4x4 RST and arranging the conversion coefficients

[0246] The 4x4 RST can be applied to a secondary transform, and can be applied secondarily to a block to which a primary transform such as DCT-type 2 has been applied. When the size of a block to which a primary transform has been applied is NxN, the size of a block to which a primary transform has been applied is generally larger than 4x4. Therefore, when applying the 4x4 RST to the NxN block, there are two methods:

[0247] Embodiment 2-1) Instead of applying 4x4 RST to the entire NxN region, it can be applied only to a part of the region. For example, it can be applied only to the upper left MxM region (M≦N).

[0248] In embodiment 2-2), after dividing the area to which the secondary transformation is applied into 4x4 blocks, a 4x4 RST can be applied to each divided block.

[0249] As an embodiment, the above-mentioned embodiments 2-1) and 2-2) can be applied in combination. For example, only the upper left MxM region can be divided into 4x4 blocks, and then 4x4 RST can be applied.

[0250] In one embodiment, a secondary transformation is applied only to the upper left 8x8 region, and if the NxN block is equal to or greater than 8x8, an 8x8 RST is applied, and if the NxN block is smaller than 8x8 (4x4, 8x4, 4x8), it can be divided into 4x4 blocks as in embodiment 2-2) and a 4x4 RST can be applied to each block. Also, if 4xN / Nx4 (N>=16), a 4x4 RST can be applied.

[0251] When L (1≦L<16) transform coefficients are generated after applying 4×4 RST, there is a degree of freedom in how to arrange the L transform coefficients. However, since there is a fixed order when processing the transform coefficients in the residual coding step, the coding performance may change depending on how the L transform coefficients are arranged in a two-dimensional block.

[0252] For example, in the case of HEVC residual coding, coding starts from the position farthest from the DC position in order to improve coding performance by utilizing the fact that the value of the quantized coefficient is closer to 0 than it is to the DC position.

[0253] Therefore, it may be advantageous in terms of coding performance to arrange the L transform coefficients so that the more important coefficients having higher energy are coded later in the residual coding order.

[0254] Figure 17 shows three forward scan orders for 4x4 transform blocks (Coefficient Group (CG)) applied in HEVC. In residual coding, the scan order in Figure 17 is reversed (i.e., coding is performed in the order from 16 to 1).

[0255] Since the three scan orders presented in FIG. 17 are selected depending on the intra prediction mode, the present invention can be configured to determine the scan order for the L transform coefficients depending on the intra prediction mode as well.

[0256] FIG. 18 illustrates the positions of valid transform coefficients and the forward scan order for each 4x4 block when a diagonal scan is applied to the upper left 4x8 block and a 4x4 RST is applied, as an embodiment to which the present invention is applied.

[0257] When the upper left 4x8 block is divided into 4x4 blocks according to the diagonal scan order in FIG. 17 and a 4x4 RST is applied to each of them, if the value of L is 8 (i.e., only 8 transform coefficients out of 16 remain), the transform coefficients can be positioned as shown in FIG. 18.

[0258] Only half of each 4x4 block may have transform coefficients, and locations where an X appears may be assigned a value of 0 by default.

[0259] Therefore, L transform coefficients are arranged for each 4x4 block according to the scan order shown in Figure 17, and residual coding can be applied to the remaining (16 - L) positions of each 4x4 block by assuming that they are filled with 0s.

[0260] FIG. 19 shows an embodiment to which the present invention is applied, in which a diagonal scan is applied to the upper left 4x8 block and a 4x4 RST is applied to combine valid transform coefficients of two 4x4 blocks into one 4x4 block.

[0261] 19, it can be seen that L transform coefficients arranged in two 4x4 blocks can be merged into one. In particular, when the value of L is 8, the transform coefficients of the two 4x4 blocks are merged to completely fill one 4x4 block, so that no transform coefficients remain in the other 4x4 block.

[0262] Therefore, for such vacated 4x4 blocks, most residual coding is not required, so the corresponding coded_sub_block_flag can be coded to 0.

[0263] In addition, in an embodiment of the present invention, various methods can be applied to how to mix the transform coefficients of two 4x4 blocks. Although they may be mixed in any order, the present invention can provide the following method.

[0264] 1) The transform coefficients of two 4x4 blocks are alternately shuffled in the scan order. That is, in FIG. 18, the transform coefficients of the upper block are JPEG2025069352000006.jpg8170, and the transformation coefficients of the lower block are When the image is JPEG2025069352000007.jpg9170, They can be mixed one after the other, like this: JPEG2025069352000008.jpg8170, or JPEG2025069352000009.jpg9119 and You can also change the order of JPEG2025069352000010.jpg8117. You can set it so that JPEG2025069352000011.jpg9119 comes first.

[0265] 2) The transform coefficients of the first 4x4 block can be arranged first followed by the transform coefficients of the second 4x4 block, i.e. You can connect and place it like this: JPEG2025069352000012.jpg9170. Or, You can also change the order, for example, JPEG2025069352000013.jpg8170.

[0266] Embodiment 3: A method for coding NSST index of 4x4 RST

[0267] When the 4x4 RST is applied as shown in FIG. 18, the L+1th to 16th transform coefficients of each 4x4 block can be filled with 0 values ​​according to the scan order of the transform coefficients.

[0268] Therefore, if a non-zero value occurs in the L + 1 to 16 positions of any one of the two 4x4 blocks, it becomes clear that the 4x4 RST does not apply.

[0269] If the 4x4 RST also has a structure for selecting and applying one of a set of prepared transformations such as Nsst, it is possible to signal a transformation index (which can be named an Nsst index in this embodiment) indicating which transformation to apply.

[0270] Suppose a decoder knows the Nsst index through bitstream analysis and performs such analysis after residual decoding.

[0271] If residual decoding is performed and it is confirmed that there is at least one non-zero transform coefficient between L+1-th and L+16-th, 4x4 RST is not applied, so it can be set not to analyze the Nsst index.

[0272] Therefore, the Nsst index is selectively analyzed only when necessary, thereby reducing signaling costs.

[0273] As shown in Fig. 18, if a 4x4 RST is applied to a plurality of 4x4 blocks in a particular region (for example, the same 4x4 RST may be applied to all of them, or a different 4x4 RST may be applied to each of them), a 4x4 RST to be applied to all of the 4x4 blocks may be specified via one Nsst index. In this case, the same 4x4 RST may be specified, or a 4x4 RST to be applied to each of the 4x4 blocks may be specified.

[0274] Since the 4x4 RST and application of all the 4x4 blocks are determined by one Nsst index, it can be checked during the residual decoding process whether a non-zero transform coefficient exists at the L+1th to 16th positions for all the 4x4 blocks. If a non-zero transform coefficient exists at an impermissible position (L+1th to 16th positions) in one 4x4 block as a result of the check, it can be configured not to code the Nsst index.

[0275] The Nsst index may be signaled separately for luma blocks and chroma blocks, and in the case of chroma blocks, separate Nsst indices may be signaled for Cb and Cr, or one Nsst index may be shared.

[0276] If Cb and Cr share one Nsst index, a 4x4 RST specified by the same Nsst index can be applied. In this case, the 4x4 RST itself for Cb and Cr may be the same, or the Nsst index may be the same but have individual 4x4 RSTs.

[0277] To apply the above-mentioned conditional signaling to the shared Nsst index, it can be configured to check that there are non-zero transform coefficients from L+1 to 16 for all 4x4 blocks of Cb and Cr, and not signal the Nsst index if there are non-zero transform coefficients.

[0278] As shown in FIG. 19, when 4x4 RST is applied to combine transform coefficients of two 4x4 blocks, it is possible to determine whether the Nsst index is to be signaled after checking whether a non-zero transform coefficient exists in a position where no valid transform coefficient exists.

[0279] For example, as shown in Figure 19(b), since the L value is 8, when 4x4 RST is applied to one 4x4 block, if there is no valid transform coefficient (block marked with X), the coded_sub_block_flag of the block without valid transform coefficients can be checked. In this case, if coded_sub_block_flag is 1, it can be set not to signal the Nsst index.

[0280] Embodiment 4: Optimization method when coding of Nsst index is performed before residual coding

[0281] When coding of the Nsst index is performed before residual coding, whether or not to apply 4x4 RST is determined in advance, so that residual coding can be omitted for positions where the transform coefficients are assigned to 0.

[0282] Here, whether or not the 4x4 RST is applied can be configured to be known via the Nsst index. For example, if the Nsst index is 0, the 4x4 RST is not applied.

[0283] Alternatively, it can be signaled via another syntax element (e.g., Nsst flag). For example, if another syntax element is Nsst flag, the Nsst flag is first parsed to determine whether 4x4 RST is applied, and if the value of the Nsst flag is 1, residual coding can be omitted at positions where no valid transform coefficients exist.

[0284] In one embodiment, when performing residual coding, the position of the last non-zero transform coefficient on the TU is coded first. If the coding of the Nsst index is performed after the coding of the position of the last non-zero transform coefficient, and it is determined that the position of the last non-zero transform coefficient is a position where a non-zero transform coefficient cannot occur when applying 4x4 RST, it is possible to configure not to code the Nsst index and not to apply 4x4 RST.

[0285] For example, when 4x4 RST is applied to the position indicated by X in Figure 18, a valid transform coefficient is not located (e.g., a value of 0 can be satisfied), so if the last non-zero transform coefficient is located in the area indicated by X, coding of the Nsst index can be omitted. If the last non-zero transform coefficient is not located in the area indicated by X, coding of the Nsst index can be performed.

[0286] As an embodiment, when the Nsst index is conditionally coded after coding the position of the last non-zero transform coefficient to check whether to apply it to the 4x4 RST, the remaining residual coding portion can be processed using the following two schemes.

[0287] 1) If 4x4 RST is not applied, general residual coding is maintained as is, that is, coding is performed under the assumption that a non-zero transform coefficient can exist at any position from the position of the last non-zero transform coefficient to DC.

[0288] 2) When applying 4x4 RST, since there are no transform coefficients for a particular position or a particular 4x4 block (e.g., the X position in Figure 18 can be filled with 0 by default), residual coding may not be performed for the corresponding position or block.

[0289] For example, when reaching the position indicated by X in Figure 18, coding of sig_coeff_flag can be omitted. Here, sig_coeff_flag indicates a flag indicating whether a non-zero transform coefficient exists at the corresponding position.

[0290] When combining transform coefficients of two blocks as shown in FIG. 19, for a 4x4 block assigned to 0, the coding of coded_sub_block_flag can be omitted and the corresponding value can be induced to 0, and for the corresponding 4x4 block, the value of all 0 can be induced without additional coding.

[0291] When coding the Nsst index after coding the position of the last non-zero transform coefficient, if the x position (Px) and y position (Py) of the last non-zero transform coefficient are less than Tx and Ty, respectively, it is possible to omit the Nsst index coding and not apply the 4x4 RST.

[0292] For example, when Tx=1 and Ty=1, it means that the Nsst index coding is omitted when the last non-zero transform coefficient is present at the DC position.

[0293] The method of determining whether to perform Nsst index coding through comparison with such a threshold can be applied differently to luma and chroma, for example, different Tx and Ty can be applied to luma and chroma, or a threshold can be applied to luma but not to chroma, or vice versa.

[0294] The above two methods, i.e., the first method of omitting Nsst index coding when the last non-zero transform coefficient is located in an area where no valid transform coefficient exists, and the second method of omitting Nsst index coding when the X coordinate and Y coordinate of the last non-zero transform coefficient are each smaller than a certain threshold value, can also be applied together.

[0295] For example, it is possible to first check the threshold value of the position coordinate of the last non-zero transform coefficient, and then check whether the last non-zero transform coefficient is located in an area where no valid transform coefficient exists. Alternatively, the order can be changed.

[0296] The method presented in the fourth embodiment can also be applied to 8x8 RST. That is, if the last non-zero transform coefficient is located in a non-upper left 4x4 area within the upper left 8x8 area, Nsst index coding can be omitted, and if not, Nsst index coding can be performed.

[0297] Also, if the values ​​of the X and Y coordinates of the position of the last non-zero transform coefficient are all less than the threshold, the coding of the Nsst index can be omitted, or the two methods can be applied together.

[0298] 5th embodiment: When RST is applied, different Nsst index coding and residual coding methods are applied for luma and chroma.

[0299] The schemes described in the third and fourth embodiments may be applied differently to luma and chroma, that is, Nsst index coding and residual coding schemes may be applied differently to luma and chroma.

[0300] For example, luma may apply the scheme of embodiment 4, and chroma may apply the scheme of embodiment 3. Or luma may apply the conditional Nsst index coding presented in embodiment 3 or embodiment 4, and chroma may not apply the conditional Nsst index coding. Or vice versa.

[0301] FIG. 20 shows a flowchart of a video signal encoding method according to an embodiment of the present invention, which is based on a Reduced Secondary Transform (RST).

[0302] The encoder may determine (or select) a forward secondary transform based on at least one of a prediction mode, a block shape, and / or a block size of the current block (S2010). In this case, the forward secondary transform candidates may include at least one of the embodiments of FIG. 6 and / or FIG. 12.

[0303] The encoder may determine an optimal forward quadratic transform through RD optimization (Rate Distortion optimization). The optimal forward quadratic transform may correspond to one of a combination of a plurality of transforms, and the combination of the plurality of transforms may be defined by a transform index. For example, for RD optimization, results of performing all of forward quadratic transform, quantization, residual coding, etc. of each candidate may be compared. In this case, modifications such as cost = rate + λ·distortion or cost = distortion + λ·rate may be used, but the present invention is not limited thereto.

[0304] The encoder may signal a secondary transform index corresponding to the optimal forward secondary transform (S2020). Here, the secondary transform index may be any of the other embodiments described herein.

[0305] For example, the secondary transform index may be configured as the transform set of FIG. 12. Since one transform set is composed of two or three transforms depending on the intra prediction mode, it may be configured to select one of up to four transforms, including the case where the secondary transform is not applied. When the four transforms are assigned indexes 0, 1, 2, and 3, respectively, the secondary transform index may be signaled for each transform coefficient block to specify the transform to be applied. In this case, the 0th index may be assigned to an identity matrix, i.e., when the secondary transform is not applied.

[0306] In another embodiment, the signaling of the secondary transform index may be performed at any one of the following stages: 1) before residual coding, 2) during residual coding (after position coding of the last non-zero transform coefficient), or 3) after residual coding. The embodiment will be described in detail as follows.

[0307] 1) Signaling secondary transform index before residual coding The encoder can determine a forward quadratic transform.

[0308] The encoder may code a secondary transform index corresponding to the forward secondary transform.

[0309] The encoder can code the position of the last non-zero transform coefficient.

[0310] The encoder may perform residual coding of syntax elements other than the position of the last non-zero transform coefficient.

[0311] 2) Signaling secondary transform index during residual coding

[0312] The encoder can determine a forward quadratic transform.

[0313] The encoder can code the position of the last non-zero transform coefficient.

[0314] If the last non-zero transform coefficient is not located in a specific region, the encoder may code an index of a secondary transform corresponding to the forward secondary transform. Here, the specific region indicates a remaining region excluding a position where a non-zero transform coefficient may exist when transform coefficients are arranged according to a scan order when a reduced secondary transform is applied. However, the present invention is not limited thereto.

[0315] The encoder may perform residual coding of syntax elements other than the position of the last non-zero transform coefficient.

[0316] 3) Signaling secondary transform index after residual coding

[0317] The encoder can determine a forward quadratic transform.

[0318] The encoder can code the position of the last non-zero transform coefficient.

[0319] If the last non-zero transform coefficient is not located in a specific region, the encoder may perform residual coding of syntax elements other than the position of the last non-zero transform coefficient. Here, the specific region refers to a remaining region excluding a position where a non-zero transform coefficient may exist when transform coefficients are arranged according to a scan order when a reduced secondary transform is applied. However, the present invention is not limited thereto.

[0320] The encoder may code a secondary transform index corresponding to the forward secondary transform.

[0321] On the one hand, the encoder can perform a forward first-order transform on the current block (residual block) (S2030). Here, the forward first-order transform can be similarly applied to the S2010 stage and / or the S2020 stage.

[0322] The encoder can perform a forward second-order transform on the current block using the optimal forward second-order transform (S2040). For example, the optimal forward second-order transform can be a reduced second-order transform. The reduced second-order transform means a transform in which N pieces of residual data (Nx1 residual vector) are input and L pieces of transform coefficient data (Lx1 transform coefficient vector) (L < N) are output.

[0323] As one embodiment, the reduced second-order transform can be applied to a specific region of the current block. For example, when the current block is NxN, the specific region can mean the upper left N / 2xN / 2 region. However, the present invention is not limited thereto, and it can be set to be different based on at least one of a prediction mode, a block shape, or a block size. For example, when the current block is NxN, the specific region can mean the upper left MxM region (M ≦ N).

[0324] On the one hand, the encoder can generate a transform coefficient block by quantizing the current block (S2050).

[0325] The encoder can perform entropy encoding on the transform coefficient block to generate a bitstream.

[0326] FIG. 21 shows a flowchart for decoding a video signal based on a reduced secondary transform (RST) as an embodiment to which the present invention is applied.

[0327] The decoder can obtain the index of the secondary transformation from the bitstream (S2110). Here, the other embodiments described in this specification can be applied to the secondary transformation index. For example, the secondary transformation index can include at least one of the embodiments of FIG. 6 and / or FIG. 12.

[0328] As another embodiment, the step of obtaining the secondary transformation index can be executed at any one of the following stages: 1) before residual decoding, 2) during the middle of residual decoding (after decoding the position of the last non-zero transformation coefficient), or 3) after residual decoding.

[0329] The decoder can induce the secondary transformation corresponding to the secondary transformation index (S2120). At this time, the candidates for the secondary transformation can include at least one of the embodiments of FIG. 6 and / or FIG. 12.

[0330] However, the steps of S2110 and S2120 are one embodiment, and the present invention is not limited thereto. For example, without obtaining the index of the secondary transformation, the decoder can induce the secondary transformation based on at least one of the block shape and / or block size in the prediction mode of the current block.

[0331] On the other hand, the decoder can entropy-decode the bitstream to obtain a transform coefficient block, and perform inverse quantization on the transform coefficient block (S2130)

[0332] The decoder can perform an inverse secondary transformation on the inverse-quantized transform coefficient block (S2140). For example, the inverse secondary transformation can be a reduced secondary transformation. The reduced secondary transformation means a transformation in which N pieces of residual data (Nx1 residual vector) are input and L pieces of transform coefficient data (Lx1 transform coefficient vector) are output (L < N).

[0333] In one embodiment, the reduced secondary transform may be applied to a specific region of the current block. For example, when the current block is NxN, the specific region may refer to an upper left N / 2xN / 2 region. However, the present invention is not limited thereto, and may be set differently based on at least one of a prediction mode, a block shape, or a block size. For example, when the current block is NxN, the specific region may refer to an upper left MxM region (M≦N) or MxL (M≦N, L≦N).

[0334] The decoder may then perform an inverse linear transform on the result of the inverse secondary transform (S2150). The decoder generates a residual block through operation S2150, and generates a reconstructed block by adding the residual block and a predicted block.

[0335] FIG. 22 shows a structural diagram of a content streaming system as an embodiment to which the present invention is applied.

[0336] As shown in FIG. 22, a content streaming system to which the present invention is applied mainly includes an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0337] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmits the bitstream to the streaming server. As another example, if a multimedia input device such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0338] The bitstream is generated by an encoding method or a bitstream generating method to which the present invention is applied, and the streaming server can temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0339] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transfers the request to the streaming server, and the streaming server transmits multimedia data to the user. Here, the content streaming system may include a separate control server, and in this case, the control server controls commands / responses between devices in the content streaming system.

[0340] The streaming server receives content from a media repository and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, the streaming server can store the bitstream for a certain period of time to provide a smooth streaming service.

[0341] Examples of the user devices include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head mounted displays (HMDs)), digital TVs, desktop computers, digital signage, and the like.

[0342] Each server in the content streaming system can be operated as a distributed server, in which case data received by each server can be processed in a distributed manner.

[0343] As described above, the embodiments described in the present invention may be implemented and performed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each drawing may be implemented and performed on a computer, processor, microprocessor, controller, or chip.

[0344] In addition, the decoder and encoder to which the present invention is applied can be included in a multimedia broadcasting transmitting / receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video conversation device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a custom video (VoD) service providing device, an over-the-top video (OTT) device, an internet streaming service providing device, a three-dimensional (3D) video device, an image telephone video device, and a medical video device, and can be used to process a video signal and a data signal. For example, an over-the-top video (OTT) device includes a game console, a Blu-ray player, an internet-connected TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), and the like.

[0345] In addition, the processing method to which the present invention is applied can be produced in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present invention can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium can include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium also includes a medium realized in the form of a carrier wave (e.g., transmission via the Internet). Also, a bit stream generated by the encoding method can be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0346] Furthermore, the embodiments of the present invention may be realized as a computer program product by program code, which is executed in a computer according to the embodiments of the present invention. The program code may be stored on a carrier readable by the computer. [Industrial Applicability]

[0347] The above-described preferred embodiments of the present invention have been disclosed for illustrative purposes, and those skilled in the art may improve, modify, substitute or add various other embodiments within the technical spirit and scope of the present invention disclosed in the appended claims.

Claims

1. 1. A method for restoring a video signal based on a reduced quadratic transform, comprising: obtaining a secondary transform index from the video signal based on the presence of zero transform coefficients at all of the particular transform coefficient positions; deriving an inverse secondary transform matrix corresponding to the secondary transform index, the inverse secondary transform matrix representing the reduced secondary transform that outputs N residual data (Nx1 residual vector) based on L (L<N) input transform coefficient data (Lx1 transform coefficient vector); obtaining non-zero transform coefficients by performing entropy decoding and inverse quantization on the current block; performing an inverse quadratic transform on the non-zero transform coefficients based on the inverse quadratic transform matrix; performing an inverse linear transform on the block to which the inverse secondary transform has been applied; and restoring the current block based on the block to which the inverse linear transformation has been applied, A method in which the positions of the particular transform coefficients are arranged in diagonal scan order from the (L+1)th position to the 16th position in the transform coefficients of the block to which the inverse quadratic transform is applied, the block is a 4x4 block, N is 16, and L is 8.

2. The inverse secondary transform is applied to a particular region of the current block, The method of claim 1 , wherein the particular region is a top-left region within the current block.

3. The method of claim 1 , wherein a 4×4 inverse quadratic transform is applied to each of the divided 4×4 blocks in the current block.

4. The method comprises: obtaining a primary transform index of the current block from the video signal, the primary transform index corresponding to any one of a plurality of transform combinations including a combination of Discrete Sine Transform 7 (DST7) and / or Discrete Cosine Transform 8 (DCT8); deriving a combination of transforms corresponding to the primary transform index; The combination of transformations includes a horizontal transformation and a vertical transformation, and the horizontal transformation and the vertical transformation correspond to any one of the DST7 or the DCT8; The method of claim 1 , wherein the inverse linear transform is performed based on a combination of the transforms.

5. 1. A method for encoding a video signal based on a reduced quadratic transform, comprising the steps of: performing a linear transformation on the transformation block; deriving a quadratic transformation matrix to be applied to the transformation block, the quadratic transformation matrix representing a reduced quadratic transformation that outputs L (L<N) pieces of transformation coefficient data (Lx1 transformation coefficient vector) based on N pieces of input residual data (Nx1 residual vector); performing a quadratic transformation on the transformation block based on the quadratic transformation matrix; obtaining transform coefficients based on the quadratic transform; performing quantization and entropy encoding on the transform coefficients; generating a secondary transformation index corresponding to the secondary transformation matrix; the secondary transform index is generated based on zero transform coefficients being present at all of the positions of a particular transform coefficient; A method in which the positions of the particular transform coefficients are arranged in diagonal scan order from the (L+1)th position to the 16th position of the transform coefficients of a block to which the secondary transform is applied, the block being a 4x4 block, N is 16, and L is 8.

6. 1. A transmission method for data including a bitstream for an image, comprising: obtaining the bitstream for the image, the bitstream comprising: performing a linear transformation on the transformation block; deriving a quadratic transformation matrix to be applied to the transformation block, the quadratic transformation matrix representing a reduced quadratic transformation that outputs L (L<N) pieces of transformation coefficient data (Lx1 transformation coefficient vector) based on N pieces of input residual data (Nx1 residual vector); performing a quadratic transformation on the transformation block based on the quadratic transformation matrix; obtaining transform coefficients based on the quadratic transform; performing quantization and entropy encoding on the transform coefficients; and encoding secondary transform indices corresponding to the secondary transform matrix; transmitting the data including the bitstream; the secondary transform index is generated based on zero transform coefficients being present at all of the positions of a particular transform coefficient; A transmission method in which the positions of the specific transform coefficients are arranged in diagonal scan order from the (L+1)th position to the 16th position in the transform coefficients of a block to which the secondary transform is applied, the block is a 4x4 block, N is 16, and L is 8.

Images