Method and apparatus for processing video signals

Abstract

To provide a video signal processing method and apparatus.SOLUTION: A method of decoding an image signal includes determining a non-separable transform set index indicating a non-separable transform set used for a non-separable transform of a current block among pre-defined non-separable transform sets based on an intra prediction mode of the current block, determining a transform kernel indicated by the non-separable transform index of the current block among transform kernels included in the non-separable transform set indicated by the non-separable transform set index as a non-separable transform matrix, and applying the non-separable transform matrix to an upper left region of the current block determined according to a width and a height of the current block, wherein each of the pre-defined non-separable transform sets includes two transform kernels.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a method and an apparatus for processing video signals, and particularly to a method and an apparatus for encoding or decoding video signals by performing conversion.

Background Art

[0002] Compression encoding (encoding) means a series of signal processing techniques for transferring digitized information via a communication line or storing it in a form suitable for a storage medium. Media such as video, video, and audio can be the targets of compression encoding, and particularly, the technique of performing compression encoding on video is called video compression.

[0003] Next-generation video content will come to have characteristics such as high spatial resolution, high frame rate, and high dimensionality of scene representation. To process such content, it brings a huge increase in terms of memory storage, memory access rate, and processing power.

[0004] Therefore, it is necessary to design coding tools for more efficiently processing next-generation video content. In particular, video codec standards after the HEVC (High Efficiency Video Coding) standard require efficient conversion techniques for converting video signals in the spatial domain to the frequency domain, along with prediction techniques having higher accuracy.

Summary of the Invention

Problems to be Solved by the Invention

[0005] Embodiments of the present invention provide a video signal processing method and apparatus that apply a conversion suitable for the current block.

[0006] The technical problems that this invention aims to solve are not limited to those mentioned above, and any other technical problems not mentioned should be clearly understood by those with ordinary skill in the art to which this invention pertains from the description below. [Means for solving the problem]

[0007] A video signal decoding method according to an embodiment of the present invention includes the steps of: determining an inseparable transformation set index that indicates an inseparable transformation set to be used for the inseparable transformation of the current block from among a predefined inseparable transformation set based on the intra-prediction mode of the current block; determining an inseparable transformation matrix from among the transformation kernels included in the inseparable transformation set indicated by the inseparable transformation index of the current block, and applying the inseparable transformation matrix to the upper left region of the current block determined according to the width and height of the current block, wherein each of the predefined inseparable transformation sets includes two transformation kernels.

[0008] Furthermore, the non-separable transformation set index can be assigned to each of the four transformation sets, which are set according to the range of the intra-prediction mode.

[0009] Furthermore, if the intra prediction mode is 0 to 1, the non-separable transformation set index is determined to the first index value; if the intra prediction mode is 2 to 12 or 56 to 66, the non-separable transformation set index is determined to the second index value; if the intra prediction mode is 13 to 23 or 45 to 55, the non-separable transformation set index is determined to the third index value; and if the intra prediction mode is 24 to 44, the non-separable transformation set index can be determined to the fourth index value.

[0010] Furthermore, two unseparable transform kernels can be set for each index value of the unseparable transform set index.

[0011] Furthermore, the non-separable transformation matrix can be applied when the non-separable transformation index is not 0 and the width and height of the current block are each 4 or greater.

[0012] Furthermore, the process may further include steps to apply horizontal and vertical transformations to the current block to which the non-separable transformation has been applied.

[0013] Furthermore, the horizontal and vertical transformations can be determined based on the prediction mode applied to the current block and the MTS (multiple transform selection) index for selecting the transformation matrix.

[0014] A video signal decoding device according to an embodiment of the present invention includes a memory for storing the video signal and a processor coupled to the memory, wherein the processor determines an inseparable transformation set index that indicates the inseparable transformation set to be used for the inseparable transformation of the current block from among predefined inseparable transformation sets based on the intra-prediction mode of the current block, obtains the inseparable transformation index of the current block, determines the transformation kernel indicated by the inseparable transformation index from among the transformation kernels included in the inseparable transformation set indicated by the inseparable transformation index using an inseparable transformation matrix, and is configured to apply the inseparable transformation matrix to the upper left region of the current block, which is determined according to the width and height of the current block, and each of the predefined inseparable transformation sets includes two transformation kernels. [Effects of the Invention]

[0015] According to embodiments of the present invention, the conversion efficiency can be increased by determining and applying a conversion that is currently suitable for the block.

[0016] The effects obtained by the present invention are not limited to those mentioned above, and any other effects not mentioned should be clearly understood by a person with ordinary skill in the art to which the present invention pertains from the following description. [Brief explanation of the drawing]

[0017] The accompanying drawings, included as part of the detailed description to aid in understanding the present invention, provide embodiments of the invention and illustrate the technical features of the invention together with the detailed description.

[0018] [Figure 1] An example of a video coding system is shown as an embodiment to which the present invention is applied. [Figure 2] As an embodiment to which the present invention is applied, a schematic block diagram of an encoding device in which video / image signals are encoded is shown. [Figure 3] A schematic block diagram of a decoding device that performs video signal decoding is shown as an embodiment to which the present invention is applied. [Figure 4] This is a structural diagram of a content streaming system, which is an embodiment to which the present invention is applied. [Figure 5a] This figure illustrates a block partitioning structure using a Quad Tree (QT, hereinafter referred to as "QT") as an embodiment to which the present invention can be applied. [Figure 5b] This figure illustrates a block partitioning structure using a BT (Binary Tree, hereinafter referred to as "BT") as an embodiment to which the present invention can be applied. [Figure 5c] This figure illustrates a block division structure using a Ternary Tree (TT) as an embodiment to which the present invention can be applied. [Figure 5d] This figure illustrates an embodiment to which the present invention can be applied, illustrating a block partitioning structure using an AT (Asymmetric Tree, hereinafter referred to as "AT"). [Figure 6]As an embodiment to which the present invention is applied, a schematic block diagram of a conversion and quantization unit and an inverse quantization and inverse conversion unit in an encoding device is shown. [Figure 7] As an embodiment to which the present invention is applied, a schematic block diagram of an inverse quantization and inverse conversion unit in a decoding device is shown. [Figure 8] It is a flowchart showing a process in which AMT (adaptive multiple transform) is executed. [Figure 9] It is a flowchart showing a decoding process in which AMT is executed. [Figure 10] It is a flowchart showing an inverse conversion process based on MTS according to an embodiment of the present invention. [Figure 11] It is a block diagram of a device that executes decoding based on MTS according to an embodiment of the present invention. [Figure 12] As an embodiment to which the present invention is applied, it is an encoding / decoding flowchart to which a secondary conversion is applied. [Figure 13] As an embodiment to which the present invention is applied, it is an encoding / decoding flowchart to which a secondary conversion is applied. [Figure 14] As an embodiment to which the present invention is applied, a diagram for explaining Givens rotation is shown. [Figure 15] As an embodiment to which the present invention is applied, a configuration of one round in a 4x4 NSST (non-separable secondary transform) composed of a Givens rotation layer and a permutation is shown. [Figure 16] As an embodiment to which the present invention is applied, the operation of RST (reduced secondary transform) is shown. [Figure 17] As an embodiment to which the present invention is applied, it is a diagram showing a process of executing an inverse scan from the 64th to the 17th in the reverse scan order. [Figure 18]An example of an encoding flowchart using a single transform indicator (STI) is shown as an embodiment to which the present invention is applied. [Figure 19] An example of an encoding flowchart using a unified transform indicator (UTI) is shown as an embodiment to which the present invention is applied. [Figure 20a] Another example of an encoding flowchart using a UTI is shown as an embodiment to which the present invention is applied. [Figure 20b] Another example of an encoding flowchart using a UTI is shown as an embodiment to which the present invention is applied. [Figure 21] An example of an encoding flowchart that performs a conversion is shown as an embodiment to which the present invention is applied. [Figure 22] An example of a decoding flowchart that performs a conversion is shown as an embodiment to which the present invention is applied. [Figure 23] An example of a detailed block diagram of the conversion unit 120 within the encoding device 100 is shown as an embodiment to which the present invention is applied. [Figure 24] An example of an embodiment to which the present invention is applied is shown, which is a detailed block diagram of the inverse conversion unit 230 in the decoding device 200. [Figure 25] A flowchart for processing a video signal is shown as an embodiment to which the present invention is applied. [Figure 26] An example of a block diagram of an apparatus for processing video signals is shown as an embodiment to which the present invention is applied. [Modes for carrying out the invention]

[0019] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The detailed description disclosed below, together with the accompanying drawings, is intended to illustrate exemplary embodiments of the present invention and is not intended to show only the only embodiments in which the present invention can be carried out. The following detailed description includes specific details to provide a complete understanding of the present invention. However, those skilled in the art will see that the present invention can be carried out without such specific details.

[0020] In some cases, known structures and devices may be omitted or shown in the form of block diagrams focusing on the core function of each structure and device, in order to avoid ambiguity of the concept of the present invention.

[0021] Furthermore, while the terminology used in this invention has been selected as widely used and common terms whenever possible, in certain cases, the applicant has chosen terminology at their discretion for explanation. In such cases, the meaning will be clearly described in the detailed explanation of that part, and it should be made clear that the terminology used in the description of this invention should not be interpreted simply by its name alone, but rather by understanding the meaning of that term.

[0022] Certain terms used in the following description are provided to aid in understanding the present invention, and the use of such specific terms can be modified in other ways without departing from the technical spirit of the invention. For example, terms such as signals, data, samples, pictures, frames, and blocks may be appropriately substituted and interpreted during each coding process.

[0023] Hereinafter, in this specification, “processing unit” means a unit on which an encoding / decoding process, such as prediction, transformation, and / or quantization, is performed. Furthermore, “processing unit” can be interpreted to include units of luminance (luma) components and units of chroma components. For example, a processing unit may correspond to a block, a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0024] Furthermore, a processing unit can be interpreted as a unit of luminance components or a unit of chrominance components. For example, a processing unit may correspond to the CTB, CB, PU, ​​or TB of the luminance components. Or, a processing unit may correspond to the CTB, CB, PU, ​​or TB of the chrominance components. And, without limitation, a processing unit may also be interpreted to include units of both luminance and chrominance components.

[0025] Furthermore, processing units are not necessarily limited to square blocks; they may also be composed of polygons with three or more vertices.

[0026] In this specification, pixels or similar elements will be referred to as samples. Using samples can be interpreted as using pixel values ​​or similar elements.

[0027] Figure 1 shows an example of a video coding system as an embodiment to which the present invention is applied.

[0028] The video coding system may include a source device 10 and a receiving device 20. The source device 10 can transmit encoded video / image information or data to the receiving device 20 via a digital storage medium or network in file or streaming form.

[0029] The source device 10 may include a video source 11, an encoding device 12, and a transmitter 13. The receiving device 20 may include a receiver 21, a decoding device 22, and a renderer 23. The encoding device 10 may be called a video / image encoding device, and the decoding device 20 may be called a video / image decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer 23 may include a display unit, or it may consist of another device or external component of the display unit.

[0030] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source can include video / image capture devices and / or video / image generation devices. Video / image capture devices can include, for example, one or more cameras, or video / image archives containing previously captured video / images. Video / image generation devices can include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated through a computer, in which case the video / image capture process can be replaced by the process of generating the associated data.

[0031] The encoding device 12 can encode the input video / image. The encoding device 12 can perform a series of steps such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0032] The transfer unit 13 can transfer encoded video / image information or data output in bitstream form to the receiving unit of a receiving device via a digital storage medium or network in file or streaming form. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, SSD, etc. The transfer unit 13 may include elements for generating media files via a predetermined file format and may include elements for transfer via a broadcast / communication network. The receiver 21 can extract the bitstream and transfer it to the decoding device 22.

[0033] The decoding device 22 can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of the encoding device 12.

[0034] Renderer 23 can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0035] Figure 2 shows a schematic block diagram of an encoding device in which video / image signal encoding is performed, as an embodiment to which the present invention is applied. The encoding device 100 in Figure 2 can correspond to the encoding device 12 in Figure 1.

[0036] The video splitting unit 110 can split the input image (or picture, frame) input to the encoding device 100 into one or more processing units. For example, the processing units may be called coding units (CUs). In this case, coding units can be recursively split from a coding tree unit (CTU) or the largest coding unit (LCU) based on a QTBT (Quad-tree binary-tree) structure. For example, a single coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure and / or a binary tree structure. In this case, for example, the quad-tree structure may be applied first and the binary tree structure later, or the binary tree structure may be applied first. Based on the final coding unit that cannot be further split, the coding procedure according to the present invention can be executed. In this case, based on coding efficiency according to the characteristics of the video, the largest coding unit can be immediately used as the final coding unit, or, if necessary, the coding unit can be recursively divided into lower-depth coding units, and the coding unit of optimal size can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and reconstruction, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can be divided or partitioned from the final coding unit described above, respectively. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0037] The term "unit" is sometimes used interchangeably with terms such as "block" or "area." Generally, an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, or only the luminance (luma) component pixel / pixel value, or only the chroma component pixel / pixel value. A sample is used as the term corresponding to a single picture (or image) pixel or pel.

[0038] The encoding device 100 can generate a residual signal (remaining blocks, remaining sample array) by subtracting the predicted signal (predicted blocks, predicted sample array) output from the inter-prediction unit 180 or intra-prediction unit 185 from the input video signal (original blocks, original sample array), and the generated residual signal is transferred to the conversion unit 120. In this case, as shown in the figure, the unit that subtracts the predicted signal (predicted blocks, predicted sample array) from the input video signal (original blocks, original sample array) within the encoder 100 can be called the subtraction unit 115. The prediction unit can predict the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes the predicted samples of the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block or CU. As will be described later in the description of each prediction mode, the prediction unit can generate various information related to prediction, such as prediction mode information, and transfer it to the entropy encoding unit 190. Prediction information can be encoded by the entropy encoding unit 190 and output in bitstream format.

[0039] The intra-prediction unit 185 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) of the current block or at a distance, depending on the prediction mode. The prediction mode in intra-prediction can include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, DC mode and Planar mode. Directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is an example, and depending on the settings, more or fewer directional prediction modes may be used. The intra-prediction unit 185 can also determine the prediction mode to apply to the current block using the prediction modes applied to the surrounding blocks.

[0040] The interprediction unit 180 can guide the predicted block of the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, to reduce the amount of motion information transferred in interprediction mode, motion information can be predicted in blocks, subblocks, or samples based on the correlation of motion information between surrounding blocks and the current block. The motion information can include motion vectors and reference picture indices. The motion information can further include information on the interprediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, surrounding blocks can include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, same location (CU colCU), etc., and the reference picture containing the temporal neighboring block may be called a collocated picture (colPic). For example, the interpretation unit 180 can construct a motion information candidate list based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 180 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, the residual signal may not be transferred.In motion vector prediction (MVP) mode, the motion vector of the current block can be indicated by signaling the motion vector difference using the motion vectors of surrounding blocks in a motion vector predictor.

[0041] The prediction signal generated via the interpretation unit 180 or the intrapretation unit 185 is used to generate a restored signal or to generate a residual signal.

[0042] The transformation unit 120 can generate transformation coefficients by applying a transformation method to the residual signal. For example, the transformation method may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph representing the relationship information between pixels. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels. The transformation process may also be applied to pixel blocks of the same size and square shape, or it may be applied to non-square, variable-size blocks.

[0043] The quantization unit 130 quantizes the conversion coefficients and transfers them to the entropy encoding unit 190, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. Information about the quantized conversion coefficients can be called residual information. The quantization unit 130 can rearrange the quantized conversion coefficients in block form into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the quantized conversion coefficients in the form of a one-dimensional vector. The entropy encoding unit 190 can perform various encoding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy encoding unit 190 can also encode information necessary for video / image restoration (such as the values ​​of syntax elements) together or separately. The encoded information (e.g., video / image information) can be transferred or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The bitstream can be transferred over a network or stored on a digital storage medium. Here, the network can include broadcast networks and / or other networks, and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 190 can be transmitted by a transmission unit (not shown) and / or stored by a storage unit (not shown) which are configured as internal / external elements of the encoding device 100, or the transfer unit may be a component of the entropy encoding unit 190.

[0044] The quantized conversion coefficients output from the quantization unit 130 can be used to generate a prediction signal. For example, the quantized conversion coefficients can be used to reconstruct the residual signal by applying inverse quantization and inverse transformation via the inverse quantization unit 140 and inverse transformation unit 150 in the loop. The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 180 or intra-prediction unit 185. If there is no residual for the block to be processed, as in the case where skip mode is applied, the predicted block is used as the reconstructed block. The adder 155 can be called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal is used for intra-prediction of the next block to be processed in the current picture, and may also be used for inter-prediction of the next picture after filtering, as described later.

[0045] The filtering unit 160 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transfer the modified restored picture to the decoded picture buffer 170. Various filtering methods can include, for example, ji-blocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 160 can generate various filtering information, as described later in the description of each filtering method, and transfer it to the entropy encoding unit 190. The filtering information can be encoded by the entropy encoding unit 190 and output in bitstream format.

[0046] The corrected restored picture transferred to the decoded picture buffer 170 is used as a reference picture by the interpretation unit 180. This allows the encoding device to avoid prediction mismatches between the encoding device 100 and the decoding device when interpretation is applied, and also improves encoding efficiency.

[0047] The decoded picture buffer 170 can store the corrected restored picture for use as a reference picture from the interpretation unit 180.

[0048] Figure 3 shows a schematic block diagram of a decoding device that decodes a video signal, as an embodiment to which the present invention is applied. The decoding device 200 in Figure 3 can correspond to the decoding device 22 in Figure 1.

[0049] Referring to Figure 3, the decoding device 200 may consist of an entropy decoding unit 210, an inverse quantization unit 220, an inverse transformation unit 230, an addition unit 235, a filtering unit 240, a decoded picture buffer (DPB) 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction unit 260 and the intra-prediction unit 265 can be collectively called the prediction unit. That is, the prediction unit may include an inter-prediction unit 180 and an intra-prediction unit 185. The inverse quantization unit 220 and the inverse transformation unit 230 can be collectively called the residual processing unit. That is, the residual processing unit may include an inverse quantization unit 220 and an inverse transformation unit 230. The aforementioned entropy decoding unit 210, inverse quantization unit 220, inverse transformation unit 230, addition unit 235, filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 may be configured by a single hardware component (e.g., a decoder or processor) depending on the embodiment. Furthermore, the decoded picture buffer 250 can be implemented by a single hardware component (e.g., memory or a digital storage medium) depending on the embodiment.

[0050] When a bitstream containing video / image information is input, the decoding device 200 can reconstruct the image in accordance with the process by which the video / image information from the encoding device 100 in Figure 2 has been processed. For example, the decoding device 200 can decode using the processing units applied in the encoding device 100. Therefore, the decoding processing unit may be, for example, a coding unit, which can be divided along a quad-tree structure and / or binary tree structure from a coding tree unit or a maximum coding unit. The reconstructed video signal decoded and output via the decoding device 200 can then be reproduced through a playback device.

[0051] The decoding device 200 can receive the signal output from the encoding device 100 in Figure 2 in bitstream form, and the received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can farce (analyze) the bitstream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). For example, the entropy decoding unit 210 can decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntactic elements necessary for image restoration and the quantized values ​​of conversion coefficients related to residuals. More specifically, the CABAC entropy decoding method receives the bins corresponding to each syntactic element from the bitstream, determines a context model using information about the syntactic element to be decoded, the surrounding and decoded information of the block to be decoded, or the symbol / bin information decoded in the previous step, predicts the probability of bin occurrence based on the determined context model, and performs arithmetic decoding of the bins to generate the symbols corresponding to the values ​​of each syntactic element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the symbol / bin information decoded for the context model of the next symbol / bin. Of the information decoded by the entropy decoding unit 210, information related to predictions is provided by the prediction unit (inter-prediction unit 260 and intra-prediction unit 265), and the residual values ​​that have been entropy decoded from the entropy decoding unit 210, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 220. Furthermore, information related to filtering from the information decoded by the entropy decoding unit 210 can be provided to the filtering unit 240. On the other hand, a receiving unit (not shown) that receives the signal output from the encoding device 100 may be further configured as an internal / external element of the decoding device 200, or the receiving unit may be a component of the entropy decoding unit 210.

[0052] The inverse quantization unit 220 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 220 can rearrange the quantized transformation coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device 100. The inverse quantization unit 220 can perform inverse quantization of the quantized transformation coefficients using quantization parameters (e.g., quantization stage size information) to obtain the transformation coefficients.

[0053] The inverse conversion unit 230 inversely converts the conversion coefficients to obtain a resistive signal (residual block, resistive sample array).

[0054] The prediction unit can perform a prediction on the current block and generate a predicted block that includes the predicted sample of the current block. Based on the prediction information output from the entropy decoding unit 210, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block, and can determine a specific intra / inter-prediction mode.

[0055] The intra-prediction unit 265 can predict the current block by referring to a sample in the current picture. Depending on the prediction mode, the referenced sample can be located in the vicinity (neighbor) of the current block or at a distance. The prediction mode in intra-prediction can include multiple non-directional modes and multiple directional modes. The intra-prediction unit 265 can also determine the prediction mode to be applied to the current block using the prediction mode applied to the surrounding block.

[0056] The interprediction unit 260 can derive the predicted block of the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, to reduce the amount of motion information transferred in interprediction mode, motion information can be predicted in blocks, subblocks, or samples based on the correlation of motion information between surrounding blocks and the current block. Motion information can include motion vectors and reference picture indices. Motion information can further include information on the interprediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In interprediction, surrounding blocks can include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 260 can construct a motion information candidate list based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes, and the prediction information can include information indicating the interprediction mode of the current block.

[0057] The summing unit 235 can generate a restored signal (restored picture, restored block, restored, sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the inter-prediction unit 260 or intra-prediction unit 265. If there is no residual for the block to be processed, as in the case where skip mode is applied, the predicted block is used as the restored block.

[0058] The addition unit 235 can be called the restoration unit or restoration block generation unit. The generated restoration signal is used for intra-prediction of the next block to be processed in the current picture, and can also be used for intra-prediction of the next picture after being filtered as described later.

[0059] The filtering unit 240 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transfer the modified restored picture to the decoded picture buffer 250. Various filtering methods can include, for example, ji-blocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), and bilateral filter.

[0060] The corrected restored picture, transferred to the decoded picture buffer 250, is used as a reference picture by the interpretation unit 260.

[0061] In this specification, the embodiments described for the filtering unit 160, inter-prediction unit 180, and intra-prediction unit 185 of the encoding device 100 can be applied identically or in a corresponding manner to the filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 of the respective decoding devices.

[0062] Figure 4 is a structural diagram of a content streaming system as an embodiment to which the present invention is applied.

[0063] The content streaming system to which the present invention is applied may broadly include an encoding server 410, a streaming server 420, a web server 430, media storage 440, a user device 450, and a multimedia input device 460.

[0064] The encoding server 410 is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and CAM coders into digital data to generate a bitstream, and then transferring this bitstream to the streaming server 420. In other cases, if the multimedia input device 460, such as a smartphone, camera, or CAM coder, directly generates the bitstream, the encoding server 410 can be omitted.

[0065] The bitstream can be generated by an encoding method or a method for generating a bitstream to which the present invention is applied, and the streaming server 420 can temporarily store the bitstream in the process of transferring or receiving the bitstream.

[0066] The streaming server 420 transfers multimedia data to the user device 450 based on a user request via the web server 430, and the web server 430 acts as an intermediary to inform the user of available services. When the user requests a desired service from the web server 430, the web server 430 transmits this to the streaming server 420, and the streaming server 420 transfers the multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands / responses between each device in the content streaming system.

[0067] The streaming server 420 can receive content from the media storage 440 and / or the encoding server 410. For example, if it receives content from the encoding server 410, it can receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server 420 can store the bitstream for a certain period of time.

[0068] Examples of user devices 450 include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (such as smartwatches, smart glasses, and HMDs), digital TVs, desktop computers, and digital signage.

[0069] Each server in a content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

[0070] Figure 5 illustrates embodiments to which the present invention can be applied, with Figure 5a illustrating a block partitioning structure using a Quad Tree (QT), Figure 5b illustrating a Binary Tree (BT), Figure 5c illustrating a Ternary Tree (TT), and Figure 5d illustrating an Asymmetric Tree (AT).

[0071] In video coding, a block can be partitioned based on QT. A subblock partitioned by QT can then be recursively partitioned further using QT. Leaf blocks that are no longer partitioned by QT can be partitioned by at least one of the following methods: BT, TT, or AT. BT can have two forms of partitioning: horizontal BT (2NxN, 2NxN) and vertical BT (Nx2N, Nx2N). TT can have two forms of partitioning: horizontal TT (2Nx1 / 2N, 2NxN, 2Nx1 / 2N) and vertical TT (1 / 2Nx2N, Nx2N, 1 / 2Nx2N). AT can have four forms of partitioning: horizontal-up AT (2Nx1 / 2N, 2Nx3 / 2N), horizontal-down AT (2Nx3 / 2N, 2Nx1 / 2N), vertical-left AT (1 / 2Nx2N, 3 / 2Nx2N), and vertical-right AT (3 / 2Nx2N, 1 / 2Nx2N). Each of BT, TT, and AT can be further recursively partitioned using BT, TT, and AT.

[0072] Figure 5a shows an example of QT partitioning. Block A can be partitioned into four subblocks (A0, A1, A2, A3) by QT. Subblock A1 can again be partitioned into four subblocks (B0, B1, B2, B3) by QT.

[0073] Figure 5b shows an example of BT partitioning. Block B3, which is no longer partitioned by QT, can be partitioned into vertical BT(C0, C1) or horizontal BT(D0, D1). Like block C0, each subblock can be recursively partitioned further into the form of horizontal BT(E0, E1) or vertical BT(F0, F1).

[0074] Figure 5c shows an example of TT partitioning. Block B3, which is no longer partitioned by QT, can be partitioned into a vertical TT (C0, C1, C2) or a horizontal TT (D0, D1, D2). Like block C1, each subblock can be recursively partitioned further into the form of a horizontal TT (E0, E1, E2) or a vertical TT (F0, F1, F2).

[0075] Figure 5d shows an example of AT partitioning. Block B3, which is no longer partitioned by QT, can be partitioned into vertical AT(C0, C1) or horizontal AT(D0, D1). Like block C1, each subblock can be recursively partitioned further into the form of horizontal AT(E0, E1) or vertical TT(F0, F1).

[0076] On the other hand, BT, TT, and AT partitions can be used together for partitioning. For example, a subblock partitioned by BT can be partitioned by TT or AT. Similarly, a subblock partitioned by TT can be partitioned by BT or AT. A subblock partitioned by AT can be partitioned by BT or TT. For example, after a horizontal BT partition, each subblock can be partitioned into vertical BTs, or after a vertical BT partition, each subblock can be partitioned into horizontal BTs. In this case, although the order of partitioning is different, the final partitioned form is the same.

[0077] Furthermore, once a block is divided, the order in which the block is searched can be defined in various ways. Generally, searching a block from left to right and from top to bottom can mean the order in which additional blocks are decided for each divided subblock, or, if the block is no longer divided, the coding order of each subblock, or the search order when a subblock refers to information from other adjacent blocks.

[0078] Transformations can be performed on each processing unit (or transformation block) divided by a partition structure as shown in Figures 5a to 5d, and in particular, the division can be separated by row and column directions, and a transformation matrix can be applied. According to an embodiment of the present invention, other transformation types are used depending on the row or column length of the processing unit (or transformation block).

[0079] The transformation is applied to the residual block, which is done to decorrelate the residual block as much as possible, concentrate the coefficients at low frequencies, and create a zero tail at the top of the block. In JEM software, the transformation part includes two main functions: core transform and secondary transform. The core transform consists of the DCT (discrete cosine transform) and DST (discrete sine transform) family of transforms, which are applied to all rows and columns of the residual block. Subsequently, the secondary transform can be applied to the top left corner of the output of the core transform. Similarly, the inverse quadratic transform and the inverse core transform can be applied in that order. First, the inverse quadratic transform can be applied to the top left corner of the coefficient block. Subsequently, the inverse core transform is applied to the rows and columns of the output of the inverse quadratic transform. The core transform or inverse transform can be referred to as the primary transform or inverse transform.

[0080] Figures 6 and 7 show embodiments to which the present invention is applied. Figure 6 shows a schematic block diagram of the conversion and quantization unit (120 / 130) and the inverse quantization and inverse conversion unit (140 / 150) within the encoding device 100 of Figure 2, and Figure 7 shows a schematic block diagram of the inverse quantization and inverse conversion unit (220 / 230) within the decoding device 200.

[0081] Careful examination of Figure 6 reveals that the transformation and quantization unit (120 / 130) can include a primary transform unit 121, a secondary transform unit 122, and a quantization unit 130. The inverse quantization and inverse transformation unit (140 / 150) can include an inverse quantization unit 140, an inverse secondary transform unit 151, and an inverse primary transform unit 152.

[0082] A careful examination of Figure 7 reveals that the inverse quantization and inverse transformation unit (220 / 230) can include an inverse quantization unit 220, an inverse secondary transform unit 231, and an inverse primary transform unit 232.

[0083] In this invention, when performing a transformation, the transformation can be carried out through multiple stages. For example, as shown in Figure 6, two stages, a primary transform and a secondary transform, can be applied, or more transformation stages may be used based on the algorithm. Here, the primary transform can be referred to as the core transform.

[0084] The primary conversion unit 121 can apply a primary conversion to the residual signal, where the primary conversion can be predetermined from the encoder and / or decoder into a table.

[0085] The secondary conversion unit 122 can apply a secondary conversion to the primary converted signal, where the secondary conversion can be predetermined in a table from the encoder and / or decoder.

[0086] In one embodiment, a non-separable secondary transform (NSST) can be conditionally applied as the secondary transform. For example, the NSST may be applied only to in-screen prediction blocks, and there may be a set of applicable transforms for each prediction mode group.

[0087] Here, prediction mode groups can be set based on the symmetry of the prediction direction. For example, prediction mode 52 and prediction mode 16 are symmetrical based on prediction mode 34 (diagonal direction), so they can form a single group and the same transform set can be applied. In this case, when applying the transform for prediction mode 52, it is applied after transposing the input data, because the transform set is the same as that for prediction mode 16.

[0088] On the other hand, in Planar mode and DC mode, there is no directional symmetry, so each has its own set of transformations, and each transformation set can consist of two transformations. For the remaining directional modes, each transformation set can consist of three transformations.

[0089] The quantization unit 130 can perform quantization on the quadratic-converted signal.

[0090] The inverse quantization and inverse transformation section (140 / 150) is performed by reversing the process described earlier, and redundant explanations are omitted.

[0091] Figure 7 shows a schematic block diagram of the inverse quantization and inverse transformation section (220 / 230) within the decoding device 200.

[0092] Referring to Figure 7, the inverse quantization and inverse transform unit (220 / 230) can include an inverse quantization unit 220, an inverse secondary transform unit 231, and an inverse primary transform unit 232.

[0093] The inverse quantization unit 220 uses quantization stage size information to obtain conversion coefficients from the entropy-decoded (decoded) signal.

[0094] In the inverse quadratic transformation unit 231, an inverse quadratic transformation is performed on the transformation coefficients. Here, the inverse quadratic transformation is the inverse transformation of the quadratic transformation explained in Figure 6.

[0095] The inverse linear transformation unit 232 performs an inverse linear transformation on the inverse quadratic transformed signal (or block) to obtain a residual signal. Here, the inverse linear transformation is the inverse transformation of the linear transformation (primary transform) described in Figure 6.

[0096] In addition to DCT-2 and 4x4 DST-4 applied to HEVC, adaptive (or explicit) multiple transform (AMT or EMT) techniques are used for residual coding of inter- and intra-encoded blocks. A number of selected transforms from HEVC and other DCT / DST families are used. Newly introduced transform matrices from JEM are DST-7, DCT-8, DST-1, and DCT-5. Table 1 below shows the basis functions for the selected DST / DCTs.

[0097] [Table 1]

[0098] EMT can be applied to CUs smaller than 64 or with the same width and height, and whether EMT is applied can be controlled by a CU level flag. If the CU level flag is 0, DCT-2 is applied to the CU to encode the residual. For intra-residual coding blocks within a CU to which EMT is applied, two additional flags are signaled to identify horizontal and vertical transformations. Like HEVC, JEM blocks can be coded in transformation skip mode. For intra-residual coding, a mode-dependent transformation candidate selection process is used, along with other residual statistics for other intra-prediction modes. Three transformation subsets are defined as shown in Table 2 below, and the transformation subset is selected based on the intra-prediction mode as shown in Table 3.

[0099] [Table 2]

[0100] Along with the subset concept, a subset of transformations is initially identified based on Table 2 by using the intra-prediction mode for CUs where the CU-level EMT_CU_flag is 1. Thereafter, for each of the horizontal (EMT_TU_horizontal_flag) and vertical (EMT_TU_vertical_flag) transformations, one of two transformation candidates within the identified subset of transformations is selected based on explicit signaling using flags, as shown in Table 3.

[0101] [Table 3]

[0102] [Table 4]

[0103] Table 4 shows a transform configuration group to which AMT (adaptive multiple transform) is applied, as an embodiment to which the present invention is applied.

[0104] A careful examination of Table 4 reveals that the transform configuration groups are determined based on the prediction mode, and there can be a total of six groups (G0 to G5). G0 to G4 apply when intra-prediction is applied, while G5 represents the combination of transformations (or transformation sets, transformation combination sets) applied to residual blocks generated by inter-prediction.

[0105] A single transformation can be achieved by applying a horizontal transform (or row transform) to the rows of the 2D block and a vertical transform (or column transform) to the columns.

[0106] Here, every group of conversion settings can have four candidate conversion combinations. The four candidate conversion combinations can be selected or determined via conversion combination indices from 0 to 3, and the conversion combination indices can be encoded and transferred from the encoder to the decoder.

[0107] In one embodiment, the residual data (or residual signal) obtained via intra-prediction may have different statistical characteristics depending on the intra-prediction mode. Therefore, as shown in Table 4, a different transformation may be applied to each intra-prediction instead of the general cosine transform. In this specification, the transformation types may be denoted, for example, as DCT-Type 2, DCT-II, and DCT-2.

[0108] Table 4, upon closer examination, reveals two cases: one using 35 intra-prediction modes and another using 67 intra-prediction modes. Multiple transformation combinations can be applied to each transformation setting group, separated by the columns for each intra-prediction mode. For example, multiple transformation combinations can consist of four combinations (row-direction transformation, column-direction transformation). Specifically, in group 0, DST-7 and DCT-5 can be applied to both the row (horizontal) and column (vertical) directions, resulting in a total of four possible combinations.

[0109] For each intra-prediction mode, a total of four transformation kernel combinations can be applied, and an index of the transformation combination for selecting one of these can be transferred for each transformation unit. In this specification, the index of the transformation combination may be called the AMT index and may be represented by amt_idx.

[0110] Furthermore, in addition to the transformation kernels presented in Table 4, there may be cases where DCT-2 is optimal for all row and column directions due to the characteristics of the residual signal. Therefore, by defining an AMT flag for each coding unit, the transformation can be applied adaptively. Here, if the AMT flag is 0, DCT-2 is applied for all row and column directions, and if the AMT flag is 1, one of four combinations can be selected or determined using the AMT index.

[0111] In one embodiment, if the AMT flag is 0, and the number of conversion coefficients for one conversion unit is less than 3, the conversion kernel in Table 4 is not applied, and DST-7 can be applied to all rows and columns.

[0112] In one embodiment, the values ​​of the conversion coefficients are parsed (analyzed) first, and if the number of conversion coefficients is less than 3, the AMT index is not analyzed, and the amount of additional information transferred can be reduced by applying DST-7.

[0113] In one embodiment, the AMT can be applied only when the width and height of the conversion unit are both 32 or less.

[0114] In one embodiment, Table 4 can be set based on offline training.

[0115] In one embodiment, the AMT index can be defined as a single index that can simultaneously refer to a combination of horizontal and vertical transformations. Alternatively, the AMT index can be defined by separate horizontal and vertical transformation indices.

[0116] Figure 8 is a flowchart showing the process by which AMT (adaptive multiple transform) is performed.

[0117] In this specification, embodiments of separable transforms, in which the transformation is applied separately to the horizontal and vertical directions, are described in principle, but the combination of transformations can also be configured as non-separable transforms.

[0118] Alternatively, combinations of transformations can be constructed using a mixture of separable and inseparable transformations. In this case, when inseparable transformations are used, row / column-specific transformation selection and horizontal / vertical-specific selection become unnecessary, and the transformation combinations in Table 4 can only be used when separable transformations are selected.

[0119] Furthermore, the methods proposed herein can be applied regardless of whether the transformation is linear or quadratic. In other words, there is no constraint that it must be applied to only one of the two, but it can be applied to both. Here, a linear transformation can mean a transformation for initially transforming the residual block, and a quadratic transformation can mean a transformation for applying the transformation to the block produced as a result of the linear transformation.

[0120] First, the encoding device 100 can determine the conversion group corresponding to the current block (S805). Here, the conversion group may refer to the conversion groups in Table 4, but the present invention is not limited thereto and may consist of other combinations of conversions.

[0121] The encoding device 100 can perform transformations on candidate transformation combinations used within the transformation group (S810). Based on the transformation results, the encoding device 100 can determine or select the transformation combination with the minimum RD (rate distortion) cost (S815). The encoding device 100 can encode the index of the transformation combination corresponding to the selected transformation combination (S820).

[0122] Figure 9 is a flowchart showing the decoding process in which AMT is performed.

[0123] First, the decoding device 200 can determine the transformation group for the current block (S905). The decoding device 200 can analyze the index of the transformation combination, where the index of the transformation combination can correspond to any one of the multiple transformation combinations in the transformation group (S910). The decoding device 200 can derive the transformation combination corresponding to the index of the transformation combination (S915). Here, the transformation combination can mean the transformation combinations described in Table 4, but the present invention is not limited thereto. In other words, configurations using other transformation combinations are also possible.

[0124] The decoding device 200 can perform an inverse transformation on the current block based on the transformation combination (S920). If the transformation combination consists of a row transformation and a column transformation, the row transformation can be applied first, followed by the column transformation. However, the present invention is not limited thereto, and the transformations can be applied in reverse, or if it consists of non-separable transformations, the non-separable transformation can be applied directly.

[0125] On the other hand, in another embodiment, the process of determining the transformation group and the process of analyzing the index of the transformation combination can be performed simultaneously.

[0126] In embodiments of the present invention, the aforementioned term "AMT" can be redefined as "MTS (multiple transform set or multiple transform selection)". The MTS-related syntax and semantics described below are defined in the VVC (versatile video coding) standard document JVET-K1001-v4.

[0127] In an embodiment of the present invention, two MTS candidates are used for the directional mode and four MTS candidates are used for the non-directional mode, as described below.

[0128] A) Non-directional mode (DC, Planner)

[0129] When the MTS index is 0, DST-7 is used for horizontal and vertical conversion.

[0130] When the MTS index is 1, DST-7 is used for vertical transformation and DCT-8 is used for horizontal transformation.

[0131] When the MTS index is 2, DCT-8 is used for vertical transformation and DST-7 is used for horizontal transformation.

[0132] When the MTS index is 3, DCT-8 is used for horizontal and vertical conversion.

[0133] B) Modes belonging to horizontal group mode

[0134] When the MTS index is 0, DST-7 is used for horizontal and vertical conversion.

[0135] When the MTS index is 1, DCT-8 is used for vertical transformation and DST-7 is used for horizontal transformation.

[0136] C) Modes belonging to vertical group mode

[0137] When the MTS index is 0, DST-7 is used for horizontal and vertical conversion.

[0138] When the MTS index is 1, DST-7 is used for vertical transformation and DCT-8 is used for horizontal transformation.

[0139] Here (in VTM 2.0, which uses 67 modes), the horizontal group mode includes intra-prediction modes 2 through 34, and the vertical mode includes intra-prediction modes 35 through 66.

[0140] In another embodiment of the present invention, three MTS candidates are used for all intra-prediction modes.

[0141] When the MTS index is 0, DST-7 is used for horizontal and vertical conversion.

[0142] When the MTS index is 1, DST-7 is used for vertical transformation and DCT-8 is used for horizontal transformation.

[0143] When the MTS index is 2, DCT-8 is used for vertical transformation and DST-7 is used for horizontal transformation.

[0144] In another embodiment of the present invention, two MTS candidates are used for the direction prediction mode and three MTS candidates are used for the non-direction prediction mode.

[0145] A) Non-directional mode (DC, Planner)

[0146] When the MTS index is 0, DST-7 is used for horizontal and vertical conversion.

[0147] When the MTS index is 1, DST-7 is used for vertical transformation and DCT-8 is used for horizontal transformation.

[0148] When the MTS index is 2, DCT-8 is used for vertical transformation and DST-7 is used for horizontal transformation.

[0149] B) Prediction mode for horizontal group mode

[0150] When the MTS index is 0, DST-7 is used for horizontal and vertical conversion.

[0151] When the MTS index is 1, DCT-8 is used for vertical transformation and DST-7 is used for horizontal transformation.

[0152] C) Prediction mode applicable to the vertical group mode

[0153] When the MTS index is 0, DST-7 is used for horizontal and vertical conversion.

[0154] When the MTS index is 1, DST-7 is used for vertical transformation and DCT-8 is used for horizontal transformation.

[0155] In another embodiment of the present invention, one MTS candidate (e.g., DST-7) is used for all intra-modes. In this case, encoding time can be reduced by up to 40%, along with minor coding loss. Furthermore, one flag is used to indicate between DCT-2 and DST-7.

[0156] Figure 10 is a flowchart showing the inverse transformation process based on MTS according to an embodiment of the present invention.

[0157] A decoding device 200 to which the present invention is applied can obtain sps_mts_intra_enabled_flag or sps_mts_inter_enabled_flag (S1005). Here, sps_mts_intra_enabled_flag indicates whether cu_mts_flag exists in the residual coding syntax of the intracoding unit. For example, if sps_mts_intra_enabled_flag = 0, then cu_mts_flag does not exist in the residual coding syntax of the intracoding unit, and if sps_mts_intra_enabled_flag = 1, then cu_mts_flag exists in the residual coding syntax of the intracoding unit. And sps_mts_inter_enabled_flag indicates whether cu_mts_flag exists in the residual coding syntax of the intercoding unit. For example, if sps_mts_inter_enabled_flag = 0, then cu_mts_flag does not exist in the recurrent coding syntax of the intercoding unit, and if sps_mts_inter_enabled_flag = 1, then cu_mts_flag exists in the recurrent coding syntax of the intercoding unit.

[0158] The decoding device 200 can obtain cu_mts_flag based on sps_mts_intra_enabled_flag or sps_mts_inter_enabled_flag (S1010). For example, when sps_mts_intra_enabled_flag = 1 or sps_mts_inter_enabled_flag = 1, the decoding device 200 can obtain cu_mts_flag. Here, cu_mts_flag indicates whether the MTS is applied to the residual samples of the Luma conversion block. For example, if cu_mts_flag = 0, the MTS is not applied to the residual samples of the Luma conversion block, and if cu_mts_flag = 1, the MTS is applied to the residual samples of the Luma conversion block.

[0159] The decoding device 200 can obtain mts_idx based on cu_mts_flag (S1015). For example, when cu_mts_flag = 1, the decoding device 200 can obtain mts_idx, where mts_idx indicates which translation kernel is currently applied to the lumaregid dual samples along the horizontal and / or vertical directions of the translation block.

[0160] For example, with respect to mts_idx, at least one of the embodiments described herein can be applied.

[0161] The decoding device 200 can induce a conversion kernel corresponding to mts_idx (S1020). For example, the conversion kernel corresponding to mts_idx can be defined by dividing it into horizontal conversion and vertical conversion.

[0162] For example, if MTS is currently applied to a block (i.e., cu_mts_flag = 1), the decoding device 200 can configure MTS candidates based on the intra-prediction mode of the current block. In this case, the decoding flowchart in Figure 10 may further include a step for configuring MTS candidates. The decoding device 200 can then use mts_idx to determine which MTS candidate is applied to the current block from among the configured MTS candidates.

[0163] As another example, horizontal and vertical transformations may be applied to each other using different transformation kernels. However, the present invention is not limited thereto, and the same transformation kernel may also be applied to the horizontal and vertical transformations.

[0164] The decoding device 200 can then perform the inverse conversion based on the conversion kernel (S1025).

[0165] Furthermore, in this document, MTS can also be expressed as AMT or EMT, and similarly, it can be expressed as mts_idx, AMT_idx, EMT_idx, AMT_TU_idx, EMT_TU_idx, etc., and the present invention is not limited to such expressions.

[0166] Figure 11 is a block diagram of an apparatus that performs decoding based on an MTS according to an embodiment of the present invention.

[0167] The decoding device 200 to which the present invention is applied may include a sequence parameter acquisition unit 1105, an MTS flag acquisition unit 1110, an MTS index acquisition unit 1115, and a conversion kernel induction unit 1120.

[0168] The sequence parameter acquisition unit 1105 can acquire sps_mts_intra_enabled_flag or sps_mts_inter_enabled_flag. Here, sps_mts_intra_enabled_flag indicates whether cu_mts_flag exists in the residual coding syntax of the intracoding unit, and sps_mts_inter_enabled_flag indicates whether cu_mts_flag exists in the residual coding syntax of the intercoding unit. A specific example can be applied to the explanation related to Figure 105.

[0169] The MTS flag acquisition unit 1110 can acquire cu_mts_flag based on sps_mts_intra_enabled_flag or sps_mts_inter_enabled_flag. For example, when sps_mts_intra_enabled_flag = 1 or sps_mts_inter_enabled_flag = 1, the MTS flag acquisition unit 1110 can acquire cu_mts_flag. Here, cu_mts_flag indicates whether MTS is applied to the residual sample of the Luma transform block. A specific example can be applied to the explanation related to Figure 10.

[0170] The MTS index acquisition unit 1115 can acquire mts_idx based on cu_mts_flag. For example, when cu_mts_flag = 1, the MTS index acquisition unit 1115 can acquire mts_idx. Here, mts_idx indicates which translation kernel is applied to the Lumarezid dual samples along the horizontal and / or vertical directions of the current translation block. A specific example can be found in the explanation in Figure 10.

[0171] The conversion kernel induction unit 1120 can induce a conversion kernel corresponding to mts_idx. Then, the decoding device 200 can perform the inverse conversion based on the induced conversion kernel.

[0172] The mode-dependent non-separable secondary transform (MDNSST) is introduced. To maintain low complexity, the MDNSST is applied only to low-frequency coefficients after the linear transform. A non-separable transform applied primarily to low-frequency coefficients can also be called the LFNST (low-frequency non-separable transform). If both the width (W) and height (H) of the transform coefficient block are 8 or greater, an 8x8 non-separable secondary transform is applied to the upper left 8x8 region of the transform coefficient block. Otherwise, if either the width or height is less than 8, a 4x4 non-separable secondary transform is applied, and the 4x4 non-separable secondary transform can be performed on the upper left min(8,W) x min(8,H) of the transform coefficient block. Here, min(A,B) is a function that outputs the smaller of A and B. Also, WxH represents the size of the block, where W is the width and H is the height.

[0173] The RT applied to the top-left 8x8 block of the block of transformation coefficients to which a linear transformation has been applied can be called an 8x8 RST. In mathematical formula 3, when the value of R is set to 16, the forward 8x8 RST has the form of a 16x64 matrix, and the reverse 8x8 RST has the form of a 64x16 matrix. Furthermore, the transformation set configurations shown in Table 5 can also be applied to the 8x8 RST. In other words, the 8x8 RST can be determined based on the transformation set corresponding to the intra-prediction mode, as shown in Table 5. Since one transformation set consists of two or three transformations depending on the intra-prediction mode, one of up to four transformations can be selected, including the case where no quadratic transformation is applied (one transformation can correspond to the identity matrix). When the four transformations are assigned the indices 0, 1, 2, and 3 respectively, the transformation to be applied can be specified by signaling the syntax element corresponding to the NSST index for each block of transformation coefficients. For example, index 0 can be assigned to the identity matrix, i.e., when no quadratic transformation is applied. In conclusion, for an 8x8 top-left block via NSST indexing, according to JEM NSST, an 8x8 NSST can be specified, and according to RST configuration, an 8x8 RST can be specified.

[0174] [Table 5]

[0175] Within the translation set, an NSST index (NSST idx) can be coded to indicate the translation kernel. If NSST is not applied, an NSST index with a value of 0 can be signaled.

[0176] Figures 12 and 13 show an encoding / decoding flowchart to which a quadratic transformation is applied, as an embodiment to which the present invention is applied.

[0177] In JEM, the quadratic transformation (MDNSST) is not applied to blocks coded in transformation-skip mode. If the MDNSST index is signaled to the CU and is not zero, MDNSST is not used for blocks of components coded in transformation-skip mode within the CU. The overall coding structure, including coefficient encoding and NSST index coding, is shown in Figures 12 and 13. The CBF (coded block flag) is encoded to determine whether coefficient encoding and NSST coding are performed. In Figures 12 and 13, the CBF flag can indicate a luma-block cbf flag (cbf_luma flag) or a chroma-block cbf flag (cbf_cb flag or cbf_cr flag). When the CBF flag is 1, the transformation coefficients are coded.

[0178] Referring to Figure 12, the encoding device 100 checks whether CBF is 1 (S1205). If CBF is 0, the encoding device 100 does not perform encoding of the conversion coefficients or encoding of the NSST index. If CBF is 1, the encoding device 100 performs encoding of the conversion coefficients (S1210). Thereafter, the encoding device 100 decides whether to perform NSST index coding (S1215) and performs NSST index coding (S1220). If NSST index coding is not applied, the encoding device 100 terminates the conversion procedure in a state where NSST is not applied and can perform subsequent stages (e.g., quantization).

[0179] Referring to Figure 13, the decoding device 200 checks whether CBF is 1 (S1305). If CBF is 0, the decoding device 200 does not decode the conversion coefficients or the NSST index. If CBF is 1, the decoding device 200 decodes the conversion coefficients (S1310). Subsequently, the decoding device 200 decides whether to code the NSST index (S1315) and analyzes the NSST index (S1320).

[0180] NSST is not applied to the entire block (TU in the case of HEVC) to which the linear transformation is applied, but rather to the upper left 8x8 or 4x4 region. For example, if the block size is 8x8 or larger, the 8x8 NSST is applied, and if it is less than 8x8, the 4x4 NSST is applied. Also, when the 8x8 NSST is applied, the 4x4 NSST can be applied to every 4x4 block. Both the 8x8 NSST and the 4x4 NSST can be determined according to the configuration of the transformation set described above. As they are both non-separable transformations, the 8x8 NSST has 64 input data and 64 output data, while the 4x4 NSST can have 16 inputs and 16 outputs.

[0181] Figures 14 and 15 show embodiments to which the present invention is applied. Figure 14 illustrates a Givens rotation, and Figure 15 shows the configuration of one round in a 4x4 NSST consisting of a Givens rotation layer and permutation.

[0182] Both 8x8 NSST and 4x4 NSST can be constructed as hierarchical combinations of Givens rotations. The matrix corresponding to one Givens rotation is the same as in mathematical formula 1, and the matrix product is represented graphically as in Figure 14.

[0183]

number

[0184] In Figure 14, the output t due to Givens rotation is m and t n This can be calculated as shown in mathematical formula 2.

[0185]

number

[0186] As shown in Figure 14, one Givens rotation rotates two data points, so 32 Givens rotations are required to process 64 data points (for 8x8 NSST) or 16 data points (for 4x4 NSST), respectively, or 8 Givens rotations. Therefore, a bundle of 32 or 8 Givens rotations can form a Givens rotation layer. As shown in Figure 15, the output data of one Givens rotation layer is transmitted to the input data of the next Givens rotation layer via shuffling. As shown in Figure 15, the patterns to be shuffled are defined regularly, and in the case of 4x4 NSST, four Givens rotation layers and their corresponding shuffles form one round. 4x4 NSST is performed in two rounds, and 8x8 NSST is performed in four rounds. Each round uses the same shuffling pattern, but the Givens rotation angles applied are different. Therefore, it is necessary to store the angle data of all Givens rotations that constitute each transformation.

[0187] In the final stage, the data output after passing through the Givens rotation layer undergoes one more substitution, and the information of this substitution is stored separately for each transformation. This substitution is performed at the end of the forward NSST, and in the inverse NSST, this inverse substitution is applied first.

[0188] Reverse NSST performs the Givens rotation layer applied in forward NSST in the reverse order of substitution, and rotates by taking a negative (-) value for each Givens rotation angle.

[0189] RST(Reduced secondary transform)

[0190] FIG. 16 shows the operation of RST as an embodiment to which the present invention is applied.

[0191] Assuming that an orthogonal matrix representing a single transform has a form of N×N, RT (reduced transform) leaves only R out of N transform basis vectors (R < N). The matrix of the forward RT for generating transform coefficients can be defined as in Mathematical Formula 3.

[0192]

Equation

[0193] Since the matrix of the inverse RT is the transpose matrix of the forward RT matrix, if the application of the forward RT and the inverse RT is illustrated, it can be the same as FIGS. 14a and 14b.

[0194] The RT applied to the top-left 8x8 block of the block of transformation coefficients to which a linear transformation has been applied can be called an 8x8 RST. In mathematical formula 3, when the value of R is set to 16, the forward 8x8 RST has the form of a 16x64 matrix, and the reverse 8x8 RST has the form of a 64x16 matrix. Furthermore, the transformation set configurations shown in Table 5 can also be applied to the 8x8 RST. In other words, the 8x8 RST can be determined based on the transformation set corresponding to the intra-prediction mode, as shown in Table 5. Since one transformation set consists of two or three transformations depending on the intra-prediction mode, one of up to four transformations can be selected, including the case where no quadratic transformation is applied (one transformation can correspond to the identity matrix). When the four transformations are assigned the indices 0, 1, 2, and 3 respectively, the transformation to be applied can be specified by signaling the syntax element corresponding to the NSST index for each block of transformation coefficients. For example, index 0 can be assigned to the identity matrix, i.e., when no quadratic transformation is applied. In conclusion, for an 8x8 top-left block via NSST indexing, according to JEM NSST, an 8x8 NSST can be specified, and according to RST configuration, an 8x8 RST can be specified.

[0195] Figure 17 shows the process of performing reverse scans from the 64th to the 17th based on the reverse scan order, as an embodiment to which the present invention is applied.

[0196] When a forward 8x8 RST like the one in mathematical formula 3 is applied, 16 effective transformation coefficients are generated. As a result, the 64 input data points that make up the 8x8 region are reduced to 16 output data points. From the perspective of a two-dimensional region, only about 1 / 4 of the region is filled with effective transformation coefficients. Therefore, by applying the forward 8x8 RST, the 16 output data points obtained fill the region at the top left of Figure 17.

[0197] In Figure 17, the 4x4 area in the upper left corner becomes the ROI (region of interest) area where valid conversion coefficients are satisfied, and the remaining area is left empty. The empty area can be filled with a value of 0 by default. If a non-zero valid conversion coefficient is found outside the ROI area in Figure 17, it is certain that the 8x8 RST will not be applied, and the corresponding coding may be omitted for the NSST index. Conversely, if no non-zero conversion coefficient is found outside the ROI area in Figure 17 (when the area outside the ROI is filled with 0 if the 8x8 RST is applied), the NSST index can be coded because the 8x8 RST may have been applied in the past. This conditional NSST index coding requires checking for the presence or absence of non-zero conversion coefficients, so it can be performed after the residual coding process.

[0198] Figure 18 shows an example of an encoding flowchart using a single transform indicator as an embodiment to which the present invention is applied.

[0199] In embodiments of the present invention, a single transform indicator (STI) is introduced. Instead of sequentially using two transforms (a linear transform and a quadratic transform), a single transform can be applied when the single transform indicator is activated (STI coding == 1). Here, the single transform can be any type of transform. For example, the single transform can be a separable transform or an inseparable transform. The single transform can be a transform approximated from an inseparable transform. A single transform index (ST_idx in Figure 18) can be signaled when the single transform indicator is activated. Here, the single transform index can indicate the transform to be applied from among the available transform candidates.

[0200] Referring to Figure 18, the encoding device 100 determines whether CBF is 1 (S1805). If CBF is 1, the encoding device 100 determines whether STI coding is applied (S1810). If STI coding is applied, the encoding device 100 encodes the STI index (STI_Idx) (S1845) and codes the conversion coefficients (S1850). If STI coding is not applied, the encoding device 100 encodes a flag (EMT_CU_Flag) indicating whether EMT (or MTS) is applied at the CU level (S1815). Thereafter, the encoding device 100 codes the conversion coefficients (S1820). Thereafter, the encoding device 100 determines whether EMT is applied to the conversion unit (TU) (S1825). If EMT is applied to the TU, the encoding device 100 encodes the index of the primary conversion applied to the TU (EMT_TU Idx) (S1830). Thereafter, the encoding device 100 determines whether NSST is applicable (S1835). If NSST is applicable, the encoding device 100 encodes an index (NSST_Idx) that indicates the applicable NSST (S1840).

[0201] In one example, when the conditions for single transformation coding are met / activated (e.g., STI_coding == 1), a single transformation index (ST_Idx) can be implicitly induced without signaling. ST_Idx can be implicitly determined based on the block size and intra-prediction mode, where ST_Idx can indicate the transformation (or transformation kernel) to be applied to the current transformation block.

[0202] A single translation indicator can be activated if one or more of the following conditions are met (STI_coding == 1):

[0203] The block size corresponds to a predetermined value, such as 4 or 8.

[0204] Block width == Block height (square block)

[0205] It is one of the predetermined intra-prediction modes, such as DC or Planner.

[0206] In another example, an STI coding flag can be signaled to indicate whether a single transformation is applicable. The STI coding flag can be signaled based on the STI coding value and CBF. For example, the STI coding flag can be signaled when the CBF is 1 and STI coding is activated. Furthermore, the STI coding flag can be conditionally signaled considering the block size, block shape (square or non-square blocks), or intra-prediction mode.

[0207] During coefficient coding, the acquired information can be used to determine ST_idx after coefficient coding. In one example, ST_idx can be implicitly determined based on the block size, intra-prediction mode, and the number of non-zero coefficients. In another example, ST_idx can be conditionally encoded / decoded based on the block size and / or block shape and / or intra-prediction mode and / or the number of non-zero coefficients. In yet another example, ST_idx signaling can be omitted depending on the distribution of non-zero coefficients (i.e., the location of the non-zero coefficients). In particular, if non-zero coefficients are found in a region other than the upper-left 4x4 area, ST_idx signaling can be omitted.

[0208] Figure 19 shows an example of an encoding flowchart using a unified transform indicator (UTI) as an embodiment to which the present invention is applied.

[0209] In embodiments of the present invention, a unified conversion indicator is introduced. The UTI encompasses both a primary conversion indicator and a secondary conversion indicator.

[0210] Referring to Figure 19, the encoding device 100 determines whether CBF is 1 (S1905). If CBF is 1, the encoding device 100 determines whether UTI coding is applied (S1910). If UTI coding is applied, the encoding device 100 encodes the UTI index (UTI_Idx) (S1945) and codes the conversion coefficients (S1950). If UTI coding is not applied, the encoding device 100 encodes a flag (EMT_CU_Flag) indicating whether EMT (or MTS) is applied at the CU level (S1915). Thereafter, the encoding device 100 encodes the conversion coefficients (S1920). Thereafter, the encoding device 100 determines whether EMT is applied to the conversion unit (TU) (S1925). If EMT is applied to TU, the encoding device 100 encodes the index of the primary transformation applied to TU (EMT_TU Idx) (S1930). Thereafter, the encoding device 100 determines whether NSST is applied (S1935). If NSST is applied, the encoding device 100 encodes the index indicating the NSST to be applied (NSST_Idx) (S1940).

[0211] UTI can be encoded for each predetermined unit (CTU or CU).

[0212] The UTI coding mode can depend on the following conditions.

[0213] Block size

[0214] Block form

[0215] Intra Predictive Mode

[0216] The method for deriving / extracting core transformation indices from UTIs is predefined. The method for deriving / extracting secondary transformation indices from UTIs is also predefined.

[0217] The syntactic structure of a UTI is used selectively. A UTI can depend on the CU (or TU) size. For example, a smaller CU (TU) may have a relatively narrower range of UTI indices. For instance, if a predefined condition (e.g., block size is smaller than a predefined threshold) is met, a UTI can only point to core translation indices.

[0218] [Table 6]

[0219] In another example, if it is not indicated that a quadratic transformation should be used (for example, if the quadratic transformation index is 0 or the quadratic transformation has already been determined), the UTI index can be treated as a core transformation index. Similarly, if the core transformation index is known, the UTI index can be treated as a quadratic transformation index. In particular, a predetermined core transformation can be used, taking into account the intra-prediction mode and block size.

[0220] Figures 20a and 20b show another example of an encoding flowchart using a UTI as an embodiment to which the present invention is applied.

[0221] In another example, the conversion encoding structure uses UTI index coding, as shown in Figures 20a and 20b. Here, the UTI index can be encoded before or after coefficient encoding.

[0222] Referring to Figure 20a, the encoding device 100 checks whether CBF is 1 (S2005). If CBF is 1, the encoding device 100 codes the UTI index (UTI_Idx) (S2010) and codes the conversion coefficients (S2015).

[0223] Referring to Figure 20b, the encoding device 100 checks whether CBF is 1 (S2055). If CBF is 1, the encoding device 100 performs coding of the conversion coefficients (S2060) and codes the UTI index (UTI_Idx) (S2065).

[0224] In another embodiment of the present invention, data hiding and implicit coding methods for transformation indicators are introduced. Here, the transformation indicator includes ST_idx, UTI_idx, EMT_CU_Flag, EMT_TU_Flag, NSST_idx, and transformation-related indices used to indicate the transformation kernel. The aforementioned transformation indicators are not signaled, and the information can be inserted into (or extracted from) the coefficient encoding process. The coefficient encoding process may include the following parts:

[0225] - Last x position (Last_position_x), Last y position (Last_position_y)

[0226] - Group flag

[0227] - Significance map

[0228] - A flag indicating whether the value is greater than 1 (Greater_than_1_flag)

[0229] - A flag indicating whether the value is greater than 2 (Greater_than_2_flag)

[0230] - Remaining level coding

[0231] - Sign coding

[0232] For example, conversion indicator information can be inserted into one or more of the coefficient coding processes described above. The following can be considered together when inserting conversion indicator information:

[0233] - Sign coding patterns

[0234] - The absolute value of the remaining level

[0235] The number of flags indicating whether the value is greater than -1 (The number of Greater_than_1_flag)

[0236] - The values ​​of the last x-position and last y-position.

[0237] The aforementioned data concealment methods can be considered conditionally. For example, the data concealment method may depend on the number of non-zero coefficients.

[0238] In another example, NSST_idx and EMT_idx can be dependent. For instance, when EMT_CU_flag is 0 (or 1), NSST_idx may not be 0. In this case, NSST_idx-1 can signal instead of NSST_idx.

[0239] In another embodiment of the present invention, mapping of NSST transformation sets based on an intra-predictive mode is shown in Table 7 below. As mentioned above, the following description focuses on NSST as an example of an inseparable transformation, but other known terms for inseparable transformations (e.g., LFNST) may be used. For example, NSST Set and NSST index may be replaced with LFNST Set and LFNST index. Also, RST as described in this document may be replaced with RST or LFNST as an example of an inseparable transformation (e.g., LFNST) that uses a non-square transformation matrix having a reduced input length and / or reduced output length with a square inseparable transformation matrix applied to at least a portion of the transformation block (the upper left 4x4, 8x8 region, or the remaining region of an 8x8 block excluding the right-lower 4x4 region).

[0240] [Table 7]

[0241] NSST set numbers can be rearranged between 0 and 3, as shown in Table 8.

[0242] [Table 8]

[0243] In NSST translation sets, four translation sets are used (instead of 35) to reduce the memory space required.

[0244] Furthermore, for each set of transformations, a different number of transformation kernels are used, as follows:

[0245] Case A: Two usable transformation kernels are used for each transformation set, and the NSST index range is 0 to 2. For example, if the NSST index is 0, the quadratic transformation (inverse quadratic transformation based on the decoder) may not be applied. If the NSST index is 1 or 2, the quadratic transformation can be applied. A transformation set may contain two transformation kernels, and the index of 1 or 2 may be mapped to these two transformation kernels.

[0246] [Table 9]

[0247] Referring to Table 9, two translation kernels are used for each set of non-separable translations (NSST or LFNST) from number 0 to 3.

[0248] Case B: Two available conversion kernels are used for conversion set 0, and one conversion kernel is used for each of the remaining conversion sets. The available NSST indices for conversion set 0 (DC, Planner) are 0-2. However, the NSST indices for the other modes (conversion sets 1, 2, and 3) are 0-1.

[0249] [Table 10]

[0250] Referring to Table 10, two non-separable translation kernels are configured for the non-separable translation (NSST) set corresponding to index 0, and one non-separable translation kernel is configured for each of the non-separable translation (NSST) sets corresponding to indices 1, 2, and 3.

[0251] Case C: One translation kernel is used for each translation set, and the NSST index range is 0 to 1.

[0252] [Table 11]

[0253] FIG. 21 shows an example of an encoding flowchart for performing conversion as an embodiment to which the present invention is applied.

[0254] The encoding device 100 performs a primary conversion on the residual block (S2105). The primary conversion can be called a core conversion. As an embodiment, the encoding device 100 can perform the primary conversion using the above-described MTS. Further, the encoding device 100 can transfer an MTS index indicating a specific MTS to the decoding device 200 from among the MTS candidates. At this time, the MTS candidates can be configured based on the intra prediction mode of the current block.

[0255] The encoding device 100 determines whether to apply a secondary conversion (S2110). As an example, the encoding device 100 can determine whether to apply a secondary conversion based on the primary-converted residual transform coefficients. For example, the secondary conversion can be NSST or RST.

[0256] The encoding device 100 determines the secondary conversion (S2115). At this time, the encoding device 100 can determine the secondary conversion based on the NSST (or RST) transform set specified according to the intra prediction mode.

[0257] Also, as an example, the encoding device 100 can determine the area to which the secondary conversion is applied based on the size of the current block prior to the S2115 stage.

[0258] The encoding device 100 performs the secondary conversion using the secondary conversion determined at the S2115 stage (S2120).

[0259] FIG. 22 shows an example of a decoding flowchart for performing conversion as an embodiment to which the present invention is applied.

[0260] The decoding device 200 decides whether to apply the inverse quadratic transform (S2205). For example, the inverse quadratic transform could be NSST or RST. As an example, the decoding device 200 may decide whether to apply the inverse quadratic transform based on the quadratic transform flag received from the encoding device 100.

[0261] The decoding device 200 determines the quadratic inverse transform (S2210). At this time, the decoding device 200 can determine the quadratic inverse transform to be applied to the current block based on the NSST (or RST) transform set specified according to the intra prediction mode described above.

[0262] As an example, prior to step S2210, the decoding device 200 can determine the region to which the quadratic inverse transform is applied based on the size of the current block.

[0263] The decoding device 200 performs a quadratic inverse transform on the inversely quantized resistive block using the quadratic inverse transform determined in step S2210 (S2215).

[0264] The decoding device 200 performs a first-order inverse transformation on the quadratic inversely transformed residual block (S2220). The first-order inverse transformation can be called a core inverse transformation. In one embodiment, the decoding device 200 can perform the first-order inverse transformation using the MTS described above. Also, as an example, the decoding device 200 can decide whether the MTS is applied to the current block prior to step S2220. In this case, the decoding flowchart in Figure 22 may further include a step of deciding whether the MTS is applied.

[0265] For example, if an MTS is applied to the current block (i.e., cu_mts_flag = 1), the decoding device 200 can construct MTS candidates based on the intra-prediction mode of the current block. In this case, the decoding flowchart in Figure 22 may further include a step for constructing MTS candidates. The decoding device 200 can then determine the linear inverse transform to be applied to the current block using mts_idx, which indicates a specific MTS from among the constructed MTS candidates.

[0266] Figure 23 shows an example of a detailed block diagram of a conversion unit 120 within an encoding device 100, as an embodiment to which the present invention is applied.

[0267] An encoding device 100 to which an embodiment of the present invention is applied may include a primary conversion unit 2310, a secondary conversion applicability determination unit 2320, a secondary conversion determination unit 2330, and a secondary conversion unit 2340.

[0268] The primary transformation unit 2310 can perform a primary transformation on a residual block. The primary transformation can be called a core transformation. In one embodiment, the primary transformation unit 2310 can perform a primary transformation using the aforementioned MTS. The primary transformation unit 2310 can also transfer an MTS index indicating a specific MTS from among the MTS candidates to the decoding device 200. At this time, the MTS candidates can be configured based on the intra-prediction mode of the current block.

[0269] The secondary transformation applicability determination unit 2320 can determine whether or not to apply a secondary transformation. For example, the secondary transformation applicability determination unit 2320 can determine whether or not to apply a secondary transformation based on the transformation coefficients of the primary transformed residual block. For example, the secondary transformation could be NSST or RST.

[0270] The quadratic transformation determination unit 2330 determines the quadratic transformation. At this time, as described above, the quadratic transformation determination unit 2330 can determine the quadratic transformation based on the NSST (or RST) transformation set specified according to the intra prediction mode.

[0271] As an example, the quadratic transformation determination unit 2330 can also determine the region to which the quadratic transformation is applied based on the size of the current block.

[0272] The quadratic conversion unit 2340 can perform a quadratic conversion using the determined quadratic conversion.

[0273] Figure 24 shows an example of a detailed block diagram of the inverse conversion unit 230 in the decoding device 200 as an embodiment to which the present invention is applied.

[0274] The decoding device 200 to which the present invention is applied includes a quadratic inverse conversion applicability determination unit 2410, a quadratic inverse conversion determination unit 2420, a quadratic inverse conversion unit 2430, and a primary inverse conversion unit 2440.

[0275] The quadratic inverse transformation applicability determination unit 2410 can determine whether or not to apply a quadratic inverse transformation. For example, the quadratic inverse transformation may be NSST or RST. As an example, the quadratic inverse transformation applicability determination unit 2410 can determine whether or not to apply a quadratic inverse transformation based on a quadratic transformation flag received from the encoding device 100. As another example, the quadratic inverse transformation applicability determination unit 2410 can also determine whether or not to apply a quadratic inverse transformation based on the transformation coefficients of the residual block.

[0276] The quadratic inverse transform determination unit 2420 can determine the quadratic inverse transform. At this time, the quadratic inverse transform determination unit 2420 can determine the quadratic inverse transform to be applied to the current block based on the NSST (or RST) transform set specified according to the intra prediction mode.

[0277] Also, as an example, the second inverse transform determination unit 2420 can determine a region to which the second inverse transform is applied based on the size of the current block.

[0278] Also, as an example, the second inverse transform unit 2430 can perform a second inverse transform on the inverse quantized residual block using the determined second inverse transform.

[0279] The first inverse transform unit 2440 can perform a first inverse transform on the second inverse transformed residual block. As an embodiment, the first inverse transform unit 2440 can execute the first transform using the above-described MTS. Also, as an example, the first inverse transform unit 2440 can determine whether the MTS is applied to the current block.

[0280] As an example, when the MTS is applied to the current block (that is, cu_mts_flag = 1), the first inverse transform unit 2440 can construct an MTS candidate based on the intra prediction mode of the current block. Then, the first inverse transform unit 2440 can determine the first transform applied to the current block using the mts_idx indicating a specific MTS from among the configured MTS candidates.

[0281] FIG. 25 shows a flowchart for processing a video signal as an embodiment to which the present invention is applied. The flowchart of FIG. 25 can be executed by the decoding device 200 or the inverse transform unit 230.

[0282] First, the decoding device 200 can determine whether to apply the inverse inseparable transformation to the current block based on the inseparable transformation index and the width and height of the current block. For example, the decoding device 200 can decide to apply the inseparable transformation if the inseparable transformation index is not 0 and the width and height of the current block are both 4 or greater. If the inseparable transformation index is 0, or if the width or height of the current block is less than 4, the decoding device 200 can skip the inverse inseparable transformation and perform the inverse linear transformation.

[0283] In step S2505, the decoding device 200 determines an inseparable transformation set index that indicates which inseparable transformation set to be used for the inseparable transformation of the current block from among the predefined inseparable transformation sets, based on the intra-prediction mode of the current block. The inseparable transformation set index can be set to be assigned to one of four transformation sets, which are set according to the range of the intra-prediction mode, as shown in Table 7 or Table 8. That is, as shown in Table 7 or Table 8, if the intra-prediction mode is 0 to 1, the inseparable transformation set index is determined to the first index value; if the intra-prediction mode is 2 to 12 or 56 to 66, the inseparable transformation set index is determined to the second index value; if the intra-prediction mode is 13 to 23 or 45 to 55, the inseparable transformation set index is determined to the third index value; and if the intra-prediction mode is 24 to 44, the inseparable transformation set index can be determined to the fourth index value.

[0284] Here, a predefined set of inseparable transforms can each contain two transform kernels, as shown in Table 9. Alternatively, a predefined set of inseparable transforms can contain one or two transform kernels, as shown in Table 10 or Table 11.

[0285] In step S2510, the decoding device 200 determines the transformation kernel indicated by the inseparable transformation index of the current block from among the transformation kernels included in the inseparable transformation set indicated by the inseparable transformation set index, using the inseparable transformation matrix. For example, two inseparable transformation kernels can be set for each index value of the inseparable transformation set index, and the decoding device 200 can determine the inseparable transformation matrix from among the two transformation matrix kernels corresponding to the inseparable transformation set index, based on the transformation kernel indicated by the inseparable transformation index.

[0286] At step S2515, the decoding device 200 applies an inseparable transformation matrix to the upper left region of the current block, which is determined according to the width and height of the current block. For example, if both the width and height of the current block are 8 or greater, an inseparable transformation is applied to the 8x8 region in the upper left of the current block. If either the width or height of the current block is less than 8, an inseparable transformation can be applied to a 4x4 region of the current block. The size of the inseparable transformation can also be set to 8x8 or 4x4, corresponding to the region to which the inseparable transformation is applied.

[0287] Furthermore, the decoding device 200 can apply horizontal and vertical transformations to the current block to which a non-separated transformation has been applied. Here, the horizontal and vertical transformations can be determined based on the prediction mode applied to the current block and the MTS index for selecting the transformation matrix.

[0288] Figure 26 shows an example block diagram of an apparatus for processing video signals as an embodiment to which the present invention is applied. The video signal processing apparatus in Figure 26 can correspond to the encoding apparatus in Figure 1 or the decoding apparatus in Figure 2.

[0289] The video processing device 2600, which processes video signals, includes a memory 2620 for storing video signals and a processor 2610 that processes video signals while being coupled with the memory.

[0290] The processor 2610 according to an embodiment of the present invention can consist of at least one processing circuit for processing video signals, and can process video signals by executing commands for encoding or decoding video signals. In other words, the processor 2610 can encode original video data or decode encoded video signals by executing the encoding or decoding methods described above.

[0291] Furthermore, the processing methods to which the present invention is applied can be produced in the form of programs executed on a computer and stored on a computer-readable storage medium. Multimedia data having the data structure according to the present invention can also be stored on a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium can include, for example, Blu-ray discs (BDs), Universal Serial Bus (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media implemented in the form of carrier waves (e.g., transmission over the Internet). Furthermore, bitstreams generated by encoding methods can be stored on computer-readable recording media or transmitted over wireless communication networks.

[0292] Furthermore, embodiments of the present invention can be realized as a computer program product using program code, and the program code can be executed on a computer according to embodiments of the present invention. The program code can be stored on a computer-readable carrier.

[0293] As described above, the embodiments described in the present invention can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure can be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0294] Furthermore, decoders and encoders to which the present invention applies can include real-time communication devices such as multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction devices, and video communication devices, as well as mobile streaming devices, storage media, CAM coders, video-on-demand (VoD) service providers, OTT video (Over the top video) devices, internet streaming service providers, 3D video devices, video phones, video equipment, and medical video equipment, and are used to process video signals or data signals. For example, OTT video (Over the top video) devices can include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, and DVRs (Digital Video Recorders).

[0295] Furthermore, the processing methods to which the present invention is applied can be produced in the form of programs executed on a computer and stored on a computer-readable storage medium. Multimedia data having the data structure according to the present invention can also be stored on a computer-readable storage medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium can include, for example, Blu-ray discs (BDs), Universal Serial Bus (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media implemented in the form of carrier waves (e.g., transmission over the Internet). Furthermore, bitstreams generated by encoding methods can be stored on computer-readable recording media or transmitted over wireless communication networks.

[0296] Furthermore, embodiments of the present invention can be realized as a computer program product using program code, and the program code can be executed on a computer according to embodiments of the present invention. The program code can be stored on a computer-readable carrier.

[0297] The embodiments described above are combinations of the components and features of the present invention in a predetermined form. Each component or feature should be considered optional unless otherwise explicitly mentioned. Each component or feature can be implemented in a form that is not combined with other components or features. It is also possible to combine some components and / or features to constitute embodiments of the present invention. The order of operations described in the embodiments of the present invention can be changed. Some components or features of any embodiment may be included in other embodiments or may be replaced by components or features corresponding to other embodiments. It is obvious that claims that are not explicitly referenced in the claims may be combined to constitute embodiments or may be included as new claims by amendments after filing.

[0298] Embodiments of the present invention can be realized by various means, such as hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one embodiment of the present invention can be realized by one or more ASICs (application-specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0299] In the case of implementation using firmware or software, one embodiment of the present invention can be implemented in the form of a module, procedure, function, etc., that performs the functions or operations described above. The software code can be stored in memory and driven by a processor. The memory can be located inside or outside the processor and can send and receive data with the processor by various already known means.

[0300] It will be obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the essential features of the invention. Therefore, the above-mentioned detailed description should not be constrained in any way restrictively, but should be considered illustrative. The scope of the invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention. [Industrial applicability]

[0301] The preferred embodiments of the present invention described above are disclosed for illustrative purposes only, and those skilled in the art can improve, modify, substitute, or add various other embodiments within the technical concept and technical scope of the present invention disclosed in the appended claims.

Claims

1. In a method for decoding video signals, A step in which it is determined whether an inseparable transform is applied to the current block based on an inseparable transform index obtained from the bitstream, A step in which, based on the determination that the non-separable transformation is applied to the current block, a non-separable transformation set index for the current block is determined based on the intra-prediction mode of the current block, wherein the non-separable transformation set index indicates a non-separable transformation set from among a predefined set of non-separable transformations; A step of determining the transformation kernel included in the inseparable transformation set, which is identified by the inseparable transformation set index, as the inseparable transformation matrix for the current block, The step includes applying the non-separable transformation matrix to the upper-left region of the current block, wherein the upper-left region is determined based on the width and height of the current block, The translation kernel for the current block is determined as one of the two translation kernels included in the non-separable translation set for the current block. The method wherein the non-separable transformation matrix is ​​applied to R coefficients contained in the upper-left region of the current block to generate N transformation coefficients, where R is less than N and N is equal to the total number of coefficients contained in the upper-left region of the current block.

2. In a device for decoding video signals, A memory for storing the aforementioned video signal, A processor coupled with the aforementioned memory, The aforementioned processor, Based on the inseparable transform index obtained from the bitstream, determine whether the inseparable transform is applied to the current block. Based on the determination that the non-separable transformation is applied to the current block, the non-separable transformation set index for the current block is determined based on the intra-prediction mode of the current block, and the non-separable transformation set index indicates a non-separable transformation set from among the predefined non-separable transformation sets. The transformation kernel included in the non-separable transformation set, identified by the non-separable transformation set index, is determined as the non-separable transformation matrix for the current block. The non-separable transformation matrix is ​​applied to the upper-left region of the current block, and the upper-left region is set to be determined based on the width and height of the current block. The translation kernel for the current block is determined as one of the two translation kernels included in the non-separable translation set for the current block. The apparatus applies the non-separable transformation matrix to R coefficients contained in the upper-left region of the current block to generate N transformation coefficients, where R is less than N and N is equal to the total number of coefficients contained in the upper-left region of the current block.

3. In a method for encoding video signals, A step in which it is determined whether an inseparable transform is applied to the current block, wherein an inseparable transform index indicating the determination is encoded in the bitstream, Based on the determination that the non-separable transformation is applied to the current block, the steps include determining a set of non-separable transformations for the current block based on the intra-prediction mode of the current block, The steps include determining the transformation kernel included in the set of inseparable transformations as the inseparable transformation matrix for the current block, The step includes applying the non-separable transformation matrix to the upper-left region of the current block, wherein the upper-left region is determined based on the width and height of the current block, The non-separable transformation set index identifies the non-separable transformation set for the current block from among the predefined non-separable transformation sets, The translation kernel for the current block is determined as one of the two translation kernels included in the non-separable translation set for the current block. The method wherein the non-separable transformation matrix is ​​applied to N coefficients contained in the upper-left region of the current block to generate R transformation coefficients, where R is less than N and N is equal to the total number of coefficients contained in the upper-left region of the current block.

4. In a method for transmitting a bitstream generated by a method for encoding a video signal, A step in which it is determined whether an inseparable transform is applied to the current block, wherein an inseparable transform index indicating the determination is encoded in the bitstream, Based on the determination that the non-separable transformation is applied to the current block, the steps include determining a set of non-separable transformations for the current block based on the intra-prediction mode of the current block, The steps include determining the transformation kernel included in the set of inseparable transformations as the inseparable transformation matrix for the current block, The step includes applying the non-separable transformation matrix to the upper-left region of the current block, wherein the upper-left region is determined based on the width and height of the current block, The non-separable transformation set index identifies the non-separable transformation set for the current block from among the predefined non-separable transformation sets, The translation kernel for the current block is determined as one of the two translation kernels included in the non-separable translation set for the current block. The method wherein the non-separable transformation matrix is ​​applied to N coefficients contained in the upper-left region of the current block to generate R transformation coefficients, where R is less than N and N is equal to the total number of coefficients contained in the upper-left region of the current block.

Images