Method for picture coding based on selective conversion and device therefor

Abstract

To achieve a method and a device for increasing the efficiency of picture coding.SOLUTION: The method for picture decoding by a decoder device according to the present invention includes deriving a residual sample of a target block on the basis of a non-separation conversion of a conversion coefficient based on a bit stream. A prediction sample is derived on the basis of an intra-prediction, one of 67 intra-prediction modes is used to derive a prediction sample, and 67 intra-prediction modes have 2 non-angular prediction mode and 65 angular prediction modes. A non-separation conversion is performed on the basis of a conversion matrix with a corrected bottom vector, and the corrected bottom vector has less elements than N number of elements. N is equal to the number of conversion coefficients located in a region to which a non-separation conversion is applied in a target block. A region to which a non-separation conversion is applied is 8x8 upper-left stage target regions of the target block and N is equal to 64.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to video coding techniques, and more particularly to a method and apparatus for video decoding following selective transformation in a video coding system. [Background technology]

[0002] Recently, the demand for high-resolution, high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images is increasing in various fields. As the resolution and quality of image data increases, the amount of information or bits to be transmitted increases relatively compared to existing image data. Therefore, when the image data is transmitted using a medium such as an existing wired or wireless broadband line or when the image data is stored using an existing storage medium, the transmission cost and storage cost increase.

[0003] This calls for highly efficient video compression techniques to effectively transmit, store, and play back high-resolution, high-quality video information. Summary of the Invention [Problem to be solved by the invention]

[0004] SUMMARY OF THE PRESENT EMBODIMENT It is an object of the present invention to provide a method and apparatus for improving video coding efficiency.

[0005] Another technical object of the present invention is to provide a method and apparatus for increasing the conversion efficiency.

[0006] It is still another technical object of the present invention to provide a method and apparatus for improving efficiency of residual coding through transform.

[0007] It is still another object of the present invention to provide a video coding method and apparatus based on selective transform. [Means for solving the problem]

[0008] According to an embodiment of the present invention, there is provided a video decoding method performed by a decoding device, the method comprising the steps of: deriving transform coefficients of a current block from a bitstream, deriving residual samples for the current block based on a selective transform of the transform coefficients, and generating a reconstructed picture based on the residual samples for the current block and the prediction samples for the current block, the selective transform being based on a modified transform matrix, the modified transform matrix being a matrix having modified basis vectors, the modified basis vectors having a specific number of elements selected from N elements.

[0009] According to another embodiment of the present invention, there is provided a decoding device for performing video decoding, comprising: an entropy decoding unit for deriving transform coefficients of a current block from a bitstream; an inverse transform unit for deriving residual samples for the current block based on a selective transform of the transform coefficients; and an adder unit for generating a reconstructed picture based on the residual samples for the current block and prediction samples for the current block, wherein the selective transform is performed based on a modified transform matrix, the modified transform matrix being a matrix having modified basis vectors, and the modified basis vectors having a specific number of elements selected from N elements.

[0010] According to yet another embodiment of the present invention, there is provided a video encoding method performed by an encoding device, the method comprising the steps of deriving residual samples of a current block, deriving transform coefficients of the current block based on a selective transform of the residual samples, and encoding information about the transform coefficients, the selective transform being based on a modified transform matrix, the modified transform matrix being a matrix having modified basis vectors, the modified basis vectors having a specific number of elements selected from N elements.

[0011] According to yet another embodiment of the present invention, there is provided a video encoding device, comprising: an adder for deriving residual samples of a current block; a transformer for deriving transform coefficients of the current block based on a selective transform of the residual samples; and an entropy encoding unit for encoding information on the transform coefficients, wherein the selective transform is performed based on a modified transform matrix, the modified transform matrix being a matrix having modified basis vectors, and the modified basis vectors having a specific number of elements selected from N elements. Effect of the Invention

[0012] According to the present invention, the amount of data that must be transmitted for residual processing can be reduced through efficient conversion, thereby improving residual coding efficiency.

[0013] According to the present invention, a non-separable transform can be performed based on a transform matrix composed of basis vectors including a selected specific number of elements, thereby reducing memory load and computational complexity for the non-separable transform and improving residual coding efficiency.

[0014] According to the present invention, a non-separable transform can be performed based on a transform matrix having a simplified structure, thereby reducing the amount of data that needs to be transmitted for residual processing and improving residual coding efficiency. [Brief description of the drawings]

[0015] [Figure 1] 1 is a diagram illustrating the schematic configuration of a video encoding device to which the present invention can be applied; [Diagram 2] 1 is a diagram illustrating the schematic configuration of a video decoding device to which the present invention can be applied; [Diagram 3] FIG. 2 illustrates a schematic diagram of a multiple transform technique according to the present invention; [Figure 4] FIG. 13 is a diagram illustrating an example of intra-directional modes of 65 prediction directions. [Figure 5A] FIG. 2 illustrates a selective transform according to one embodiment of the present invention. [Figure 5B] FIG. 2 illustrates a selective transform according to one embodiment of the present invention. [Figure 5C] FIG. 2 illustrates a selective transform according to one embodiment of the present invention. [Figure 6] FIG. 2 is a diagram illustrating a multiple transform technique in which the selective transform is applied to a secondary transform. [Figure 7] FIG. 2 is a diagram illustrating an arrangement of transform coefficients based on a current block according to an embodiment of the present invention. [Figure 8] 13 is a diagram illustrating an example of deriving transform coefficients through a transform in which the simplified transform and the selective transform are combined; [Figure 9] FIG. 2 is a diagram illustrating an example of deriving transform coefficients through the selective transform. [Figure 10] FIG. 13 is a diagram showing an example of deriving and selectively transforming related vectors based on two factors for the related vectors. [Figure 11] 1 is a diagram illustrating a video encoding method using an encoding device according to the present invention; [Figure 12] 1 is a diagram illustrating an encoding device for performing a video encoding method according to the present invention; [Figure 13] 2 is a diagram illustrating a video decoding method using a decoding device according to the present invention; [Figure 14] 1 is a diagram illustrating a schematic configuration of a decoding device for performing a video decoding method according to the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] The present invention can be modified in various ways and can have various embodiments, and a specific embodiment will be illustrated in the drawings and described in detail. However, this does not limit the present invention to a specific embodiment. The terms used in this specification are used only to describe a specific embodiment and are not intended to limit the technical idea of ​​the present invention. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this specification, the terms "include" or "have" specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0017] Meanwhile, each component in the drawings described in the present invention is illustrated independently for the convenience of explaining the different characteristic functions, and does not mean that each component is realized by separate hardware or software. For example, two or more components among the components may be combined to form one component, or one component may be divided into multiple components. An embodiment in which each component is integrated and / or separated is also included in the scope of the present invention as long as it does not deviate from the essence of the present invention.

[0018] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following, the same reference numerals are used to refer to the same components in the drawings, and duplicated descriptions of the same components will be omitted.

[0019] Meanwhile, the present invention relates to video / image coding, for example, the methods / embodiments disclosed in the present invention can be applied to methods disclosed in the Versatile Video Coding (VVC) standard or next generation video / image coding.

[0020] In this specification, a picture generally means a unit showing one image at a specific time, and a slice is a unit constituting a part of a picture in coding. One picture may be composed of multiple slices, and pictures and slices may be mixed together as necessary.

[0021] A pixel or pel may refer to the smallest unit constituting one picture (or image). Also, a term corresponding to a pixel may be "sample." A sample generally indicates a pixel or a pixel value, and may indicate only a pixel / pixel value of a luminance (luma) component, or may indicate only a pixel / pixel value of a chroma component.

[0022] A unit refers to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information about the region. A unit may be mixed with terms such as a block or an area in some cases. In a general case, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows.

[0023] FIG. 1 is a diagram for explaining the schematic configuration of a video encoding device to which the present invention can be applied.

[0024] 1, the video encoding device 100 may include a picture division unit 105, a prediction unit 110, a residual processing unit 120, an entropy encoding unit 130, an addition unit 140, a filter unit 150, and a memory 160. The residual processing unit 120 may include a subtraction unit 121, a transformation unit 122, a quantization unit 123, a realignment unit 124, an inverse quantization unit 125, and an inverse transformation unit 126.

[0025] The picture division unit 105 can divide an input picture into at least one processing unit.

[0026] As an example, the processing unit is called a coding unit (CU). In this case, the coding unit may be recursively divided from the largest coding unit (LCU) according to a quad-tree binary-tree (QTBT) structure. For example, one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad-tree structure and / or a binary tree structure. In this case, for example, the quad-tree structure may be applied first, and then the binary tree structure may be applied. Alternatively, the binary tree structure may be applied first. A coding procedure according to the present invention may be performed based on the final coding unit that is not further divided. In this case, the largest coding unit may be used as the final coding unit based on coding efficiency according to image characteristics, or the coding unit may be recursively divided into coding units of a lower depth as necessary, and a coding unit of an optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later.

[0027] As another example, the processing unit may include a coding unit (CU), a prediction unit (PU), or a transform unit (TU). The coding unit may be split from a largest coding unit (LCU) into deeper coding units according to a quadtree structure. In this case, the largest coding unit may be used as the final coding unit based on coding efficiency according to image characteristics, or the coding unit may be recursively split into coding units of lower depths as necessary, and a coding unit of an optimal size may be used as the final coding unit. When a smallest coding unit (SCU) is set, the coding unit may not be split into coding units smaller than the smallest coding unit. Here, the final coding unit refers to a coding unit that is a base for partitioning or splitting into prediction units or transform units. A prediction unit is a unit that is partitioned from a coding unit, and is a unit of sample prediction. In this case, the prediction unit may be divided into subblocks. The transform unit may be divided from the coding unit according to a quadtree structure, and is a unit that derives transform coefficients and / or a unit that derives a residual signal from the transform coefficients. Hereinafter, the coding unit is also called a coding block (CB), the prediction unit is also called a prediction block (PB), and the transform unit is also called a transform block (TB). A prediction block or a prediction unit refers to a specific region in a block form within a picture, and may include an array of prediction samples.Also, a transform block or transform unit refers to a specific region in block form within a picture and may include an array of transform coefficients or residual samples.

[0028] The prediction unit 110 may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including a prediction sample for the current block. The unit of prediction performed by the prediction unit 110 is a coding block, a transform block, or a prediction block.

[0029] The prediction unit 110 may determine whether intra prediction or inter prediction is applied to the current block. For example, the prediction unit 110 may determine whether intra prediction or inter prediction is applied on a CU basis.

[0030] In the case of intra prediction, the prediction unit 110 may derive a prediction sample for the current block based on a reference sample outside the current block in a picture to which the current block belongs (hereinafter, the current picture). In this case, the prediction unit 110 may (i) derive a prediction sample based on an average or an interpolation of neighboring reference samples of the current block, and (ii) derive the prediction sample based on a reference sample that exists in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. The case (i) is called a non-directional mode or a non-angular mode, and the case (ii) is called a directional mode or an angular mode. The prediction mode in intra prediction may have, for example, 33 directional prediction modes and at least two or more non-directional modes. The non-directional modes may include a DC prediction mode and a planar mode. The prediction unit 110 may also determine a prediction mode to be applied to the current block by using a prediction mode applied to a neighboring block.

[0031] In the case of inter prediction, the prediction unit 110 may derive a prediction sample for the current block based on a sample specified by a motion vector on a reference picture. The prediction unit 110 may derive a prediction sample for the current block by applying any one of a skip mode, a merge mode, and a Motion Vector Prediction (MVP) mode. In the case of the skip mode and the merge mode, the prediction unit 110 may use motion information of a neighboring block as motion information of the current block. In the case of the skip mode, unlike the merge mode, the difference (residual) between the prediction sample and the original sample is not transmitted. In the case of the MVP mode, the motion vector of the neighboring block may be used as a motion vector predictor, and the motion vector of the current block may be derived by using the motion vector of the neighboring block as the motion vector predictor of the current block.

[0032] In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in a reference picture. The reference picture including the temporal neighboring blocks is also called a collocated picture (colPic). The motion information may include a motion vector and a reference picture index. Information such as prediction mode information and motion information may be (entropy) encoded and output in the form of a bitstream.

[0033] When motion information of a temporally neighboring block is used in skip mode and merge mode, the top picture on a reference picture list may be used as a reference picture. Reference pictures included in a reference picture list may be sorted based on the difference in POC (Picture Order Count) between the current picture and the corresponding reference picture. POC corresponds to the display order of pictures and may be distinguished from the coding order.

[0034] The subtractor 121 generates residual samples, which are the difference between the original samples and the predicted samples, or, if skip mode is applied, does not generate residual samples, as described above.

[0035] The transform unit 122 transforms the residual samples in units of transform blocks to generate transform coefficients. The transform unit 122 may perform the transform according to the size of the corresponding transform block and a prediction mode applied to a coding block or a predictive block that spatially overlaps with the corresponding transform block. For example, if intra prediction is applied to the coding block or the predictive block that overlaps with the transform block and the transform block is a 4×4 residual array, the residual samples may be transformed using a Discrete Sine Transform (DST) transform kernel, and otherwise, the residual samples may be transformed using a Discrete Cosine Transform (DCT) transform kernel.

[0036] The quantization unit 123 may quantize the transform coefficients to generate quantized transform coefficients.

[0037] The rearrangement unit 124 rearranges the quantized transform coefficients. The rearrangement unit 124 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form through a coefficient scanning method. Here, the rearrangement unit 124 has been described as a separate configuration, but may be a part of the quantization unit 123.

[0038] The entropy encoding unit 130 may perform entropy encoding on the quantized transform coefficients. The entropy encoding may include encoding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), CABAC (Context-Adaptive Binary Arithmetic Coding), etc. The entropy encoding unit 130 may also encode information required for video restoration (e.g., syntax element values, etc.) together with or separately from the quantized transform coefficients using entropy encoding or a preset method. The encoded information may be transmitted or stored in the form of a bitstream in Network Abstraction Layer (NAL) unit units.

[0039] The inverse quantization unit 125 inverse quantizes the values ​​quantized by the quantization unit 123 (quantized transformation coefficients), and the inverse transform unit 126 inverse transforms the values ​​inverse quantized by the inverse quantization unit 125 to generate residual samples.

[0040] The adder 140 reconstructs a picture by adding residual samples and prediction samples. The residual samples and prediction samples may be added in units of blocks to generate reconstructed blocks. Although the adder 140 has been described as a separate configuration, it may be a part of the prediction unit 110. Meanwhile, the adder 140 may also be referred to as a reconstruction module or a reconstructed block generator.

[0041] The filter unit 150 may apply a deblocking filter and / or a sample adaptive offset to the reconstructed picture. Through the deblock filtering and / or the sample adaptive offset, artifacts at block boundaries in the reconstructed picture and distortion in the quantization process may be corrected. The sample adaptive offset may be applied on a sample basis and may be applied after the deblock filtering process is completed. The filter unit 150 may also apply an adaptive loop filter (ALF) to the reconstructed picture. The ALF may be applied to the reconstructed picture after the deblock filtering and / or the sample adaptive offset are applied.

[0042] The memory 160 may store a reconstructed picture (decoded picture) or information required for encoding / decoding. Here, the reconstructed picture is a reconstructed picture that has been subjected to a filtering procedure by the filter unit 150. The stored reconstructed picture may be used as a reference picture for (inter) prediction of another picture. For example, the memory 160 may store (reference) pictures used for inter prediction. In this case, the pictures used for inter prediction may be specified by a reference picture set or a reference picture list.

[0043] FIG. 2 is a diagram for explaining the schematic configuration of a video decoding device to which the present invention can be applied.

[0044] 2, the video decoding device 200 may include an entropy decoding unit 210, a residual processing unit 220, a prediction unit 230, an addition unit 240, a filter unit 250, and a memory 260. Here, the residual processing unit 220 may include a reordering unit 221, an inverse quantization unit 222, and an inverse transform unit 223.

[0045] When a bitstream containing video information is input, the video decoding device 200 can restore the video in accordance with the process by which the video information was processed in the video encoding device.

[0046] For example, the video decoding device 200 may perform video decoding using a processing unit applied in a video encoding device. Thus, a processing unit block of the video decoding may be a coding unit as an example, and may be a coding unit, a prediction unit, or a transform unit as another example. The coding unit may be divided from the maximum coding unit by a quadtree structure and / or a binary tree structure.

[0047] A prediction unit and a transform unit may also be used in some cases, where the prediction block is a block derived or partitioned from the coding unit and is a unit of sample prediction. At this time, the prediction unit may be divided into sub-blocks. The transform unit may be divided from the coding unit by a quadtree structure and is a unit that derives transform coefficients or a unit that derives a residual signal from the transform coefficients.

[0048] The entropy decoding unit 210 may parse the bitstream and output information required for video reconstruction or picture reconstruction. For example, the entropy decoding unit 210 may decode information in the bitstream based on a coding method such as Exponential Golomb Coding, CAVLC, or CABAC, and output values ​​of syntax elements required for video reconstruction and quantized values ​​of transform coefficients for residuals.

[0049] More specifically, the CABAC entropy decoding method receives BINs corresponding to each syntax element in a bitstream, determines a context model using information on the syntax element to be decoded and decoding information on adjacent and blocks to be decoded or information on symbols / BINs decoded in a previous step, predicts the occurrence probability of BINs according to the determined context model, and performs arithmetic decoding of BINs to generate symbols corresponding to the values ​​of each syntax element. In this case, the CABAC entropy decoding method may update the context model using information on the decoded symbols / BINs for the context model of the next symbol / BIN after determining the context model.

[0050] Prediction information among the information decoded by the entropy decoding unit 210 is provided to the prediction unit 230, and residual values, i.e., quantized transform coefficients, on which entropy decoding is performed by the entropy decoding unit 210 can be input to the reordering unit 221.

[0051] The reordering unit 221 may reorder the quantized transform coefficients in a two-dimensional block format. The reordering unit 221 may perform the reordering in response to a coefficient scan performed in the encoding device. Here, the reordering unit 221 has been described as a separate configuration, but may be a part of the inverse quantization unit 222.

[0052] The inverse quantization unit 222 may inversely quantize the quantized transform coefficients based on the (inverse) quantization parameter to output the transform coefficients. At this time, information for deriving the quantization parameter may be signaled from the encoding device.

[0053] The inverse transform unit 223 may inverse transform the transform coefficients to derive residual samples.

[0054] The prediction unit 230 may perform prediction on a current block and generate a predicted block including prediction samples for the current block. The unit of prediction performed by the prediction unit 230 is a coding block, a transform block, or a prediction block.

[0055] The prediction unit 230 can determine whether to apply intra prediction or inter prediction based on the information on the prediction. At this time, the unit for determining which of intra prediction and inter prediction to apply is different from the unit for generating a prediction sample. In addition, the unit for generating a prediction sample is also different in inter prediction and intra prediction. For example, which of inter prediction and intra prediction to apply can be determined in units of CU. Also, for example, in inter prediction, a prediction mode can be determined in units of PU to generate a prediction sample, and in intra prediction, a prediction mode can be determined in units of PU to generate a prediction sample in units of TU.

[0056] In the case of intra prediction, the prediction unit 230 may derive a prediction sample for the current block based on a neighboring reference sample in the current picture. The prediction unit 230 may derive a prediction sample for the current block by applying a directional mode or a non-directional mode based on the neighboring reference sample of the current block. In this case, a prediction mode to be applied to the current block may be determined using the intra prediction mode of the neighboring block.

[0057] In the case of inter prediction, the prediction unit 230 may derive a prediction sample for the current block based on a sample identified on the reference picture by a motion vector on the reference picture. The prediction unit 230 may derive a prediction sample for the current block by applying any one of a skip mode, a merge mode, and an MVP mode. In this case, motion information required for inter prediction of the current block provided by the video encoding apparatus, such as information on a motion vector, a reference picture index, etc., may be obtained or derived based on the information on the prediction.

[0058] In the case of skip mode and merge mode, the motion information of the neighboring blocks may be used as the motion information of the current block, where the neighboring blocks may include spatial neighboring blocks and temporal neighboring blocks.

[0059] The prediction unit 230 can construct a merge candidate list with the motion information of available neighboring blocks, and use the information indicated by the merge index on the merge candidate list as the motion vector of the current block. The merge index can be signaled from an encoding device. The motion information can include a motion vector and a reference picture. When the motion information of the temporal neighboring blocks is used in the skip mode and the merge mode, the top picture on the reference picture list can be used as the reference picture.

[0060] In skip mode, unlike merge mode, the difference between the predicted samples and the original samples (residuals) is not transmitted.

[0061] In the MVP mode, the motion vector of the current block can be derived using the motion vector of the neighboring blocks as a motion vector predictor, where the neighboring blocks can include spatial and temporal neighboring blocks.

[0062] For example, when a merge mode is applied, a merge candidate list may be generated using the motion vectors of the restored spatial neighboring blocks and / or the motion vector corresponding to the Col block, which is a temporal neighboring block. In the merge mode, a motion vector of a candidate block selected from the merge candidate list is used as the motion vector of the current block. The prediction information may include a merge index indicating a candidate block having an optimal motion vector selected from among the candidate blocks included in the merge candidate list. In this case, the prediction unit 230 may derive a motion vector of the current block using the merge index.

[0063] As another example, when a Motion Vector Prediction (MVP) mode is applied, a motion vector predictor candidate list may be generated using the motion vector of the restored spatial neighboring block and / or the motion vector corresponding to the Col block, which is a temporal neighboring block. That is, the motion vector of the restored spatial neighboring block and / or the motion vector corresponding to the Col block, which is a temporal neighboring block, may be used as a motion vector candidate. The prediction information may include a predicted motion vector index indicating an optimal motion vector selected from the motion vector candidates included in the list. In this case, the prediction unit 230 may select a predicted motion vector of the current block from the motion vector candidates included in the motion vector candidate list using the motion vector index. A prediction unit of an encoding device may obtain a motion vector difference (MVD) between the motion vector of the current block and a motion vector predictor, encode the motion vector difference, and output the motion vector difference in the form of a bitstream. That is, the MVD is obtained by subtracting the motion vector predictor from the motion vector of the current block. In this case, the prediction unit 230 may obtain the motion vector difference included in the prediction information, and derive the motion vector of the current block by adding the motion vector difference and the motion vector predictor. The prediction unit can also obtain or derive a reference picture index indicating a reference picture from the information related to the prediction.

[0064] The adder 240 may reconstruct a current block or a current picture by adding a residual sample and a prediction sample. The adder 240 may also reconstruct a current picture by adding a residual sample and a prediction sample in units of a block. When a skip mode is applied, a residual is not transmitted, and therefore a prediction sample may become a reconstructed sample. Here, the adder 240 has been described as a separate configuration, but may be a part of the prediction unit 230. Meanwhile, the adder 240 may also be referred to as a reconstruction module or a reconstructed block generation module.

[0065] The filter unit 250 may apply deblock filtering, sample adaptive offset, and / or ALF to the reconstructed picture. In this case, the sample adaptive offset may be applied on a sample basis or may be applied after deblock filtering. The ALF may be applied after deblock filtering and / or sample adaptive offset.

[0066] The memory 260 may store a reconstructed picture (a decoded picture) or information required for decoding. Here, the reconstructed picture is a reconstructed picture for which a filtering procedure by the filter unit 250 has been completed. For example, the memory 260 may store a picture used in inter prediction. At this time, the picture used in inter prediction may be specified by a reference picture set or a reference picture list. The reconstructed picture may be used as a reference picture for another picture. In addition, the memory 260 may output the reconstructed picture according to an output order.

[0067] Meanwhile, through the above-described transformation, a transform coefficient of a lower frequency for a residual block of the current block can be derived, and a zero tail can be derived at an end of the residual block.

[0068] Specifically, the transform may be composed of two main processes, which may include a core transform and a secondary transform. The transform including the core transform and the secondary transform may be referred to as a multiple transform technique.

[0069] FIG. 3 illustrates a schematic of a multiple conversion technique according to the present invention.

[0070] Referring to Figure 3, the conversion unit may correspond to the conversion unit in the encoding device of Figure 1 described above, and the inverse conversion unit may correspond to the inverse conversion unit in the encoding device of Figure 1 described above or the inverse conversion unit in the decoding device of Figure 2.

[0071] The transform unit may perform a first transform based on the residual samples (residual sample array) in the residual block to derive (first) transform coefficients (S310). Here, the first transform may include an adaptive multiple core transform (AMT). The adaptive multiple core transform may be referred to as a multiple transform set (MTS).

[0072] The adaptive multi-core transform may refer to a method of transforming by additionally using a Discrete Cosine Transform (DCT) type 2 and a Discrete Sine Transform (DST) type 7, DCT type 8, and / or DST type 1. That is, the adaptive multi-core transform may refer to a transform method of transforming a residual signal (or a residual block) in a spatial domain into a transform coefficient (or a first transform coefficient) in a frequency domain based on a plurality of transform kernels selected from the DCT type 2, the DST type 7, the DCT type 8, and the DST type 1. Here, the first transform coefficient may be referred to as a temporary transform coefficient from the perspective of a transform unit.

[0073] In other words, when an existing transform method is applied, a spatial domain to frequency domain transform may be applied to the residual signal (or the residual block) based on DCT type 2 to generate transform coefficients. Differently, when the adaptive multi-core transform is applied, a spatial domain to frequency domain transform may be applied to the residual signal (or the residual block) based on DCT type 2, DST type 7, DCT type 8, and / or DST type 1, etc. to generate transform coefficients (or primary transform coefficients). Here, DCT type 2, DST type 7, DCT type 8, DST type 1, etc. may be referred to as transform types, transform kernels, or transform cores.

[0074] For reference, the DCT / DST transformation types can be defined based on basis functions, and the basis functions can be shown as in the following table.

[0075] [Table 1]

[0076] When the adaptive multi-core transform is performed, a vertical transform kernel and a horizontal transform kernel for a current block may be selected from the transform kernels, and a vertical transform may be performed on the current block based on the vertical transform kernel, and a horizontal transform may be performed on the current block based on the horizontal transform kernel. Here, the horizontal transform may indicate a transform for a horizontal component of the current block, and the vertical transform may indicate a transform for a vertical component of the current block. The vertical transform kernel / horizontal transform kernel may be adaptively determined based on a transform index indicating a prediction mode and / or a transform subset of a current block (CU or sub-block) that encompasses a residual block.

[0077] For example, the adaptive multi-core transform may be applied when the width and height of the target block are all less than or equal to 64, and whether the adaptive multi-core transform of the target block is applied may be determined based on a CU level flag. Specifically, when the CU level flag is 0, the existing transform method described above may be applied. That is, when the CU level flag is 0, a spatial domain to frequency domain transform may be applied to a residual signal (or a residual block) based on the DCT type 2 to generate transform coefficients, and the transform coefficients may be encoded. Meanwhile, here, the target block may be a CU. When the CU level flag is 0, the adaptive multi-core transform may be applied to the target block.

[0078] Also, for a luma block of a target block to which the adaptive multi-core transform is applied, two additional flags may be signaled, and a vertical transform kernel and a horizontal transform kernel may be selected based on the flags. The flag for the vertical transform kernel may be indicated as an AMT vertical flag, and AMT_TU_vertical_flag (or EMT_TU_vertical_flag) may indicate a syntax element of the AMT vertical flag. The flag for the horizontal transform kernel may be indicated as an AMT horizontal flag, and AMT_TU_horizontal_flag (or EMT_TU_horizontal_flag) may indicate a syntax element of the AMT horizontal flag. The AMT vertical flag may indicate one transform kernel candidate among the transform kernel candidates included in a transform subset for the vertical transform kernel, and the transform kernel candidate indicated by the AMT vertical flag may be derived as a vertical transform kernel for the target block. In addition, the AMT horizontal flag may indicate one of the transform kernel candidates included in the transform subset for the horizontal transform kernel, and the transform kernel candidate indicated by the AMT horizontal flag may be derived as a horizontal transform kernel for the target block. Meanwhile, the AMT vertical flag may be indicated as an MTS vertical flag, and the AMT horizontal flag may be indicated as an MTS horizontal flag.

[0079] Meanwhile, three transform subsets may be preset, and one of the transform subsets may be derived as a transform subset for the vertical transform kernel based on an intra prediction mode applied to the current block. Also, one of the transform subsets may be derived as a transform subset for the horizontal transform kernel based on an intra prediction mode applied to the current block. For example, the preset transform subsets may be derived as shown in the following table.

[0080] [Table 2]

[0081] Referring to Table 2, a transform subset with an index value of 0 may indicate a transform subset that includes DST type 7 and DCT type 8 as transform kernel candidates, a transform subset with an index value of 1 may indicate a transform subset that includes DST type 7 and DST type 1 as transform kernel candidates, and a transform subset with an index value of 2 may indicate a transform subset that includes DST type 7 and DCT type 8 as transform kernel candidates.

[0082] The transform subset for the vertical transform kernel and the transform subset for the horizontal transform kernel derived based on the intra prediction mode applied to the current block may be derived as shown in the following table.

[0083] [Table 3]

[0084] Here, V denotes a transform subset for the vertical transform kernel, and H denotes a transform subset for the horizontal transform kernel.

[0085] When the value of the AMT flag (or EMT_CU_flag) for the target block is 1, a transform subset for the vertical transform kernel and a transform subset for the horizontal transform kernel may be derived based on the intra prediction mode of the target block, as shown in Table 3. Then, among the transform kernel candidates included in the transform subset for the vertical transform kernel, a transform kernel candidate indicated by the AMT vertical flag of the target block may be derived as the vertical transform kernel of the target block, and among the transform kernel candidates included in the transform subset for the horizontal transform kernel, a transform kernel candidate indicated by the AMT horizontal flag of the target block may be derived as the horizontal transform kernel of the target block. Meanwhile, the AMT flag may also be indicated as an MTS flag.

[0086] For reference, for example, the intra prediction modes may include two non-directional (or non-angular) intra prediction modes and 65 directional (or angular) intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode numbered 0 and a DC intra prediction mode numbered 1, and the directional intra prediction modes may include 65 intra prediction modes numbered 2 to 66. However, this is merely an example, and the present invention may be applied to cases where the number of intra prediction modes is different. Meanwhile, a 67th intra prediction mode may also be used in some cases, and the 67th intra prediction mode may indicate a Linear Model (LM) mode.

[0087] FIG. 4 exemplarily shows intra-directional modes of 65 prediction directions.

[0088] Referring to FIG. 4, the intra prediction modes may be classified into those having horizontal directionality and those having vertical directionality, with the 34th intra prediction mode having a left-up diagonal prediction direction as the center. H and V in FIG. 4 respectively mean horizontal directionality and vertical directionality, and the numbers from -32 to 32 indicate a displacement of 1 / 32 units on a sample grid position. The 2nd to 33rd intra prediction modes have horizontal directionality, and the 34th to 66th intra prediction modes have vertical directionality. The 18th and 50th intra prediction modes indicate horizontal and vertical intra prediction modes, respectively, and the 2nd intra prediction mode may be called a left-down diagonal intra prediction mode, the 34th intra prediction mode may be called a left-up diagonal intra prediction mode, and the 66th intra prediction mode may be called a right-up diagonal intra prediction mode.

[0089] The transform unit may derive (secondary) transform coefficients by performing a secondary transform based on the (first) transform coefficients (S320). If the first transform is a transform from a spatial domain to a frequency domain, the secondary transform may be considered as a transform from a frequency domain to a frequency domain. The secondary transform may include a non-separable transform. In this case, the secondary transform may be referred to as a non-separable secondary transform (NSST) or a mode-dependent non-separable secondary transform (MDNSST). The non-separable secondary transform may refer to a transform in which the (first) transform coefficients derived through the first transform are subjected to a secondary transform based on a non-separable transform matrix to generate transform coefficients (or secondary transform coefficients) for a residual signal. Here, a transform may be applied at once to the (first) transform coefficients based on the non-separable transform matrix without applying a vertical transform and a horizontal transform separately (or applying a horizontal-vertical transform independently). In other words, the non-separable secondary transform may refer to a transform method in which the vertical and horizontal components of the (first-order) transform coefficients are not separated based on the non-separable transform matrix, but are transformed together to generate transform coefficients (or second-order transform coefficients). The non-separable secondary transform may be applied to a top-left region of a block (hereinafter, may be referred to as a transform coefficient block or a target block) composed of the (first-order) transform coefficients. For example, if the width (W) and height (H) of the transform coefficient block are both 8 or more, an 8×8 non-separable secondary transform may be applied to a top-left 8×8 region (hereinafter, top-left target region) of the transform coefficient block. Also, if the width (W) and height (H) of the transform coefficient block are both 4 or more and the width (W) or height (H) of the transform coefficient block is less than 8, a 4×4 non-separable secondary transform may be applied to a top-left min(8,W)×min(8,H) region of the transform coefficient block.

[0090] Specifically, for example, if a 4×4 input block is used, a non-separable quadratic transform can be performed as follows.

[0091] The above 4×4 input block X can be expressed as follows:

[0092] <Formula 1>

number

[0093] If the above X is expressed in vector form, the vector TIFF2024177555000006.tif84 can be displayed as follows:

[0094] <Formula 2>

number

[0095] In this case, the above second-order non-separable transform can be calculated as follows:

[0096] <Formula 3>

number

[0097] Where: TIFF2024177555000009.tif75 denotes a transform coefficient vector, and T denotes a 16x16 (non-separable) transform matrix.

[0098] The 16×1 transform coefficient vector TIFF2024177555000010.tif75 can be derived from the above TIFF2024177555000011.tif75 can be re-organized into 4x4 blocks through the scan order (horizontal, vertical, diagonal, etc.). However, the above calculation is an example, and in order to reduce the computational complexity of the non-separable quadratic transform, HyGT (Hypercube-Givens Transform) etc. can be used to calculate the non-separable quadratic transform.

[0099] Meanwhile, in the non-separable quadratic transform, the transform kernel (or transform core, transform type) may be selected to be mode dependent, where the mode may include intra prediction mode and / or inter prediction mode.

[0100] As described above, the non-separable secondary transform may be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. That is, the non-separable secondary transform may be performed based on an 8×8 sub-block size or a 4×4 sub-block size. For example, for the mode-dependent transform kernel selection, 35 sets of three non-separable secondary transform kernels for the non-separable secondary transform may be configured for both the 8×8 sub-block size and the 4×4 sub-block size. That is, 35 transform sets may be configured for the 8×8 sub-block size, and 35 transform sets may be configured for the 4×4 sub-block size. In this case, the 35 transform sets for the 8×8 sub-block size may each include three 8×8 transform kernels, and the 35 transform sets for the 4×4 sub-block size may each include three 4×4 transform kernels. However, the transform sub-block size, the number of sets, and the number of transform kernels in a set may be sizes other than, for example, 8×8 or 4×4, or n sets may be constructed with k transform kernels in each set.

[0101] The transform set may be referred to as an NSST set, and the transform kernels in the NSST set may be referred to as NSST kernels. Selection of a particular set from the transform set may be performed based on, for example, an intra prediction mode of a current block (CU or sub-block).

[0102] In this case, the mapping between the 35 transform sets and the intra prediction modes may be shown, for example, as follows: For reference, when the LM mode is applied to the current block, the secondary transform may not be applied to the current block.

[0103] [Table 4]

[0104] Meanwhile, if it is determined that a specific set is to be used, one of the k transform kernels in the specific set may be selected through a non-separable secondary transform index. The encoding apparatus may derive a non-separable secondary transform index indicating a specific transform kernel based on a rate-distortion (RD) check, and may signal the non-separable secondary transform index to a decoding apparatus. The decoding apparatus may select one of the k transform kernels in the specific set based on the non-separable secondary transform index. For example, an NSST index value of 0 may indicate a first non-separable secondary transform kernel, an NSST index value of 1 may indicate a second non-separable secondary transform kernel, and an NSST index value of 2 may indicate a third non-separable secondary transform kernel. Alternatively, an NSST index value of 0 may indicate that a first non-separable secondary transform is not applied to a current block, and NSST index values ​​of 1 to 3 may indicate the three transform kernels.

[0105] 3, the transform unit may perform the non-separable second-order transform based on the selected transform kernel to obtain (second-order) transform coefficients. The transform coefficients may be derived as quantized transform coefficients through the quantizer, as described above, and may be encoded and signaled to a decoding device and transmitted to an inverse quantization / inverse transform unit in an encoding device.

[0106] On the other hand, if the secondary transform is omitted, the (primary) transform coefficients, which are the output of the primary (separate) transform, can be derived as quantized transform coefficients through a quantization unit as described above, and can be encoded and signaled to a decoding device and transmitted to an inverse quantization / inverse transform unit in an encoding device.

[0107] The inverse transform unit may perform a series of procedures in the reverse order of the procedures performed by the transform unit described above. The inverse transform unit may receive (dequantized) transform coefficients, perform a secondary (inverse) transform to derive (primary) transform coefficients (S350), and perform a primary (inverse) transform on the (primary) transform coefficients to obtain residual blocks (residual samples) (S360). Here, the primary transform coefficients may be referred to as modified transform coefficients from the perspective of the inverse transform unit. As described above, the encoding device and the decoding device may generate a reconstructed block based on the residual block and the predicted block, and generate a reconstructed picture based on the reconstructed block.

[0108] Meanwhile, as described above, when the secondary (inverse) transform is omitted, the (dequantized) transform coefficients are received and the primary (separate) transform is performed to obtain a residual block (residual sample). The encoding device and the decoding device generate a reconstructed block based on the residual block and the predicted block, and a reconstructed picture can be generated based on the reconstructed block, as described above.

[0109] 5A-5C are diagrams illustrating selective transforms according to one embodiment of the present invention.

[0110] In this specification, a "current block" may refer to a current block or a residual block to be coded.

[0111] FIG. 5A shows an example of deriving transform coefficients through a transform.

[0112] Transform in video coding may refer to a process of transforming an input vector R based on a transform matrix to generate a transform coefficient vector C for the input vector R, as shown in FIG. 5A. The input vector R may refer to a first-order transform coefficient. Alternatively, the input vector R may refer to a residual vector, i.e., a residual sample. Meanwhile, the transform coefficient vector C may be referred to as an output vector C.

[0113] FIG. 5B shows a specific example of deriving transform coefficients through transform. FIG. 5B shows the transform process shown in FIG. 5A in detail. As described above in FIG. 3, in the non-separable second-order transform (hereinafter, referred to as 'NSST'), block data of transform coefficients obtained by applying a first-order transform is divided into MxM blocks, and MxM is calculated for each MxM block. 2 xM 2 NSST can be performed. M can be, for example, but is not limited to, 4 or 8. 2 can be N. In this case, as shown in FIG. 5B, the input vector R is divided into (first-order) transform coefficients r1 to r N The transform coefficient vector C may be a (1×N) dimensional vector including transform coefficients c1 to c N That is, the input vector R can be an (Nx1) dimensional vector including N (first-order) transform coefficients r1 to r N The input vector R may have a size of 1×N. The transform coefficient vector C may include N transform coefficients c1 to c N and the size of the transform coefficient vector C may be Nx1.

[0114] The input vector R may go through the transformation matrix to derive the transformation coefficient vector C. That is, the input vector R may be derived as the transformation coefficient vector C based on the transformation matrix.

[0115] On the other hand, the above transformation matrix is ​​made up of N basis vectors B1 to B N As shown in FIG. 5B, the basis vectors B1 to B N can be a (1xN) dimensional vector. That is, the above basis vectors B1 to B N may have a size of 1xN. A transform coefficient vector C may be generated based on the (first-order) transform coefficients of the input vector R and each of the basis vectors of the transform matrix. For example, an inner product of the input vector and each basis vector may be derived as the transform coefficient vector C.

[0116] However, the above transformations have two main issues: high computational complexity associated with the number of multiplications and additions required to generate the output vector, and memory requirements for storing the generated coefficients.

[0117] For example, the computational complexity and memory requirements for a separable transform and a non-separable transform can be derived as follows:

[0118] [Table 5]

[0119] Referring to Table 5, the memory for storing the coefficients generated through the above separation transformation is N2 The number of calculations required is 2N 3 The number of calculations indicates the computational complexity. In addition, a memory for storing coefficients generated through the non-separable transform may require N4, and the number of calculations may be N4. The number of calculations indicates the computational complexity. That is, the more the number of calculations, the higher the computational complexity may be, and the less the number of calculations, the lower the computational complexity may be.

[0120] As shown in Table 5, the memory requirements and calculation times for the non-separable transform can be significantly increased compared to the separable transform. Also, as the size of the block to which the non-separable transform is performed increases, i.e., as N increases, the mismatch between the memory requirements and calculation times for the separable transform and the non-separable transform can increase.

[0121] The non-separable transform provides better coding gain compared to the separable transform. However, as shown in Table 5, due to the computational complexity of the non-separable transform, the non-separable transform is not used in existing video coding standards. In addition, since the computational complexity of the separate transform increases as the size of the target block increases, the existing HEVC standard restricts the use of the separate transform to only target blocks whose size is 32x32 or less.

[0122] Here, the present invention proposes a selective transform. The selective transform can significantly reduce computational complexity and memory requirements, thereby increasing the efficiency of computationally intensive transform blocks and improving coding efficiency. That is, the selective transform can be used to solve the problem of computational complexity that occurs when transforming large blocks or performing non-separable transforms. The selective transform can be used for any type of transform, such as a primary transform (which can also be called a core transform), a secondary transform, etc. For example, the selective transform can be applied to the core transform for an encoding device / decoding device, and can significantly reduce encoding time / decoding time.

[0123] 5C illustrates an example of deriving transform coefficients through the selective transform, which may refer to a transform performed on a current block based on a transform matrix including basis vectors including a selective number of elements.

[0124] The simplified transformation is a method proposed based on the motivation that, among the N elements of the basis vectors of the transformation matrix, there may be duplicated or insignificant elements, and by excluding the elements, the computational complexity and memory requirements can be reduced. For example, referring to FIG. 5C, among the N elements of the basis vector B1, Z0 elements may be insignificant elements, and in this case, a truncated basis vector B1 including only N1 elements may be derived. Here, the N1 may be N-Z0. The truncated basis vector B1 may be referred to as a modified basis vector B1.

[0125] Referring to FIG. 5C, when the modified basis vector B1 is applied to the input vector R as a part of a transformation matrix, a transformation coefficient C1 can be derived. From experimental results, it is observed that the transformation coefficient C1 is the same value as the transformation coefficient C1 derived by applying the existing basis vector B1 to the input vector R as a part of a transformation matrix. That is, deriving the result assuming that the infinitesimal elements of each basis vector are 0 means that the number of required multiplications can be significantly reduced without a large difference in the result. In addition, the number of elements (i.e., elements of the transformation matrix) that must be stored for this operation can be reduced.

[0126] An association vector is proposed to define the position of an element that is not important (or meaningful) among the elements of the basis vector. The (1xN) dimensional association vector A1 can be considered to derive the N1 dimensional modified basis vector B1. That is, the 1xN sized association vector A1 can be considered to derive the 1xN1 sized modified basis vector B1 (i.e., the modified basis vector B1 including N1 elements).

[0127] Referring to FIG. 5C, the input vector R is assigned to the basis vectors B1 to B N A value derived by applying an associated vector to each of the input vectors R may be transferred to the basis vector. Through this, only some elements of the input vector R may be calculated as elements of the basis vector. Specifically, the associated vector may include 0 and 1, and among the elements of the input vector R, selected elements are multiplied by 1 and unselected elements are multiplied by 0, and only the selected elements may be passed through and transferred to the basis vector.

[0128] For example, the related vector A1 may be applied to the input vector R, and only N1 elements of the input vector R designated by the related vector A1 may be used to calculate an inner product with the basis vector B1. The inner product may indicate C1 of the transform coefficient vector C. Here, the basis vector B1 may include N1 elements, and N1 may be less than or equal to N. The above-mentioned operation of the input vector R with the related vector A1 and the basis vector B1 may be performed using the related vectors A2 to A1. N and the basis vectors B2 to B N It can also be performed on

[0129] Since the associated vectors only contain binary values, 0 and / or 1, it may be advantageous to store the associated vectors. A 0 in the associated vector may indicate that the element of the input vector R corresponding to the 0 is not transferred to the transformation matrix for the dot product calculation, and a 1 in the associated vector may indicate that the element of the input vector R corresponding to the 1 is transferred to the transformation matrix for the dot product calculation. For example, a 1xN size associated vector A may be k A k1 or A kN The above related vector A k A kn If is 0, r of the above input vector R n In other words, r of the above input vector R may not be able to pass. n is the above transformation vector B n In addition, the above A kn If is 1, then r of the above input vector R n can pass. That is, r of the above input vector R n is the above transformation vector B n and the above transformation coefficient vector C can be transferred to c n can be used in the calculations to derive

[0130] FIG. 6 shows a schematic diagram of a multiple transform technique in which the selective transforms described above are applied to a secondary transform.

[0131] Referring to Figure 6, the conversion unit may correspond to the conversion unit in the encoding device of Figure 1 described above, and the inverse conversion unit may correspond to the inverse conversion unit in the encoding device of Figure 1 described above or the inverse conversion unit in the decoding device of Figure 2.

[0132] The transform unit may perform a first-order transform based on the residual samples (residual sample array) in the residual block to derive (first-order) transform coefficients (S610). Here, the first-order transform may include the AMT described above.

[0133] When the adaptive multi-core transform is applied, a spatial-domain to frequency-domain transform may be applied to the residual signal (or the residual block) based on DCT type 2, DST type 7, DCT type 8, and / or DST type 1, to generate transform coefficients (or primary transform coefficients). Here, the primary transform coefficients may be referred to as temporary transform coefficients from the perspective of the transform unit. Also, DCT type 2, DST type 7, DCT type 8, and DST type 1 may be referred to as transform types, transform kernels, or transform cores. For reference, the DCT / DST transform types may be defined based on basis functions, and the basis functions may be shown as in Table 1 above. Specifically, the process of deriving the primary transform coefficients by applying the adaptive multi-core transform is as described above.

[0134] The transform unit may perform selective transform based on the (primary) transform coefficients to derive (secondary) transform coefficients (S620). The selective transform may refer to a transform performed on the (primary) transform coefficients for the current block based on a transform matrix including modified basis vectors and an association matrix including association vectors for the basis vectors. The modified basis vector may indicate a basis vector including N or less elements. That is, the modified basis vector may indicate a basis vector including a specific number of elements selected from the N elements. For example, the modified basis vector B n is (1xN n )-dimensional vector, n can be smaller or equal to N above. That is, the modified basis vector B n The size of is (1xN n ) size, above N n may be less than or equal to N. Here, N may be a product of the height and width of the top-left object region of the object block to which the selected transform is applied. Alternatively, N may be a total number of transform coefficients of the top-left object region of the object block to which the selected transform is applied. Meanwhile, the transform matrix including the modified basis vectors may be referred to as a modified transform matrix. The transform matrix may also be referred to as a Transform Bases Block (TBB), and the association matrix may be referred to as an Association Vectors Block (AVB).

[0135] The transform unit may perform the selective transform based on the modified transform matrix and the associated matrix to obtain (secondary) transform coefficients. The transform coefficients may be derived as quantized transform coefficients through a quantizer as described above, and may be encoded and signaled to a decoding device and transmitted to an inverse quantizer / inverse transform unit in an encoding device.

[0136] The inverse transform unit may perform a series of procedures in the reverse order of the procedures performed by the transform unit described above. The inverse transform unit may receive (dequantized) transform coefficients, perform selective (inverse) transform to derive (first-order) transform coefficients (S650), and perform first-order (inverse) transform on the (first-order) transform coefficients to obtain residual blocks (residual samples) (S660). Here, the first-order transform coefficients may be referred to as modified transform coefficients from the perspective of the inverse transform unit. As described above, the encoding device and the decoding device may generate a reconstructed block based on the residual block and the predicted block, and generate a reconstructed picture based on the reconstructed block.

[0137] Meanwhile, in one embodiment of the selective transformation, the present invention proposes a selective transformation combined with a simplifying transformation.

[0138] As used herein, a "simplification transform" may refer to a transform performed on residual samples for a current block based on a transform matrix whose size is reduced by a simplification factor.

[0139] In the simplified transformation according to an embodiment, an N dimensional vector may be mapped to an R dimensional vector located in another space to determine a simplified transformation matrix, where R is smaller than N. That is, the simplified transformation may refer to a transformation performed on a residual sample for a target block based on a simplified transform matrix including R basis vectors. Here, N may refer to the square of the length of one side of a block (or a target region) to which the transformation is applied or the total number of transform coefficients corresponding to the block (or a target region) to which the transformation is applied, and the simplification factor may refer to an R / N value. The simplification factor may be referred to by various terms such as a reduced factor, a reduction factor, a simplified factor, a simple factor, etc. Meanwhile, R may be referred to as a reduced coefficient, and the simplification factor may refer to R in some cases. In addition, the simplification factor may refer to an N / R value in some cases.

[0140] According to an embodiment, the size of the simplified transformation matrix is ​​RxN, which is smaller than the size NxN of the normal transformation matrix, and can be defined as Equation 4 below.

[0141] <Formula 4>

number

[0142] The simplified transformation matrix T RxN When the (second order) transform coefficients for the current block are multiplied by:

[0143] When the RST is applied, a simplified transform matrix of size RxN is applied to the secondary transform, so that the transform coefficients R+1 to N can be implicitly 0. In other words, when the transform coefficients of the target block are derived by applying the RST, the values ​​of the transform coefficients R+1 to N can be 0. Here, the transform coefficients R+1 to N can indicate the R+1th transform coefficient to the Nth transform coefficient among the transform coefficients. Specifically, the arrangement of the transform coefficients of the target block can be described as follows.

[0144] FIG. 7 is a diagram illustrating an arrangement of transform coefficients based on a target block according to an embodiment of the present invention. The description of transform described below with reference to FIG. 7 can be applied to the inverse transform as well. NSST based on a first transform and a simplified transform can be performed on a target block (or a residual block) 700. In one example, a 16x16 block illustrated in FIG. 7 represents a target block 700, and 4x4 blocks represented by A to P may represent subgroups of the target block 700. The first transform can be performed on the entire range of the target block 700, and after the first transform is performed, NSST can be applied to an 8x8 block (hereinafter, the target area in the upper left) consisting of subgroups A, B, E, and F. In this case, if NSST based on the simplified transform is performed, only R NSST transform coefficients (here, R means a simplified coefficient, and R is smaller than N) are derived, and therefore, the NSST transform coefficients in the range from R+1th to Nth can be determined to be 0, respectively. For example, when R is 16, 16 transform coefficients derived by performing NSST based on the simplified transform may be assigned to each block included in subgroup A, which is the upper left 4x4 block included in the upper left target area of ​​target block 700, and a transform coefficient of 0 may be assigned to each of NR blocks, i.e., 64-16=48 blocks, included in subgroups B, E, and F. Primary transform coefficients not subjected to NSST based on the simplified transform may be assigned to each block included in subgroups C, D, G, H, I, J, K, L, M, N, O, and P.

[0145] Figure 8 shows an example of deriving a transform coefficient through a transform in which the simplified transform and the selective transform are combined. Referring to Figure 8, the transform matrix may include R basis vectors, and the association matrix may include R association vectors. Here, the transform matrix including the R basis vectors may be referred to as a reduced transform matrix, and the association matrix including the R association vectors may be referred to as a reduced association matrix.

[0146] Also, each of the basis vectors may include only selected elements of the N elements. For example, referring to FIG. 8, basis vector B1 may be a 1×N1 dimensional vector including N1 elements, basis vector B2 may be a 1×N2 dimensional vector including N2 elements, and basis vector B R is N R 1xN containing elements R The above N1, N2, and N R can be a value equal to or less than N. The combined transformation of the simplifying transformation and the selective transformation can be used for any type of transformation, such as a quadratic transformation and a linear transformation.

[0147] Referring to Fig. 8, the encoding device and the decoding device may apply a transform obtained by combining the simplified transform and the selective transform to a secondary transform. For example, the selective transform may be performed based on a simplified transform matrix and a simplified associated matrix including the modified basis vectors, and (secondary) transform coefficients may be obtained. In addition, in order to reduce the computational complexity of the selective transform, a Hypercube-Givens Transform (HyGT) or the like may be used for computing the selective transform.

[0148] 9 shows an example of deriving transform coefficients through the selective transformation. In one embodiment of the selective transformation, the associated vectors A1, A2, ... A N, A1, A2, ... A2 of the association matrix may be derived in a different form. N can be derived in the same form.

[0149] Specifically, for example, the associated vectors may contain the same number of 1's. For example, if the number of 1's is M, the associated vector may contain M 1's and NM 0's. In this case, M transform coefficients among the (first-order) transform coefficients of the input vector R may be transferred to the basis vectors. Therefore, the length of the basis vectors may also be M. That is, as shown in FIG. 9, the basis vectors may contain M elements and may be (1xM)-dimensional vectors, where N1=N2=...=N N = M. The associated matrix and modified transform matrix structure illustrated in Figure 9 can be shown to be a symmetric architecture for the above selective transforms.

[0150] In another example, there may be a particular pattern of elements whose values ​​are 1, and this pattern may be repeated, rotated, and / or translated in any manner to derive the associated vector.

[0151] Meanwhile, the selective conversion described above can be applied in conjunction with other conversion techniques, not just the simplified conversion and / or the HyGT.

[0152] In addition, the present invention proposes a method of simplifying the association vector in the selective conversion. By simplifying the association vector, the storage of information for performing the selective conversion and the handling of the selective conversion can be improved. That is, the memory load for performing the selective conversion can be reduced, and the handling ability of the selective conversion can be improved.

[0153] The effect of simplifying the related vector may be more evident when the non-zero elements in the related vector have a continuous distribution. For example, the related vector A k can contain a continuous string of 1's. In this case, k is a function of two factors A ks and A kL That is, the above factor A ks and A kL Based on the above related vector A k Here, the above A ks can be a factor indicating the start of a non-zero element (e.g., 1), kL can be a factor indicating the length of a non-zero element. k can be derived as shown in the following table:

[0154] [Table 6]

[0155] Referring to Table 6, the above related vector A k can contain 16 elements. That is, the above related vector A k can be a 1x16 dimensional vector. The related vector A k A factor that indicates the start of the non-zero elements of ks The value of can be derived as 0, in which case the above factor A ks The starting point of the non-zero element is the related vector A k Also, the above related vector A k A factor indicating the length of the non-zero elements of kL The value of can be derived as 8, in which case the above factor A kLcan indicate that the length of the non-zero elements is 8. Therefore, the above A k can be derived based on the above factors as a vector with the first through eighth elements being 1 and the remaining elements being 0, as shown in Table 6.

[0156] 10 shows an example of deriving related vectors based on two factors for the related vectors to perform selective transformation. Referring to FIG. 10, a related matrix can be derived based on factors for each related vector, and transformation coefficients for a current block can be derived based on the related matrix and the modified transformation matrix.

[0157] On the other hand, the starting point of each non-zero element of the associated vector and the number of non-zero elements can be derived as fixed values, or the starting point of each non-zero element of the associated vector and the number of non-zero elements can be derived in various ways.

[0158] Also, for example, the starting point of the non-zero elements of the related vector and the number of non-zero elements can be derived based on the size of the upper left target region to be transformed. Here, the size of the upper left target region can indicate the number of transform coefficients of the upper left target region, or can indicate the product of the height and width of the upper left target region. Also, in another example, the starting point of the non-zero elements of the related vector and the number of non-zero elements can be derived based on the intra prediction mode of the target block. Specifically, for example, the starting point of the non-zero elements of the related vector and the number of non-zero elements can be derived based on whether the intra prediction mode of the target block is a non-directional intra prediction mode.

[0159] Alternatively, for example, the start point of the non-zero element of the associated vector and the number of non-zero elements can be preset. Alternatively, for example, information indicating the start point of the non-zero element of the associated vector and information indicating the number of non-zero elements can be signaled, and the associated vector can be derived based on the information indicating the start point of the non-zero element and the information indicating the number of non-zero elements. Alternatively, other information can be used instead of the information indicating the start point of the non-zero element. For example, information indicating the last position of the non-zero element can be used instead of the information indicating the start point of the non-zero element, and the associated vector can be derived based on the information indicating the last position of the non-zero element.

[0160] Meanwhile, the method of deriving the related vectors based on the factors can be applied to separable transformations and non-separable transformations such as the simplified transformation HyGT.

[0161] Fig. 11 is a schematic diagram of a video encoding method by an encoding device according to the present invention. The method disclosed in Fig. 11 can be performed by the encoding device disclosed in Fig. 1. Specifically, for example, S1100 in Fig. 11 can be performed by a subtraction unit of the encoding device, S1110 can be performed by a transformation unit of the encoding device, and S1120 can be performed by an entropy encoding unit of the encoding device. Also, although not shown, a process of deriving a predicted sample can be performed by a prediction unit of the encoding device.

[0162] The encoding apparatus derives a residual sample of a current block (S1100). For example, the encoding apparatus may determine whether to perform inter prediction or intra prediction on the current block, and may determine a specific inter prediction mode or a specific intra prediction mode based on an RD cost. Depending on the determined mode, the encoding apparatus may derive a prediction sample for the current block, and may derive the residual sample by adding an original sample for the current block and the prediction sample.

[0163] The encoding apparatus derives transform coefficients of the target block based on a selective transform of the residual samples (S1110). The selective transform may be performed based on a modified transform matrix, the modified transform matrix being a matrix including a modified basis vector, and the modified basis vector may include a selected specific number of elements among N elements. The selective transform may be performed on an upper left target region of the target block, and N may be the number of residual samples located in the upper left target region. Alternatively, N may be a value obtained by multiplying a width and a height of the upper left target region. For example, N may be 16 or 64.

[0164] The encoding device may perform a core transform on the residual samples to derive modified transform coefficients, and may perform the selective transform on the modified transform coefficients located in the upper left target area of ​​the target block based on the modified transform matrix and an association matrix including an association vector for the modified basis vector, to derive transform coefficients of the target block.

[0165] Specifically, the core transform for the residual samples may be performed as follows: The encoding apparatus may determine whether to apply an Adaptive Multiple core Transform (AMT) for the current block. In this case, an AMT flag indicating whether the adaptive multi-core transform for the current block is applied may be generated. If the AMT is not applied to the current block, the encoding apparatus may derive a DCT type 2 as a transform kernel for the current block, and may perform a transform for the residual samples based on the DCT type 2 to derive the modified transform coefficients.

[0166] When the AMT is applied to the current block, the encoding apparatus may configure a transform subset for a horizontal transform kernel and a transform subset for a vertical transform kernel, derive a horizontal transform kernel and a vertical transform kernel based on the transform subset, and perform a transform on the residual sample based on the horizontal transform kernel and the vertical transform kernel to derive modified transform coefficients. Here, the transform subset for the horizontal transform kernel and the transform subset for the vertical transform kernel may include DCT type 2, DST type 7, DCT type 8, and / or DST type 1 as candidates. In addition, transform index information may be generated, and the transform index information may include an AMT horizontal flag indicating the horizontal transform kernel and an AMT vertical flag indicating the vertical transform kernel. Meanwhile, the transform kernel may be referred to as a transform type or a transform core.

[0167] When the modified transform coefficients are derived, the encoding device may derive the transform coefficients of the target block by performing the selective transform on the modified transform coefficients located in the upper left target area of ​​the target block based on the modified transform matrix and an association matrix including an association vector for the modified basis vector. The modified transform coefficients other than the modified transform coefficient located in the upper left target area of ​​the target block may be derived as the transform coefficients of the target block as they are.

[0168] Specifically, modified transform coefficients for elements of the associated vector that are 1 among the modified transform coefficients located in the upper left target area may be derived, and transform coefficients of the target block may be derived based on the derived modified transform coefficients and the modified basis vector. Here, the associated vector for the modified basis vector may include N elements, the N elements may include elements of 1 and / or elements of 0, and the number of elements of 1 may be A. Also, the modified basis vector may include the A elements.

[0169] Meanwhile, in one example, the modified transformation matrix may include N modified basis vectors, and the association matrix may include N association vectors. The association vectors may include the same number of 1 elements, and the modified basis vectors may all include the same number of elements. Alternatively, the association vectors may not include the same number of 1 elements, and the modified basis vectors may not all include the same number of elements.

[0170] Alternatively, in another example, the modified transformation matrix may include R modified basis vectors and the association matrix may include R association vectors. The R may be a reduced coefficient and the R may be less than N. The association vectors may include the same number of 1 elements and the modified basis vectors may all include the same number of elements. Alternatively, the association vectors may not include the same number of 1 elements and the modified basis vectors may not all include the same number of elements.

[0171] Meanwhile, the related vector may be configured such that elements of 1 are continuous. In this case, in one example, information on the related vector may be entropy encoded. For example, the information on the related vector may include information indicating the start point of the elements of 1 and information indicating the number of elements of 1. Alternatively, for example, the information on the related vector may include information indicating the last position of the elements of 1 and information indicating the number of elements of 1.

[0172] In another example, the association vector may be derived based on the size of the upper left target region, for example, the starting point of one element of the association vector and the number of elements of the one element may be derived based on the size of the upper left target region.

[0173] Alternatively, in another example, the related vector may be derived based on the intra prediction mode of the current block. For example, the starting point of one element of the related vector and the number of the one element may be derived based on the intra prediction mode. Also, for example, the starting point of one element of the related vector and the number of the one element may be derived based on whether the intra prediction mode is a non-directional intra prediction mode.

[0174] The encoding device encodes information about the transform coefficients (S1330). The information about the transform coefficients may include information about the size, position, etc. of the transform coefficients. As described above, the information about the associated vectors may be entropy encoded. For example, the information about the associated vectors may include information indicating the start point of an element of 1 and information indicating the number of elements of 1. Alternatively, for example, the information about the associated vectors may include information indicating the end position of an element of 1 and information indicating the number of elements of 1.

[0175] The image information including the information about the transform coefficients and / or the information about the associated vectors may be output in the form of a bitstream. The image information may further include prediction information. The prediction information may include prediction mode information and information about motion information (e.g., when inter prediction is applied) as information related to the prediction procedure.

[0176] The output bitstream can be transmitted to a decoding device via a storage medium or a network.

[0177] Fig. 12 is a schematic diagram of an encoding device for performing a video encoding method according to the present invention. The method disclosed in Fig. 11 can be performed by the encoding device disclosed in Fig. 12. Specifically, for example, an adder unit of the encoding device of Fig. 12 can perform S1100 of Fig. 11, a transform unit of the encoding device can perform S1110, and an entropy encoding unit of the encoding device can perform S1120 to S1130. Although not shown, a process of deriving a predicted sample can be performed by a prediction unit of the encoding device.

[0178] Fig. 13 is a schematic diagram showing a video decoding method by a decoding device according to the present invention. The method disclosed in Fig. 13 can be performed by the decoding device disclosed in Fig. 2. Specifically, for example, S1300 to S1310 in Fig. 13 can be performed by an entropy decoding unit of the decoding device, S1320 can be performed by an inverse transform unit of the decoding device, and S1330 can be performed by an adder unit of the decoding device. Although not shown, the process of deriving a predicted sample can be performed by a prediction unit of the decoding device.

[0179] The decoding device derives transform coefficients of the target block from the bitstream (S1300). The decoding device may derive the transform coefficients of the target block by decoding information on the transform coefficients of the target block received through the bitstream. The received information on the transform coefficients of the target block may be represented as residual information.

[0180] The decoding apparatus derives residual samples for the target block based on a selective transform of the transform coefficients (S1310). The selective transform may be performed based on a modified transform matrix, the modified transform matrix being a matrix including a modified basis vector, and the modified basis vector may include a selected specific number of elements among N elements. The selective transform may be performed on transform coefficients located in an upper left target area of ​​the target block, and N may be the number of transform coefficients located in the upper left target area, etc. Alternatively, N may be a value obtained by multiplying a width and a height of the upper left target area. For example, N may be 16 or 64.

[0181] The decoding device can perform the selective transformation on the transformation coefficients located in the upper left target area of ​​the target block based on the modified transformation matrix and an association matrix including an association vector for the modified basis vector, to derive modified transformation coefficients.

[0182] Specifically, a transform coefficient for one element of the associated vector among the transform coefficients located in the upper left target area may be derived, and a modified transform coefficient may be derived based on the derived transform coefficient and the modified basis vector. Here, the associated vector for the modified basis vector may include N elements, the N elements may include elements of 1 and / or elements of 0, and the number of elements of 1 may be A. Also, the modified basis vector may include the A elements.

[0183] Meanwhile, in one example, the modified transformation matrix may include N modified basis vectors and the association matrix may include N association vectors. The association vectors may include the same number of 1 elements and the modified basis vectors may all include the same number of elements. Alternatively, the association vectors may not include the same number of 1 elements and the modified basis vectors may not all include the same number of elements.

[0184] Alternatively, in another example, the modified transformation matrix may include R modified basis vectors and the association matrix may include R association vectors. The R may be a reduced coefficient and the R may be less than N. The association vectors may include the same number of 1 elements and the modified basis vectors may all include the same number of elements. Alternatively, the association vectors may not include the same number of 1 elements and the modified basis vectors may not all include the same number of elements.

[0185] On the other hand, the related vector may be configured such that elements of 1 are continuous. In this case, in one example, information on the related vector may be acquired from a bitstream, and the related vector may be derived based on the information on the related vector. For example, the information on the related vector may include information indicating the start point of the element of 1 and information indicating the number of elements of 1. Alternatively, for example, the information on the related vector may include information indicating the last position of the element of 1 and information indicating the number of elements of 1.

[0186] Alternatively, in another example, the association vector may be derived based on the size of the upper left target region, for example, the starting point of one element of the association vector and the number of the one elements may be derived based on the size of the upper left target region.

[0187] Alternatively, in another example, the related vector may be derived based on the intra prediction mode of the current block. For example, the starting point of one element of the related vector and the number of the one element may be derived based on the intra prediction mode. Also, for example, the starting point of one element of the related vector and the number of the one element may be derived based on whether the intra prediction mode is a non-directional intra prediction mode.

[0188] If the modified transform coefficients are derived, the decoding device may perform a core transform on the current block including the modified transform coefficients to derive the residual samples.

[0189] The core transform for the target block may be performed as follows: A decoding device may acquire an Adaptive Multiple core Transform (AMT) flag indicating whether an AMT is applied from a bitstream, and if the value of the AMT flag is 0, the decoding device may derive a DCT type 2 as a transform kernel for the target block, and may perform an inverse transform for the target block including the modified transform coefficients based on the DCT type 2 to derive the residual samples.

[0190] When the value of the AMT flag is 1, the decoding apparatus may configure a transform subset for a horizontal transform kernel and a transform subset for a vertical transform kernel, derive a horizontal transform kernel and a vertical transform kernel based on the transform index information acquired from the bitstream and the transform subset, and perform an inverse transform on the target block including the modified transform coefficients based on the horizontal transform kernel and the vertical transform kernel to derive the residual sample. Here, the transform subset for the horizontal transform kernel and the transform subset for the vertical transform kernel may include DCT type 2, DST type 7, DCT type 8, and / or DST type 1 as candidates. In addition, the transform index information may include an AMT horizontal flag indicating one of the candidates included in the transform subset for the horizontal transform kernel and an AMT vertical flag indicating one of the candidates included in the transform subset for the vertical transform kernel. Meanwhile, the transform kernel may be referred to as a transform type or a transform core.

[0191] The decoding apparatus generates a reconstructed picture based on the residual sample (S1320). The decoding apparatus may generate a reconstructed picture based on the residual sample. For example, the decoding apparatus may perform inter prediction or intra prediction on a current block based on prediction information received through a bitstream to derive a prediction sample, and may generate the reconstructed picture by adding the prediction sample and the residual sample. As described above, in-loop filtering procedures such as deblock filtering, SAO and / or ALF procedures may be applied to the reconstructed picture as necessary to improve subjective / objective image quality.

[0192] Figure 14 is a schematic diagram of a decoding device performing a video decoding method according to the present invention. The method disclosed in Figure 13 can be performed by the decoding device disclosed in Figure 14. Specifically, for example, the entropy decoding unit of the decoding device of Figure 14 can perform S1300 of Figure 13, the inverse transform unit of the decoding device of Figure 14 can perform S1310 of Figure 13, and the adder unit of the decoding device of Figure 16 can perform S1320 of Figure 15. Although not shown, the process of deriving a predicted sample can be performed by a prediction unit of the decoding device of Figure 14.

[0193] According to the present invention described above, the amount of data that must be transmitted for residual processing can be reduced through efficient conversion, thereby improving residual coding efficiency.

[0194] In addition, according to the present invention, a non-separable transform can be performed based on a transform matrix composed of basis vectors including a selected specific number of elements, thereby reducing memory load and computational complexity for the non-separable transform and improving residual coding efficiency.

[0195] In addition, according to the present invention, a non-separable transform can be performed based on a transform matrix having a simplified structure, thereby reducing the amount of data that must be transmitted for residual processing and improving residual coding efficiency.

[0196] In the above embodiments, the method is described based on a sequence (flowchart) of a series of steps or blocks, but the present invention is not limited to the sequence of steps, and certain steps may occur in different steps and different sequences than those described above, or simultaneously. In addition, those skilled in the art will understand that the steps shown in the flow chart are not exclusive, and other steps may be included, or one or more steps of the flow chart may be deleted without affecting the scope of the present invention.

[0197] The above-described method according to the present invention can be embodied in software form, and the encoding device and / or decoding device according to the present invention can be included in an image processing device such as a TV, a computer, a smartphone, a set-top box, or a display device.

[0198] In the present invention, when an embodiment is implemented in software, the above-described method may be implemented as a module (process, function, etc.) performing the above-described function. The module may be stored in a memory and executed by a processor. The memory may be internal or external to the processor and may be coupled to the processor by various well-known means. The processor may include an Application-Specific Integrated Circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory may include Read-Only Memory (ROM), Random Access Memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. That is, the embodiments described in the present invention may be implemented and performed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each drawing may be implemented and performed on a computer, processor, microprocessor, controller, or chip.

[0199] In addition, the decoding device and encoding device to which the present invention is applied may be included in a multimedia broadcast transmitting / receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video interactive device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a video on demand (VoD) service providing device, an over-the-top video (OTT) device, an internet streaming service providing device, a three-dimensional (3D) video device, an image telephone video device, and a medical video device, and may be used to process a video signal or a data signal. For example, an over-the-top video (OTT) device may include a game console, a Blu-ray player, an internet-connected TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), and the like.

[0200] In addition, the processing method to which the present invention is applied may be produced in the form of a program executed by a computer and may be stored in a computer-readable recording medium. In addition, multimedia data having a data structure according to the present invention may also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. In addition, the computer-readable recording medium may include a medium embodied in the form of a carrier wave (e.g., a transmission through the Internet). In addition, a bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network. In addition, an embodiment of the present invention may be embodied in a computer program product by a program code, and the program code may be executed by a computer according to an embodiment of the present invention. The program code may be stored on a computer-readable carrier.

[0201] Furthermore, the content streaming system to which the present invention is applied can broadly include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0202] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, or camcorder into digital data to generate a bitstream and transmits the bitstream to the streaming server. In another example, when a multimedia input device such as a smartphone, camera, or camcorder directly generates a bitstream, the encoding server may be omitted. The bitstream may be generated by an encoding method or a bitstream generation method to which the present invention is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0203] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server serves as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server controls commands / responses between devices in the content streaming system.

[0204] The streaming server may receive the content from a media storage and / or an encoding server. For example, when the content is received from the encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a certain period of time in order to provide a smooth streaming service.

[0205] Examples of the user device include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistant), a PMP (Portable Multimedia Player), a navigation system, a slate PC, a tablet PC, an ULTRABOOK (registered trademark), a wearable device (e.g., a smartwatch, smart glass, a Head Mounted Display (HMD)), a digital TV, a desktop computer, a digital signage, etc. Each server in the content streaming system can be operated as a distributed server, and in this case, data received by each server can be processed in a distributed manner.

Claims

1. A video decoding method performed by a decoding device, comprising: obtaining prediction mode information and information about transform coefficients of a current block from a bitstream; deriving a prediction sample of the current block based on the prediction mode information; deriving transform coefficients for the current block based on the information about the transform coefficients; deriving residual samples for the current block based on a non-separable transform on the transform coefficients; generating reconstructed samples based on the prediction samples and the residual samples; The prediction samples are derived based on intra prediction for the current block; The non-separable transformation is performed based on a transformation matrix; the transformation matrix is ​​a matrix containing modified basis vectors; the modified basis vectors contain fewer than N elements; The N is equal to the number of transform coefficients located in the region to which the non-separable transform is applied in the current block, the region to which the non-separable transform is applied is an 8x8 top left region of the current block; A video decoding method, wherein N is equal to 64.

2. A video encoding method performed by an encoding device, comprising: deriving a prediction sample for a current block; deriving residual samples for the current block based on the prediction samples; deriving transform coefficients for the current block based on a non-separable transform on the residual samples; generating prediction mode information relating to a prediction mode used to derive the prediction samples; generating information about the transform coefficients; encoding the prediction mode information and the information about the transform coefficients, The prediction samples are derived based on intra prediction for the current block; The non-separable transformation is performed based on a transformation matrix; the transformation matrix is ​​a matrix containing modified basis vectors; the modified basis vectors contain fewer than N elements; The N is equal to the number of transform coefficients located in the region to which the non-separable transform is applied in the current block, the region to which the non-separable transform is applied is an 8x8 top left region of the current block; A video encoding method, wherein N is equal to 64.

3. A method for transmitting data including a bitstream relating to video, comprising the steps of: Obtaining the bitstream relating to the video, the bitstream comprising: deriving a prediction sample for a current block; deriving residual samples for the current block based on the prediction samples; deriving transform coefficients for the current block based on a non-separable transform on the residual samples; generating prediction mode information relating to a prediction mode used to derive the prediction samples; generating information about the transform coefficients; encoding the prediction mode information and the information about the transform coefficients; transmitting the data including the bitstream; The prediction samples are derived based on intra prediction for the current block; The non-separable transformation is performed based on a transformation matrix; the transformation matrix is ​​a matrix containing modified basis vectors; the modified basis vectors contain fewer than N elements; The N is equal to the number of transform coefficients located in the region to which the non-separable transform is applied in the current block, the region to which the non-separable transform is applied is an 8x8 top left region of the current block; The method, wherein N is equal to 64.

Images