Method and apparatus for non-separable linear transform-based video coding

Abstract

This embodiment discloses a non-separable primary transform-based video coding method and apparatus. In this embodiment, a video decoding apparatus obtains dequantized transform coefficients for a transform block of a current block and decodes a non-separable primary transform (NSPT) flag, which indicates whether a non-separable primary transform is applied. The video decoding apparatus checks the NSPT flag, and if the NSPT flag is true, determines a non-separable primary inverse transform kernel based on the size of the transform block, the intra prediction mode of the current block, and characteristics of the dequantized transform coefficients. The video decoding apparatus generates a residual signal by applying the non-separable primary inverse transform kernel to the transform coefficients.

Description

[Technical Field]

[0001] The present disclosure relates to a method and apparatus for non-separable linear transform-based video coding. [Background technology]

[0002] The following disclosure merely provides background information related to the present invention and may not constitute prior art.

[0003] Video data has a larger amount of data than audio data or still image data, and therefore requires a large amount of hardware resources, including memory, to store or transmit the video data without compression processing.

[0004] Therefore, when video data is typically stored or transmitted, it is compressed using an encoder and then stored or transmitted, and a decoder receives the compressed video data, decompresses it, and plays it back. Such video compression technologies include H.264 / AVC, High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC), which has improved coding efficiency by about 30% or more compared to HEVC.

[0005] However, as video size, resolution, and frame rate gradually increase, the amount of data to be encoded also increases accordingly. Therefore, there is a demand for new compression technologies that have better encoding efficiency and a greater effect of improving image quality than existing compression technologies.

[0006] The Non-separable Primary Transform (NSPT) technology performs a non-separable primary transform using a pre-trained transform kernel instead of performing vertical and horizontal transforms on the residual signal of a block predicted in intra prediction mode. tb ×H tb, the size of the non-separable linear transformation kernel K is (W tb ×H tb )×(W tb ×H tb ) That is, W tb ×H tb The residual signals are transformed into W tb ×H tb transform coefficients are generated.

[0007] For non-separable linear transforms, transform kernels are defined according to the block shape and intra prediction mode. For example, by utilizing the symmetry property of square blocks, there are 35 transform kernels for square blocks depending on the intra prediction mode. Furthermore, there are 67 transform kernels for rectangular blocks.

[0008] A non-separable linear transform is applied to the luma component. Also, when a non-separable linear transform is applied, a secondary transform (i.e., a low-frequency non-separable transform (LFNST)) may not be performed. Therefore, in order to improve video coding efficiency and video quality, improvements to the non-separable linear transform and comprehensive operation methods need to be considered. Summary of the Invention [Problem to be solved by the invention]

[0009] The present disclosure provides a video coding method and apparatus for performing a non-separable primary transform (NSPT) based on an intra prediction mode of a current block, a size of a transform block, and characteristics of transform coefficients. Furthermore, the video coding method and apparatus according to the present disclosure perform a non-separable primary transform using implicit partitioning for a large transform block to which a non-separable transform cannot be applied. [Means for solving the problem]

[0010] According to an embodiment of the present disclosure, a method for restoring a current block performed by a video decoding device includes the steps of: obtaining inverse-quantized transform coefficients for a transform block of the current block; decoding an NSPT (Non-separable Primary Transform) flag from a bitstream, where the NSPT flag indicates whether a non-separable primary transform is applied; and checking the NSPT flag. If the NSPT flag is true, the method includes the steps of determining a non-separable primary inverse transform kernel based on a size of the transform block, an intra prediction mode of the current block, and characteristics of the inverse-quantized transform coefficients; and applying the non-separable primary inverse transform kernel to the transform coefficients to generate a residual signal, thereby performing a primary inverse transform.

[0011] According to another embodiment of the present disclosure, a method for encoding a current block performed by a video encoding device is provided, comprising the steps of: obtaining a residual signal for a transform block of the current block; determining a non-separable primary transform kernel based on the size of the transform block, the intra prediction mode of the current block, and characteristics of the quantized transform coefficients; applying the non-separable primary transform kernel to the residual signal to generate first primary transform coefficients; explicitly or implicitly determining vertical and horizontal primary transform kernel pairs for the transform block; and applying the primary transform kernel pair to the residual signal to generate second primary transform coefficients.

[0012] According to another embodiment of the present disclosure, there is provided a computer-readable recording medium for storing a bitstream generated by a video encoding method, wherein the video encoding method includes the steps of: obtaining a residual signal for a transform block of a current block; determining a non-separable primary transform kernel based on the size of the transform block, the intra prediction mode of the current block, and characteristics of the quantized transform coefficients; applying the non-separable primary transform kernel to the residual signal to generate first primary transform coefficients; explicitly or implicitly determining vertical and horizontal primary transform kernel pairs for the transform block; and applying the primary transform kernel pair to the residual signal to generate second primary transform coefficients. [Effects of the Invention]

[0013] As described above, according to this embodiment, a video coding method and apparatus are provided that perform non-separable linear transformation based on the intra prediction mode of the current block, the size of the transformation block, and the characteristics of the transformation coefficients, thereby improving video coding efficiency and video quality. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is an exemplary block diagram of a video encoding device capable of implementing the techniques of this disclosure. [Figure 2] FIG. 2 is a diagram for explaining a method of dividing a block using a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure. [Figure 3a] FIG. 3a is a diagram illustrating multiple intra-prediction modes, including a wide-angle intra-prediction mode. [Figure 3b] FIG. 3b is a diagram illustrating multiple intra-prediction modes, including a wide-angle intra-prediction mode. [Figure 4] FIG. 4 is an example diagram of neighboring blocks of a current block. [Figure 5] FIG. 5 is an exemplary block diagram of a video decoding device capable of implementing the techniques of this disclosure. [Figure 6] FIG. 6 is a block diagram showing in detail a portion of a video decoding device according to one embodiment of the present disclosure. [Figure 7] FIG. 7 is an exemplary diagram illustrating a method for determining an inverse transform kernel according to an embodiment of the present disclosure. [Figure 8] FIG. 8 is an exemplary diagram illustrating a method for determining an inverse transform kernel according to another embodiment of the present disclosure. [Figure 9] FIG. 9 is an exemplary diagram illustrating division of a transform block according to one embodiment of the present disclosure. [Figure 10] FIG. 10 is an exemplary diagram illustrating division of a transform block according to another embodiment of the present disclosure. [Figure 11] FIG. 11 is an exemplary diagram illustrating a restoration order of sub-blocks according to an embodiment of the present disclosure. [Figure 12a] FIG. 12a is an exemplary diagram illustrating an inverse transform process of reconstructed transform coefficients according to one embodiment of the present disclosure. [Figure 12b] FIG. 12b is an exemplary diagram illustrating an inverse transform process of the reconstructed transform coefficients according to one embodiment of the present disclosure. [Figure 13] FIG. 13 is an example diagram illustrating a scan order for generating a primary transform coefficient vector according to one embodiment of the present disclosure. [Figure 14] FIG. 14 is an example diagram illustrating a scan order for generating a primary transform coefficient vector according to another embodiment of the present disclosure. [Figure 15] FIG. 15 is an example diagram illustrating a scan order for generating a primary transform coefficient vector according to one embodiment of the present disclosure. [Figure 16a] FIG. 16a is a flowchart illustrating a method for a video encoding device to transform a transform block according to one embodiment of the present disclosure. [Figure 16b] FIG. 16b is a flowchart illustrating a method for a video encoding device to transform a transform block according to one embodiment of the present disclosure. [Figure 17]FIG. 17 is a flowchart illustrating a method in which a video decoding apparatus inversely transforms a transform block according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. When assigning reference numerals to components in each drawing, it should be noted that the same reference numerals are used to designate identical components even when they appear in different drawings. Furthermore, when describing the present embodiment, if a detailed description of related known structures or functions is deemed to obscure the gist of the present embodiment, the detailed description will be omitted.

[0016] 1 is an exemplary block diagram of a video encoding device capable of implementing the technology of the present disclosure. The video encoding device and its sub-components will be described below with reference to the diagram of FIG.

[0017] The video encoding device is configured to include a picture division unit 110, a prediction unit 120, a subtractor 130, a transform unit 140, a quantization unit 145, a rearrangement unit 150, an entropy encoding unit 155, an inverse quantization unit 160, an inverse transform unit 165, an adder 170, a loop filter unit 180, and a memory 190.

[0018] Each component of the video encoding device may be implemented in hardware or software, or a combination of hardware and software. Furthermore, the function of each component may be implemented in software, and a microprocessor may execute the software function corresponding to each component.

[0019] A video consists of one or more sequences, each containing multiple pictures. Each picture is divided into multiple regions, and coding is performed for each region. For example, a picture is divided into one or more tiles and / or slices. Here, one or more tiles are defined as a tile group. Each tile and / or slice is divided into one or more coding tree units (CTUs). Each CTU is then divided into one or more coding units (CUs) according to a tree structure. Information applicable to each CU is coded as CU syntax, and information commonly applied to all CUs included in a CTU is coded as CTU syntax. Furthermore, information commonly applied to all blocks in a slice is coded as slice header syntax, and information commonly applied to all blocks constituting one or more pictures is coded in a picture parameter set (PPS) or picture header. Furthermore, information commonly referenced by multiple pictures is coded as a sequence parameter set (SPS). Information commonly referenced by one or more SPSs is coded into a Video Parameter Set (VPS). Furthermore, information commonly applied to one tile or tile group may be coded as syntax in a tile or tile group header. The syntax included in the SPS, PPS, slice header, tile, or tile group header is called high-level syntax.

[0020] The picture division unit 110 determines the size of a coding tree unit (CTU). Information about the size of the CTU (CTU size) is coded as an SPS or PPS syntax and transmitted to a video decoding device.

[0021] The picture division unit 110 divides each picture constituting a video into a plurality of CTUs (Coding Tree Units) having a predetermined size, and then recursively divides the CTUs using a tree structure, in which leaf nodes become CUs (Coding Units), which are basic units of coding.

[0022] The tree structure may be a quad tree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a binary tree (BT) in which an upper node is divided into two lower nodes, a ternary tree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that combines two or more of the QT, BT, and TT structures. For example, a QTBT (Quad Tree plus Binary Tree) structure or a QTBTTT (Quad Tree plus Binary Tree Ternary Tree) structure may be used. Here, BTTT and TT are collectively referred to as MTT (Multiple-Type Tree).

[0023] FIG. 2 is a diagram for explaining a method for dividing a block using the QTBTTT structure.

[0024] As shown in FIG. 2, a CTU is first split into a QT structure. The quadtree splitting is repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of a leaf node allowed in QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes in a lower layer is coded by the entropy coding unit 155 and signaled to the video decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of a root node allowed in BT, it is further split into one or more BT structures or TT structures. The BT structure and / or TT structure can have multiple splitting directions. For example, the block of the corresponding node can be split horizontally or vertically. As shown in Figure 2, when MTT splitting is initiated, a second flag mtt_split_flag indicating whether the node has been split, and if split, a flag indicating the split direction (vertical or horizontal) and / or a flag indicating the split type (binary or ternary) are additionally coded by the entropy coding unit 155 and signaled to the video decoding device.

[0025] Alternatively, before encoding the first flag QT_split_flag indicating whether each node is split into four nodes in the lower layer, the CU split flag split_cu_flag indicating whether the node is split may be encoded. If the CU split flag split_cu_flag value indicates no split, the block of the corresponding node becomes a leaf node in the partition tree structure and becomes a coding unit (CU), which is the basic unit of coding. If the CU split flag split_cu_flag value indicates split, the video encoding device starts encoding from the first flag in the above-mentioned manner.

[0026] When QTBT is used as another example of a tree structure, there are two types: a type in which the block of a corresponding node is split horizontally into two blocks of the same size (i.e., symmetric horizontal splitting) and a type in which the block of a corresponding node is split vertically (i.e., symmetric vertical splitting). A split flag split_flag indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are coded by the entropy coding unit 155 and transmitted to the video decoding device. Meanwhile, there may also be an additional type in which the block of a corresponding node is split into two blocks of an asymmetric shape. The asymmetric shape may include a type in which the block of a corresponding node is split into two rectangular blocks with a size ratio of 1:3, or a type in which the block of a corresponding node is split diagonally.

[0027] CUs have various sizes depending on the QTBT or QTBTTT division from the CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of QTBTTT) is referred to as a "current block." Depending on the QTBTTT division used, the shape of the current block may be rectangular as well as square.

[0028] The prediction unit 120 predicts the current block to generate a predicted block. The prediction unit 120 includes an intra prediction unit 122 and an inter prediction unit .

[0029] Generally, each current block in a picture is predictively coded. Generally, prediction of the current block is performed using intra-prediction techniques (using data from the picture containing the current block) or inter-prediction techniques (using data from a picture coded before the picture containing the current block). Inter-prediction includes both unidirectional prediction and bidirectional prediction.

[0030] The intra prediction unit 122 predicts pixels in a current block using pixels (reference pixels) located around the current block in a current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in FIG. 3A, the multiple intra prediction modes include two non-directional modes, including a planar mode and a DC mode, and 65 directional modes. Different neighboring pixels and calculation formulas are defined for each prediction mode.

[0031] For efficient directional prediction of a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dashed arrows in FIG. 3b are additionally used. These are called "wide-angle intra-prediction modes." In FIG. 3b, the arrows indicate the corresponding reference samples used for prediction, not the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. A wide-angle intra-prediction mode is a mode that performs prediction in the opposite direction to a specific directional mode without additional bit transmission when the current block is rectangular. In this case, some wide-angle intra-prediction modes available for the current block are determined according to the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) with angles less than 45 degrees are available when the current block has a rectangular shape with its height smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) with angles greater than -135 degrees are available when the current block has a rectangular shape with its width greater than its height.

[0032] The intra predictor 122 determines the intra prediction mode to use to encode the current block. In some examples, the intra predictor 122 may encode the current block using various intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, the intra predictor 122 may calculate a bit rate-distortion value using a bit rate-distortion analysis for the various tested intra prediction modes and select the intra prediction mode with the best bit rate-distortion characteristics from the tested modes.

[0033] The intra prediction unit 122 selects one intra prediction mode from among a plurality of intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an arithmetic equation determined by the selected intra prediction mode. Information about the selected intra prediction mode is coded by the entropy coding unit 155 and transmitted to the video decoding device.

[0034] The inter prediction unit 124 generates a prediction block for a current block using a motion compensation process. The inter prediction unit 124 searches for a block most similar to the current block in a reference picture that has been coded and decoded prior to the current picture, and generates a prediction block for the current block using the searched block. The inter prediction unit 124 then generates a motion vector (MV) corresponding to the displacement between the current block in the current picture and the prediction block in the reference picture. Generally, motion estimation is performed on the luma component, and a motion vector calculated based on the luma component is used for both the luma component and the chroma component. Motion information including information about the reference picture used to predict the current block and information about the motion vector is coded by the entropy coding unit 155 and transmitted to the video decoding device.

[0035] The inter prediction unit 124 may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, a sub-sample between two consecutive integer samples is interpolated by applying a filter coefficient to a plurality of consecutive integer samples including the two integer samples. When a step of searching for a block most similar to the current block in the interpolated reference picture is performed, a motion vector is expressed with decimal precision rather than integer sample precision. The precision or resolution of the motion vector is set differently for each target region to be coded, such as a slice, tile, CTU, or CU. When such adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution applied to each target region must be signaled for each target region. For example, if the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution indicates the precision of a differential motion vector, which will be described later.

[0036] Meanwhile, the inter prediction unit 124 performs inter prediction using bi-prediction. In bi-prediction, two reference pictures and two motion vectors representing the position of a block most similar to the current block within each reference picture are used. The inter prediction unit 124 selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and generates a first reference block and a second reference block by searching for a block similar to the current block within each reference picture. Then, the inter prediction unit 124 averages or weighted-averages the first reference block and the second reference block to generate a predicted block for the current block. Then, the inter prediction unit 124 transmits motion information, including information about the two reference pictures used to predict the current block and information about the two motion vectors, to the encoding unit 150. Here, reference picture list 0 is composed of pictures that are before the current picture in display order among the previously reconstructed pictures, and reference picture list 1 is composed of pictures that are after the current picture in display order among the previously reconstructed pictures. However, this is not necessarily limited thereto, and previously reconstructed pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, previously reconstructed pictures that are before the current picture may be additionally included in reference picture list 1.

[0037] Various methods are used to minimize the amount of bits required to encode the motion information.

[0038] For example, if the reference picture and motion vector of the current block are the same as those of a neighboring block, the motion information of the current block can be transmitted to the video decoding device by encoding information that identifies the neighboring block. This method is called a "merge mode."

[0039] In the merge mode, the inter prediction unit 124 selects a predetermined number of merge candidate blocks (hereinafter referred to as "merge candidates") from the neighboring blocks of the current block.

[0040] As shown in FIG. 4, all or some of the neighboring blocks for deriving merging candidates are used, which are the left block A0, the bottom left block A1, the top block B0, the top right block B1, and the top left block A2 adjacent to the current block in the current picture. Furthermore, blocks located in a reference picture (which may be the same as or different from the reference picture used to predict the current block) rather than the current picture in which the current block is located may be used as merging candidates. For example, a block co-located with the current block in the reference picture or a block adjacent to the co-located block may be additionally used as a merging candidate. If the number of merging candidates selected by the above-described method is less than a predetermined number, a zero vector is added to the merging candidates.

[0041] The inter prediction unit 124 generates a merge list including a predetermined number of merge candidates using the neighboring blocks. The inter prediction unit 124 selects a merge candidate to be used as motion information for the current block from among the merge candidates included in the merge list, and generates merge index information for identifying the selected candidate. The generated merge index information is encoded by the encoding unit 150 and transmitted to the video decoding device.

[0042] Merge skip mode is a special case of merge mode, in which, after quantization, when all the transform coefficients for entropy coding are close to zero, only neighboring block selection information is transmitted without transmitting a residual signal. By using merge skip mode, relatively high coding efficiency can be achieved for low-motion images, still images, screen content images, etc.

[0043] Hereinafter, merge mode and merge skip mode will be collectively referred to as merge / skip mode.

[0044] Another method for encoding motion information is the Advanced Motion Vector Prediction (AMVP) mode.

[0045] In AMVP mode, the inter prediction unit 124 derives motion vector predictor candidates for the motion vector of the current block using neighboring blocks of the current block. The neighboring blocks used to derive the motion vector predictor candidates include all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1, and the upper left block A2 adjacent to the current block in the current picture shown in FIG. 4. Furthermore, blocks located in a reference picture (which may be the same as or different from the reference picture used to predict the current block) rather than the current picture in which the current block is located may be used as neighboring blocks used to derive the motion vector predictor candidates. For example, a block collocated with the current block in the reference picture or a block adjacent to the collocated block may be used. If the number of motion vector candidates obtained by the above-described method is less than a predetermined number, a zero vector is added to the motion vector candidates.

[0046] The inter prediction unit 124 derives motion vector predictor candidates using the motion vectors of the surrounding blocks, determines a motion vector predictor for the motion vector of the current block using the motion vector predictor candidates, and calculates a motion vector difference by subtracting the motion vector predictor from the motion vector of the current block.

[0047] The motion vector predictor is obtained by applying a predefined function (e.g., median, average, etc.) to motion vector predictor candidates. In this case, the video decoder also knows the predefined function. In addition, since the neighboring blocks used to derive the motion vector predictor candidates are blocks that have already been encoded and decoded, the video decoder also already knows the motion vectors of the neighboring blocks. Therefore, the video encoder does not need to encode information for identifying the motion vector predictor candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

[0048] Alternatively, the motion vector predictor may be determined by selecting one of the motion vector predictor candidates, in which case information for identifying the selected motion vector predictor candidate is additionally coded along with information about the differential motion vector and information about the reference picture used to predict the current block.

[0049] The subtractor 130 subtracts the predicted block generated by the intra predictor 122 or the inter predictor 124 from the current block to generate a residual block.

[0050] The transform unit 140 transforms a residual signal in a residual block having pixel values ​​in the spatial domain into transform coefficients in the frequency domain. The transform unit 140 may transform the residual signal in the residual block using the entire size of the residual block as a transform unit, or may divide the residual block into a plurality of sub-blocks and transform each of the sub-blocks as a transform unit. Alternatively, the transform unit 140 may divide the residual block into two sub-blocks, a transform domain and a non-transform domain, and transform the residual signal using only the transform domain sub-block as a transform unit. Here, the transform domain sub-block is one of two rectangular blocks having a size ratio of 1:1 along the horizontal axis (or vertical axis). In this case, a flag cu_sbt_flag indicating that only the sub-block has been transformed, directionality information cu_sbt_horizontal_flag, and / or position information cu_sbt_pos_flag are coded by the entropy coding unit 155 and signaled to the video decoding device. In addition, the size of the transform domain sub-block has a size ratio of 1:3 based on the horizontal axis (or vertical axis), and in this case, a flag cu_sbt_quad_flag that distinguishes the corresponding division is additionally coded by the entropy coding unit 155 and signaled to the video decoding device.

[0051] Meanwhile, the transform unit 140 performs separate transforms on the residual block in the horizontal and vertical directions. Various types of transform functions or transform matrices are used for the transform. For example, a pair of transform functions for a horizontal transform and a vertical transform is defined as an MTS (Multiple Transform Set). The transform unit 140 selects one transform function pair with the highest transform efficiency from the MTS, and transforms the residual block in the horizontal and vertical directions, respectively. Information mts_idx about the transform function pair selected from the MTS is coded by the entropy coding unit 155 and signaled to the video decoding device.

[0052] The quantization unit 145 quantizes the transform coefficients output from the transform unit 140 using a quantization parameter and outputs the quantized transform coefficients to the entropy coding unit 155. For any block or frame, the quantization unit 145 may directly quantize an associated residual block without transforming it. The quantization unit 145 may apply different quantization coefficients (scaling values) depending on the position of the transform coefficients within the transform block. The quantization matrix applied to the two-dimensionally arranged quantized transform coefficients is coded and signaled to a video decoding device.

[0053] The reordering unit 150 performs reordering of coefficient values ​​for the quantized residual values.

[0054] The reordering unit 150 converts a two-dimensional coefficient array into a one-dimensional coefficient sequence using coefficient scanning. For example, the reordering unit 150 scans from a DC coefficient to a high-frequency coefficient using a zig-zag scan or a diagonal scan to output a one-dimensional coefficient sequence. Depending on the size of the transform unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block configuration coefficients in the row direction may be used instead of the zig-zag scan. That is, the scan method to be used may be determined from among a staggered scan, a diagonal scan, a vertical scan, and a horizontal scan depending on the size of the transform unit and the intra prediction mode.

[0055] The entropy coding unit 155 generates a bitstream by encoding the sequence of one-dimensional quantized transform coefficients output from the reordering unit 150 using various coding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0056] The entropy encoder 155 also encodes information related to block partitioning, such as a CTU size, a CU partition flag, a QT partition flag, an MTT partition type, and an MTT partition direction, so that a video decoder can partition blocks in the same manner as a video encoder. The entropy encoder 155 also encodes information related to a prediction type, indicating whether the current block has been coded using intra prediction or inter prediction, and encodes intra prediction information (i.e., information related to an intra prediction mode) or inter prediction information (information related to a motion information coding mode (merge mode or AMVP mode), a merge index in the merge mode, and information related to a reference picture index and a differential motion vector in the AMVP mode) according to the prediction type. The entropy encoder 155 also encodes information related to quantization, i.e., information related to a quantization parameter and information related to a quantization matrix.

[0057] The inverse quantization unit 160 generates transform coefficients by inverse quantizing the quantized transform coefficients output from the quantization unit 145. The inverse transform unit 165 transforms the transform coefficients output from the inverse quantization unit 160 from the frequency domain to the spatial domain to reconstruct residual blocks.

[0058] The adder 170 reconstructs the current block by adding the reconstructed residual block and the predicted block generated by the predictor 120. The pixels in the reconstructed current block are used as reference pixels when intra-predicting the next block.

[0059] The loop filter unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transform / quantization. The filter unit 180 includes all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186 as in-loop filters.

[0060] The deblocking filter 182 filters boundaries between reconstructed blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter 184 and alf 186 perform additional filtering on the deblock-filtered image. The SAO filter 184 and alf 186 are filters used to compensate for differences between reconstructed pixels and original pixels caused by lossy coding. The SAO filter 184 applies an offset in units of CTUs to improve not only subjective image quality but also coding efficiency. In contrast, the ALF 186 performs block-based filtering and compensates for distortion by applying different filters to distinguish edges and degree of change of the corresponding block. Information about the filter coefficients used in the ALF is coded and signaled to the video decoder.

[0061] The reconstructed blocks filtered through the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in a memory 190. Once all the blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter-predicting blocks in a picture to be coded later.

[0062] 5 is an exemplary block diagram of a video decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the video decoding device and its sub-components will be described with reference to FIG.

[0063] The video decoding apparatus includes an entropy decoding unit 510 , a reordering unit 515 , an inverse quantization unit 520 , an inverse transform unit 530 , a prediction unit 540 , an adder 550 , a loop filter unit 560 , and a memory 570 .

[0064] 1, each component of the video decoding apparatus may be implemented in hardware or software, or a combination of hardware and software. Furthermore, the function of each component may be implemented in software, with a microprocessor executing the software function corresponding to each component.

[0065] The entropy decoding unit 510 decodes the bitstream generated by the video encoding device and extracts information related to block division to determine the current block to be decoded, and extracts prediction information and information related to the residual signal required to restore the current block.

[0066] The entropy decoding unit 510 extracts information about the CTU size from the SPS (Sequence Parameter Set) or PPS (Picture Parameter Set) to determine the size of the CTU, and divides the picture into CTUs of the determined size.The entropy decoding unit 510 then determines the CTU as the top layer of a tree structure, i.e., the root node, and extracts division information about the CTU, thereby dividing the CTU using the tree structure.

[0067] For example, when splitting a CTU using the QTBTTT structure, first, the first flag QT_split_flag related to QT splitting is extracted, and each node is split into four nodes in the lower layer. Then, for nodes corresponding to QT leaf nodes, the second flag MTT_split_flag related to MTT splitting and split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf node is split into an MTT structure. As a result, each node below the QT leaf node is recursively split into a BT or TT structure.

[0068] As another example, when splitting a CTU using the QTBTTT structure, a CU split flag "split_cu_flag" indicating whether the CU can be split may be extracted first, and if the corresponding block is split, a first flag "QT_split_flag" may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo immediate MTT splits, or conversely, may undergo only multiple QT splits.

[0069] As another example, when splitting a CTU using the QTBT structure, the first flag QT_split_flag related to QT splitting is extracted and each node is split into four nodes in the lower layer. Then, for nodes corresponding to QT leaf nodes, the split flag split_flag indicating whether the node is further split by BT and split direction information are extracted.

[0070] Meanwhile, after determining the current block to be decoded using the tree structure division, the entropy decoding unit 510 extracts information regarding the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit 510 extracts syntax elements regarding intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit 510 extracts syntax elements regarding inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

[0071] The entropy decoding unit 510 also extracts information related to quantization and information related to the quantized transform coefficients of the current block as information related to the residual signal.

[0072] The reordering unit 515 can reorder the sequence of one-dimensional quantized transform coefficients entropy decoded by the entropy decoding unit 510 into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning order performed by the video encoding device.

[0073] The inverse quantization unit 520 inverse quantizes the quantized transform coefficients using a quantization parameter. The inverse quantization unit 520 may apply different quantization coefficients (scaling values) to the two-dimensionally arranged quantized transform coefficients. The inverse quantization unit 520 performs inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video encoding device to the two-dimensional array of quantized transform coefficients.

[0074] The inverse transform unit 530 inversely transforms the dequantized transform coefficients from the frequency domain to the spatial domain to restore a residual signal, thereby generating a residual block for the current block.

[0075] In addition, when the inverse transform unit 530 inverse transforms only a portion (sub-block) of the transform block, it extracts a flag cu_sbt_flag indicating that only the sub-block of the transform block has been transformed, the sub-block's directionality (vertical / horizontal) information cu_sbt_horizontal_flag, and / or the sub-block's position information cu_sbt_pos_flag, and restores the residual signal by inverse transforming the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain, and generates a final residual block for the current block by filling the residual signal with ``0'' values ​​for the areas that are not inverse transformed.

[0076] Also, when MTS is applied, the inverse transform unit 530 determines the transform function or transform matrix to be applied in the horizontal and vertical directions, respectively, using the MTS information mts_idx signaled from the video encoding device, and performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions using the determined transform function.

[0077] The prediction unit 540 includes an intra prediction unit 542 and an inter prediction unit 544. The intra prediction unit 542 is activated when the prediction type of the current block is intra prediction, and the inter prediction unit 544 is activated when the prediction type of the current block is inter prediction.

[0078] The intra prediction unit 542 determines the intra prediction mode of the current block from among multiple intra prediction modes from the syntax element for the intra prediction mode extracted from the entropy decoding unit 510, and predicts the current block using reference pixels surrounding the current block according to the intra prediction mode.

[0079] The inter prediction unit 544 determines the motion vector of the current block and the reference picture to which the motion vector refers using the syntax element for the inter prediction mode extracted from the entropy decoding unit 510, and predicts the current block using the motion vector and the reference picture.

[0080] The adder 550 reconstructs a current block by adding a residual block output from the inverse transform unit and a predicted block output from the inter or intra predictor. The reconstructed pixels in the current block are used as reference pixels when intra predicting a block to be decoded later.

[0081] The loop filter unit 560 includes a deblocking filter 562, an SAO filter 564, and an ALF 566 as in-loop filters. The deblocking filter 562 deblocks and filters the boundaries between reconstructed blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking to compensate for differences between reconstructed pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about the filter coefficients decoded from the bitstream.

[0082] The reconstructed blocks filtered through the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in a memory 570. Once all the blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter-predicting blocks in a picture to be coded later.

[0083] This embodiment relates to the encoding and decoding of video as described above. More specifically, a video coding method and apparatus for performing a non-separable primary transform (NSPT) based on an intra prediction mode of a current block, a size of a transform block, and characteristics of transform coefficients is provided. In addition, the video coding method and apparatus according to this embodiment performs a non-separable primary transform using implicit partitioning for large transform blocks to which a non-separable transform cannot be applied.

[0084] The following embodiment is performed by the transform unit 140 and the inverse transform unit 165 in a video encoding device, and by the inverse transform unit 530 in a video decoding device.

[0085] The video encoder generates signaling information related to this embodiment from the viewpoint of optimizing bit rate and distortion when encoding the current block. The video encoder encodes the signaling information using the entropy encoder 155 and then transmits it to the video decoder. The video decoder decodes the signaling information related to decoding of the current block from the bitstream using the entropy decoder 510.

[0086] In the following description, the term "current block" may be used synonymously with a current block or a coding unit (CU), or may refer to a portion of a coding unit.

[0087] A flag's value being true means that the flag is set to 1. A flag's value being false means that the flag is set to 0.

[0088] I. Conversion Technology - Primary Conversion Technology As described above, for efficient video compression, quantization or scaling is additionally applied to the residual signals remaining after prediction using various prediction techniques. At this time, scaling is performed after applying a transform technique based on the importance of cognitive visual information contained in the residual signal to concentrate the residual signal on one side according to frequency components. However, for non-natural signals such as screen contents, such frequency-based transform techniques are inefficient. In such cases, the transform technique is omitted and only scaling is performed, or encoding / decoding is performed without applying scaling.

[0089] When a transform is applied in HEVC, DCT-II is used as the transform kernel (hereinafter, referred to interchangeably as the transform type) for transforming the residual signal. However, Multiple Transform Selection (MTS) is used to apply a more appropriate transform technique depending on the diversity of residual signal characteristics. MTS determines one or two optimal types from multiple transform types and then transforms the block according to the determined transform type. For example, in VVC, as shown in Table 1, two different transform types, DCT-VIII and DST-VII, are added in addition to DCT-II, allowing the residual signal to be transformed in a variety of ways.

[0090] [Table 1]

[0091] Here, the basis functions constitute the transformation matrix that defines each transform type. Hereinafter, DCT-II, DCT-VIII, and DST-VII will be used interchangeably with DCT2, DCT8, and DST7, respectively.

[0092] Meanwhile, the flag that determines whether or not to use the MTS can be controlled on a block-by-block basis, and whether or not to use the MTS may be controlled using an activation flag at a higher SPS level.

[0093] When MTS is enabled in SPS, a CU level flag indicating whether MTS is applied is displayed. Here, MTS is applied to the luma component. The CU level flag is displayed if both the width and height of the TB are 32 pixels or less and the Coded Block Flag (CBF), which indicates whether there is a non-zero value among the transform coefficient levels, is true.

[0094] If the CU level flag is 0, DCT2 is used as the kernel for both the horizontal and vertical directions. On the other hand, if the CU level flag is not 0, MTS is applied. There are two types of MTS: explicit MTS and implicit MTS.

[0095] In explicit MTS, the kernel to be used for TB is explicitly transmitted. Generally, the index of the transformation kernel is transmitted. For example, mts_idx, which is the kernel index, is defined as shown in Table 2.

[0096] [Table 2]

[0097] Here, trTypeHor and trTypeVer represent the horizontal and vertical transform types, respectively, and 0 represents DCT2, 1 represents DCT7, and 2 represents DCT8.

[0098] On the other hand, in the case of an implicit MTS, for example, in the case of an intra block, the transform type is implicitly determined without explicitly signaling the MTS. In the case of VVC, the horizontal and vertical transform types are implicitly determined as shown in Equation 1.

[0099]

number

[0100] Here, nTbW and nTbH represent the horizontal and vertical lengths of the transform block, respectively.

[0101] For example, when a specific coding technique is applied, explicit MTS or implicit MTS is applied. For example, in the case of MIP (Matrix-weighted Intra Prediction), explicit intra MTS is used. In the case of ISP (Intra Sub-Partitions) mode, implicit intra MTS is used, and DST7 or DCT2 is used as the transform type.

[0102] On the other hand, if the transform block includes at least one non-DC coefficient, mts_idx is signaled, i.e., if the position of the last significant coefficient in the scan order is greater than 0, mts_idx is signaled. On the other hand, if the transform block includes only one non-DC coefficient, mts_idx signaling is omitted, mts_idx=0 is set, and DCT2 is applied as the transform kernel.

[0103] The first bin of the signaled mts_idx indicates whether mts_idx is greater than 0. If mts_idx is greater than 0 (i.e., mts_idx indicates either 1 or 4), an additional 2-bit fixed length code is signaled to indicate the signaled mts_idx out of the four candidates.

[0104] Meanwhile, the next-generation technology, Enhanced Compression Model (ECM) software, increases the number and types of MTS kernels, adding DST7, DCT8, DCT5, DST4, DST1, and identity transforms.

[0105] II.LFNST(Low-frequency Non-separable Transform) During intra prediction, the LFNST technique performs a secondary transform on the low-frequency region of transform coefficients generated by a primary transform of a transform unit (TU). In terms of coding, the LFNST technique performs a secondary transform on L low-frequency primary transform coefficients among the W×H primary transform coefficients to generate K (where K≦L) secondary transform coefficients. Here, the size of the LFNST transform kernel is L×K. That is, the LFNST technique represents the L low-frequency primary transform coefficients among the W×H primary transform coefficients as 1×L vectors and then applies the L×K transform kernel to generate 1×K vectors. The LFNST technique then represents the 1×K vectors as a two-dimensional array in the low-frequency region for subsequent processes such as quantization.

[0106] Compared to linear transformations that apply separate transformation kernels in the horizontal and vertical directions, LFNST technology performs a non-separable transform that transforms one-dimensional vectors.

[0107] Meanwhile, the type of transform kernel is determined by the intra prediction mode of the current TU, the size of the TU, and the LFNST index (lfnst_idx). For example, a transform kernel set is determined as shown in Table 3 according to the intra prediction mode (IntraPredMode) of the current TU.

[0108] [Table 3]

[0109] Here, the intra prediction mode (IntraPredMode) is as shown in FIG. 3b. Furthermore, lfnstTrSetIdx is an index indicating a kernel set. In Table 3, when IntraPredMode is 81, 82, or 83, it represents a CCLM (Cross-component Linear Model) prediction mode.

[0110] Two types of kernels are defined for each kernel set (lfnstTrSetIdx). The LFNST index determines which of the two kernels to select. An LFNST index of 0 means that LFNST is not performed, while an LFNST index of 1 or 2 applies a different LFNST kernel within the same kernel set. There is one more kernel set depending on the TU size, so there are a total of 4 x 2 x 2 = 16 transformation kernels. The LFNST kernel sizes are defined as 16 x 16 and 16 x 48. The kernel size is adjusted according to the TU size as shown in Table 4.

[0111] [Table 4]

[0112] On the other hand, if a DCT2 / DCT2 transform kernel is applied to an intra-predicted TU as a primary transform, the LFNST technique is applied as a secondary transform.

[0113] Although the following embodiments will be described mainly with respect to a video decoder, they may be implemented in the same or similar manner in a video encoder.

[0114] III. Examples of the Present Disclosure FIG. 6 is a block diagram showing in detail a portion of a video decoding device according to one embodiment of the present disclosure.

[0115] The video decoding apparatus according to this embodiment determines a prediction and transform unit, and performs prediction and inverse transform on a current block corresponding to the determined unit using the determined prediction technique and prediction mode, thereby ultimately generating a reconstructed block of the current block. The example illustrated in FIG. 6 is performed by the entropy decoding unit 510, the inverse quantization unit 520, the inverse transform unit 530, the prediction unit 540, and the adder 550 of the video decoding apparatus. Meanwhile, the same operations as those illustrated in FIG. 6 are performed by the inverse quantization unit 160, the inverse transform unit 165, the picture division unit 110, the prediction unit 120, and the adder 170 of the video encoding apparatus. Herein, the video decoding apparatus uses coding information parsed from the bitstream, but the video encoding apparatus can use coding information set from a higher level in terms of bit rate and distortion minimization. Hereinafter, for convenience, this embodiment will be described focusing on the video decoding apparatus.

[0116] As illustrated in FIG. 5, the prediction unit 540 includes an intra prediction unit 542 and an inter prediction unit 544 according to the prediction technique, while as illustrated in FIG. 6, the prediction unit 540 includes a prediction mode determination unit 602 and a prediction performing unit 604.

[0117] 6, the sub-block division unit 606 is part of the entropy decoding unit 510, the inverse transform unit 530, or the prediction unit 540. In terms of a video encoding device, the operation of the sub-block division unit 606 is performed by the inverse transform unit 165, the picture division unit 110, or the prediction unit 120.

[0118] When the color format of the input video is a YUV format (YUV420, YUV411, YUV422, YUV444, etc.), the video decoding apparatus predicts and restores the luma component first, and then predicts and restores the chroma component. That is, the luma component and the chroma component are sequentially restored using the components illustrated in Fig. 6. Here, in the case of a YUV format, the color format represents the correspondence between the pixels of the luma component and the pixels of the chroma component.

[0119] The prediction mode determination unit 602 determines a prediction technique for the current block (e.g., intra prediction, inter prediction, IBC (Intra Block Copy) mode, palette mode, etc.). The prediction mode determination unit 602 also determines a detailed prediction mode for the prediction technique. The prediction performer 604 generates a prediction block for the current block according to the determined prediction technique and prediction mode.

[0120] The inverse quantization unit 520 inverse quantizes the decoded quantized transform coefficients for the current transform block to generate an inverse quantized signal. The inverse quantization unit 520 performs inverse quantization using one or more inverse quantizers. When multiple inverse quantizers are used, the number of inverse quantizers is N. q If so, the video encoding device and the video decoding device Nq The inverse quantizer is selected based on a state machine having identical states. At this time, the inverse quantizer is selected based on the current state and the LSB (Least Significant Bit) of the previous transform coefficient value. k is the LSB of the previous transform coefficient value, and N q If .times. ...

[0121] [Table 5]

[0122] Also, k is the LSB of the previous transform coefficient value, and N q If .times. ...

[0123] [Table 6]

[0124] The sub-block division unit 610 divides the current block into sub-blocks based on a sub-block division activation flag, a division method flag (or index), and / or a division direction (vertical or horizontal) flag, and the aspect ratio / width / height of the current block. According to an embodiment, parsing of the division method flag and / or division direction flag (or index) according to the aspect ratio / width / height of the current block is omitted, and the division method and / or division direction are implicitly induced according to the agreement between the video encoding device and the video decoding device. Here, flags and indexes for guiding sub-block division are defined according to the prediction technique (inter or intra) of the current block. Furthermore, prediction and / or transformation are performed on a sub-block basis.

[0125] The inverse transform unit 530 inverse transforms the TU represented by the inverse quantized signal to generate a residual signal.

[0126] The adder 550 adds the predicted block and the residual signal to generate a reconstructed block, which remains stored in memory and is subsequently used to predict other blocks.

[0127] Hereinafter, TU is used interchangeably with transform block.

[0128] 6, the inverse transform unit 530 includes all or part of an inverse transform kernel determination unit 610, a transform unit determination unit 612, and an inverse transform performing unit 614. The inverse transform unit 530 uses these components to perform a non-separable primary inverse transform (NSPIT) on the transform coefficients.

[0129] The inverse transform kernel determination unit 610 determines the type of inverse transform kernel for the current transform block according to the example of FIG. 7 or FIG.

[0130] FIG. 7 is an exemplary diagram illustrating a method for determining an inverse transform kernel according to an embodiment of the present disclosure.

[0131] For example, the inverse transform kernel determination unit 610 parses explicit_transform_flag (hereinafter referred to as "explicit transform flag"), which is a flag indicating whether to use an explicit inverse transform kernel. If the parsed explicit_transform_flag is 0, the horizontal and vertical inverse transform kernels are implicitly determined as DCT-2. On the other hand, if explicit_transform_flag is 1, the inverse transform kernel determination unit 610 additionally parses NSPT_flag (hereinafter referred to as "non-separable linear transform flag" or "NSPT flag"), which is a flag indicating whether to apply a non-separable linear inverse transform. If NSPT_flag is 0, the horizontal and vertical inverse transform kernels are explicitly determined by the MTS. The inverse transform kernel determination unit 610 parses mts_idx to obtain a horizontal and vertical transform kernel pair indicated by mts_idx. In this case, the vertical / horizontal kernel pair is defined according to the agreement between the video encoding device and the video decoding device. When NSPT_flag is 1, the inverse transform kernel determination unit 610 additionally parses NSPT_idx (hereinafter referred to as "non-separable linear transform index" or "NSPT index") to determine the type of non-separable linear inverse transform kernel. That is, when the kernel set includes two or more inverse transform kernels, the inverse transform kernel determination unit 610 parses NSPT_idx and then selects the linear inverse transform kernel indicated by NSPT_idx from the kernel set. In this case, the kernel set is selected based on the intra prediction mode of the current block, the size of the transform block, etc. Alternatively, when the kernel set includes one inverse transform kernel, parsing of NSPT_idx is omitted, and the non-separable linear inverse transform kernel is set as the single inverse transform kernel.

[0132] FIG. 8 is an exemplary diagram illustrating a method for determining an inverse transform kernel according to another embodiment of the present disclosure.

[0133] As another example, the inverse transform kernel determination unit 610 first parses NSPT_flag to determine whether to apply a non-separable inverse transform. If NSPT_flag is 1, the inverse transform kernel determination unit 610 additionally parses NSPT_idx to determine the type of non-separable linear inverse transform kernel. If NSPT_flag is 0, the inverse transform kernel determination unit 610 parses explicit_transform_flag. If the parsed explicit_transform_flag is 0, the horizontal and vertical inverse transform kernels are implicitly determined as DCT-2. On the other hand, if explicit_transform_flag is 1, the horizontal and vertical inverse transform kernels are explicitly determined. The inverse transform kernel determination unit 610 parses mts_idx to obtain the horizontal and vertical transform kernel pair indicated by mts_idx.

[0134] For example, a transform kernel of a current transform block is determined according to mts_idx, and if the current block is predicted by an intra prediction mode and / or an additive combination of an intra prediction mode and inter prediction, an MTS kernel candidate list is determined based on the intra prediction mode of the current block and the size of the current transform block. That is, the kernel indicated by mts_idx is changed based on the intra prediction mode, the size of the transform block, etc.

[0135] For example, for the current transform block, the MTS list is determined based on the sum of absolute values ​​of the quantized transform coefficients decoded by the entropy decoder 510, the sum of absolute values ​​of the transform coefficients dequantized by the inverse quantizer 520, or the position (lastScanPos) where the first non-zero transform coefficient exists. Here, lastScanPos is formed according to the scan order, and the scan order is defined by an agreement between the video encoder and the video decoder.

[0136] For example, the number of MTS lists is determined based on the lastScanPos value as follows: Depending on the embodiment, the number of sets (the number of MTS lists) determined based on the threshold and the number of transformation kernels included in each set may vary.

[0137] Transformation kernel candidates for MTS: {K0, K1, K2, K3, K4, K5} List candidate set 0:{K0}, if lastScanPos ≦ th0 List candidate set 1: {K0, K1, K2, K3}, if th0 < lastScanPos ≦ th1 List candidate set 2: {K0, K1, K2, K3, K4, K5}, if lastScanPos > th1

[0138] In this case, the threshold is defined by an agreement between the video encoding device and the video decoding device. If the size of the list candidate set determined by the threshold is 1, the signaling of mtx_idx for determining the kernel is omitted.

[0139] As another example, the inverse transformation kernel determination unit 610 parses MTS_ver_idx and MTS_hor_idx to determine the kernels in the vertical and horizontal directions. MTS_ver_idx and MTS_hor_idx indicate the kernels in the vertical and horizontal directions, respectively.

[0140] On the other hand, when a non-separable linear inverse transform is applied, a kernel set for the non-separable linear inverse transform is determined based on the intra prediction mode of the current block and / or the size of the current transform block. W , Tb H ≦maxNSPT" (Tb W ×Tb H Assume that there are M kernel sets based on the transform block size and intra prediction mode, and C inverse transform kernel candidates for each kernel set. Here, minNSPT and maxNSPT are defined by an agreement between a video encoding device and a video decoding device.

[0141] For example, the intra prediction modes illustrated in FIG. 3b are divided into six mode sets as shown in Table 7.

[0142] [Table 7]

[0143] According to Table 7, the number of kernel sets, M, is defined as 6 × T. That is, the kernel sets are divided into M sets based on the size of the transform block and the intra prediction mode (directional / non-directional mode, and in the directional mode, the prediction direction).

[0144] As an example, using the symmetry of a square block, the mode set of Table 7 is partitioned into mode sets as shown in Table 8 for a square block.

[0145] [Table 8]

[0146] When divided into mode sets as shown in Table 8, mode m and mode 68-m are included in the same mode set. Therefore, for mode m and mode 68-m, the inverse transform kernel determination unit 610 uses the same kernel set for the non-separable linear transform.

[0147] In Tables 7 and 8, modes -1 to -14 and modes 67 to 80 correspond to wide-angle prediction modes. The wide-angle prediction modes are either classified into a separate prediction mode set or included in the same prediction mode set as the closest directional mode.

[0148] Furthermore, the matrix-based intra prediction modes may be included in the non-directional mode set (mode set 0) or may be classified into a separate set.

[0149] As another example, for a rectangular block, M kernel sets are divided based on the mode set according to Table 7. In this case, for a non-separable transform, a transform block having an A×B size and predicted in m mode uses the same kernel as a block having a B×A size and predicted in 68-m mode.

[0150] For example, for a current transform block, the number C of inverse transform kernel candidates is adaptively determined for each kernel set based on the sum of absolute values ​​of quantized transform coefficients decoded by the entropy decoding unit 510, the sum of absolute values ​​of transform coefficients dequantized by the inverse quantization unit 520, or the position (lastScanPos) where the first non-zero transform coefficient exists. Based on one or more thresholds, the video decoding apparatus determines the number of inverse transform kernel candidates by comparing the sum of absolute values ​​or lastScanPos with the thresholds.

[0151] For example, the number of inverse transformation kernel candidates is determined by comparing lastScanPos with a threshold th. If lastScanPos is equal to or less than the threshold, the inverse transformation kernel determination unit 610 sets the number of inverse transformation kernel candidates to N0 (an integer equal to or greater than 0). On the other hand, if lastScanPos is greater than the threshold, the inverse transformation kernel determination unit 610 sets the number of inverse transformation kernel candidates to N1 (an integer equal to or greater than 1).

[0152] Thereafter, the inverse transformation kernel determination unit 610 parses the NSPT_idx and determines, from among the inverse transformation kernel candidates, the candidate indicated by the parsed NSPT_idx as the non-separable first-order inverse transformation kernel. If there is one inverse transformation kernel candidate, that candidate is determined as the non-separable first-order inverse transformation kernel without additional parsing of the NSPT_idx.

[0153] If the width and / or height of the current transform block is greater than maxNSPT, which is the maximum applicable size of a non-separable linear inverse transform, the transform unit determination unit 612 implicitly divides the current transform block until the width and height of the block become smaller than maxNSPT.

[0154] As an example, if one of the width or height of the current transform block (or transform sub-block) is greater than maxNSPT, the transform unit determination unit 612 divides the current transform block (or transform sub-block) as follows.

[0155] The width of the current transformation block is T W (or the width of the transform sub-block sbT W If the height T of the current transform block is greater than maxNSPT, the transform unit determination unit 612 recursively performs SPLIT_BT_VER division (vertical BT division) until the width of the divided transform block becomes equal to or less than maxNSPT. H (or the height of the transformation sub-block sbT H If the height of the divided transform block is greater than maxNSPT, the transform unit determination unit 612 recursively performs SPLIT_BT_HOR division (horizontal BT division) until the height of the divided transform block is equal to or less than maxNSPT.

[0156] As another example, if the width and height of the current transform block (or transform sub-block) are all greater than maxNSPT, the transform unit determination unit 612 divides the current transform block into four sub-blocks using SPLIT_QT partitioning.

[0157] For example, when maxNSPT=16, the conversion unit determination unit 612 determines T W =64, T H 9. In addition, when maxNSPT=16, the transform unit determination unit 612 implicitly divides T W =32, T H The =64 transform block is implicitly divided as shown in the example of FIG.

[0158] Meanwhile, the divided sub-blocks are sequentially inverse transformed and restored according to the z-scan order. At this time, the inverse transform order of the sub-blocks is determined based on the intra-prediction mode of the current block. If the intra-prediction mode of the current block is a vertical mode (mode 50) or a specific mode k (k>50), the sub-blocks are inverse transformed using the z-scan order in which the starting block is the upper right sub-block, as shown in the left example of Figure 11. If the intra-prediction mode is a horizontal mode (mode 18) or a specific mode k (k<18), the sub-blocks are inverse transformed using the z-scan order in which the starting block is the lower left sub-block, as shown in the right example of Figure 11.

[0159] For example, if the width and / or height of the current transform block is larger than a specific size, maxNTsize, determined according to an agreement between the video encoding device and the video decoding device, parsing of NSPT_flag and NSPT_idx is omitted, and a non-separable transform / inverse transform is not performed. Also, if the width and / or height of the current transform block is smaller than minNSPT, parsing of NSPT_flag and NSPT_idx is omitted, and a non-separable transform / inverse transform is not performed. maxNTsize represents the maximum size of the transform block for which NSPT_flag is parsed, and maxNSPT represents the maximum size of the transform block to which a non-separable transform / inverse transform is actually applied. Furthermore, minNSPT represents the minimum size of the transform block to which a non-separable transform / inverse transform is actually applied.

[0160] Hereinafter, maxNTsize is referred to as the "preset maximum size of the NSPT." Also, maxNSPT is referred to as the "preset maximum applicable size of the NSPT," and minNSPT is referred to as the "preset minimum applicable size of the NSPT."

[0161] The inverse transform performer 614 performs an inverse transform based on the inverse transform kernel and the size of the transform block. The inverse transform performer 614 parses a secondary transform flag or a secondary transform index to determine whether to perform a secondary inverse transform (i.e., LFNST). The secondary transform index is parsed, and if the parsed index is 0, the inverse transform performer 614 does not perform a secondary inverse transform. If a secondary inverse transform is to be performed, the inverse transform performer 614 performs a secondary inverse transform on the dequantized secondary transform coefficients to restore the primary transform coefficients, and then performs a primary inverse transform on the primary transform coefficients to restore the residual signal. If a secondary inverse transform is not to be performed, the inverse transform performer 614 performs a primary inverse transform on the dequantized primary transform coefficients to restore the residual signal.

[0162] Depending on the embodiment, the secondary inverse transform and the inseparable primary inverse transform may not be performed simultaneously. That is, if the secondary inverse transform is determined to be performed by the secondary inverse transform flag or secondary inverse transform index parsing, the parsing of NSPT_flag and NSPT_idx is omitted. Alternatively, if NSPT_flag is 1, the secondary transform flag or secondary transform index is implicitly set to 0.

[0163] Width and height are T W , T H When a non-separable linear inverse transform is performed on a current transform block, the inverse transform performing unit 614 generates P-sized reconstructed transform coefficients as invT =(P)×(T W ×T H ) is multiplied by the inverse transformation matrix (i.e., the inverse transformation kernel) of Equation 2 to perform the inverse transformation.

[0164]

number

[0165] where:

number

number

[0166] On the other hand, P ≤ T W ×T H For example, the size of P is determined by a multiple of the coefficient group (CG). The size of the CG is W CG ×H CG The size of P varies, such as 4x4, 8x8, etc., depending on the embodiment. The size of P is determined based on the size of the current transform block.

[0167] As an example, P≦T W ×T H 12a and 12b, the inverse transform process of the reconstructed transform coefficients is expressed as shown in Fig. 12a and 12b. In this case, the size of P, the vectorization scan order of the reconstructed transform coefficients, and the packing method (i.e., scan order) of the reconstructed residual signals may vary depending on the embodiment.

[0168] When a non-separable linear inverse transform is applied to a square transform block, the inverse transform performing unit 614 performs an inverse transform on a block predicted by intra prediction mode m and a block predicted by mode 68-m using the same inverse transform kernel. At this time, the reconstructed transform coefficients are converted into a linear transform coefficient vector.

number

[0169] For example, for an intra prediction mode L (2≦L≦34) that is closer to the horizontal mode (mode 18) than the vertical mode, the inverse transform unit 614 performs vector transformation using the scan order 1 (or scan order 2) illustrated in FIG. 13. [Number] is generated. Also, for an intra prediction mode L (34 < L ≤ 66) closer in the vertical direction mode (mode 50), the inverse transform execution unit 614 uses scan order 2 (or scan order 1) to obtain the vector [Number] is generated.

[0170] When a non-separable first-order inverse transform is applied to a rectangular transform block, the inverse transform execution unit 614 has a size of A × B and performs a non-separable first-order transform on the transform block predicted in mode m (2 ≤ m ≤ 34) and the transform block predicted in mode 68 - m having a size of B × A using the same kernel. At this time, for the A × B block predicted in mode m (2 ≤ m ≤ 34), the inverse transform execution unit 614 uses the scan order 1 (or scan order 2) illustrated in FIG. 14 to obtain the vector [Number] is generated. Also, for the B × A block predicted in mode 68 - m, the inverse transform execution unit 614 uses the scan order 2 (or scan order 1) illustrated in FIG. 15 to obtain the vector [Number] is generated.

[0171] When the transform block of the current block is implicitly divided into sub-blocks by the transform unit determination unit 612, the prediction execution unit 604 performs intra prediction of the current sub-block using the restoration reference sample of the sub-block restored earlier according to the z scan order.

[0172] Hereinafter, a method for transforming and inverse-transforming the transform block of the current block will be described.

[0173] 16a and 16b are flowcharts illustrating a method for a video encoding device to transform a transform block according to one embodiment of the present disclosure.

[0174] The video encoding apparatus obtains a residual signal for a transform block of a current block (S1600).

[0175] The video encoding apparatus determines a non-separable linear transform kernel based on the size of the transform block, the intra prediction mode of the current block, and the characteristics of the quantized transform coefficients (S1602).

[0176] The video encoding apparatus selects a mode set including an intra prediction mode of the current block from among preset mode sets, and selects a kernel set based on the size of the transform block and the selected mode set.

[0177] If the kernel set includes one transform kernel candidate, the video encoding apparatus determines the one kernel candidate as a non-separable linear transform kernel.

[0178] When the kernel set includes multiple candidate transform kernels, the video encoder determines one of the multiple candidate transform kernels as a non-separable linear transform kernel from the viewpoint of bitrate-distortion optimization, and then encodes an index indicating the determined candidate.

[0179] The video encoding apparatus applies a non-separable linear transform kernel to the residual signal to generate first linear transform coefficients (S1604).

[0180] The video encoding apparatus generates a one-dimensional residual signal vector from all or part of the residual signal according to a predetermined first scan order based on the size and type of the non-separable linear transform kernel. The video encoding apparatus generates a linear transform coefficient vector by performing a matrix multiplication between the residual signal vector and the non-separable linear transform kernel. The video encoding apparatus generates linear transform coefficients by allocating the linear transform coefficient vector to transform blocks according to the predetermined scan order.

[0181] The video encoding device adaptively determines the number of inverse transform kernel candidates for the kernel set based on the sum of the absolute values ​​of the transform coefficients or the position where the first non-zero transform coefficient exists.

[0182] The video encoding apparatus determines a pair of linear transform kernels in the vertical and horizontal directions for the transform block (S1606).

[0183] The video encoding apparatus applies the vertical and horizontal linear transform kernel pairs to the residual signal to generate second linear transform coefficients (S1608).

[0184] The video encoding apparatus generates third linear transform coefficients by applying predetermined linear transform kernels in the vertical and horizontal directions to the residual signal (S1610).

[0185] The video encoding apparatus determines an NSPT flag based on the first, second, and third linear transform coefficients (S1612), where the NSPT flag indicates whether to apply a non-separable linear transform.

[0186] In terms of bitrate-distortion optimization, the video encoder determines the NSPT flag. For example, if the first primary transform coefficient is optimal, the video encoder sets the NSPT flag to true. On the other hand, if the second or third primary transform coefficient is optimal, the video encoder sets the NSPT flag to false.

[0187] The video encoding device encodes the NSPT flag (S1614).

[0188] The video encoding device checks the NSPT flag (S1616).

[0189] If the NSPT flag is true (YES in S1616), the video encoding device encodes the first primary transform coefficients (S1618). The video encoding device quantizes and entropy encodes the first primary transform coefficients to generate a bitstream of the first primary transform coefficients.

[0190] On the other hand, if the NSPT flag is false ("No" in S1616), the video encoder performs the following steps.

[0191] The video encoding apparatus determines an explicit transform flag based on the second and third primary transform coefficients (S1630), where the explicit transform flag indicates whether to use a pair of vertical and horizontal primary transform kernels.

[0192] In terms of bitrate-distortion optimization, the video encoder determines an explicit transform flag. For example, if the second primary transform coefficient is optimal, the video encoder sets the explicit transform flag to true. On the other hand, if the third primary transform coefficient is optimal, the video encoder sets the explicit transform flag to false.

[0193] The video encoding device encodes the explicit conversion flag (S1632).

[0194] The video encoder checks the explicit conversion flag (S1634).

[0195] If the explicit conversion flag is true (YES in S1634), the video encoding apparatus performs the following steps.

[0196] The video encoding apparatus encodes the index indicating the pair of linear transform kernels in the vertical and horizontal directions (S1636).

[0197] The video encoding apparatus encodes the second primary transform coefficients (S1638). The video encoding apparatus quantizes and entropy encodes the second primary transform coefficients to generate a bitstream of the second primary transform coefficients.

[0198] On the other hand, if the explicit transform flag is false ("No" in S1634), the video encoding device encodes the third primary transform coefficients (S1640). The video encoding device quantizes and entropy encodes the third primary transform coefficients to generate a bitstream of the third primary transform coefficients.

[0199] FIG. 17 is a flowchart illustrating a method in which a video decoding apparatus inversely transforms a transform block according to one embodiment of the present disclosure.

[0200] The video decoding apparatus obtains dequantized transform coefficients for a transform block of a current block (S1700).

[0201] The video decoding apparatus decodes the NSPT flag from the bitstream (S1702), where the NSPT flag indicates whether or not a non-separable linear transform is applied.

[0202] The video decoding device checks the NSPT flag (S1704).

[0203] If the NSPT flag is true (YES in S1704), the video decoder performs the following steps.

[0204] The video decoding apparatus determines a non-separable linear inverse transform kernel based on the size of the transform block, the intra prediction mode of the current block, and the characteristics of the quantized transform coefficients (S1706).

[0205] The video decoding apparatus selects a mode set including an intra prediction mode of the current block from among preset mode sets, and selects a kernel set based on the size of the transform block and the selected mode set.

[0206] The video decoding apparatus adaptively determines the number of inverse transform kernel candidates for the kernel set based on the sum of the absolute values ​​of the dequantized transform coefficients or the position where the first non-zero transform coefficient exists.

[0207] If the kernel set includes one inverse transform kernel candidate, the video decoding device determines the one kernel candidate as a non-separable linear inverse transform kernel.

[0208] Alternatively, if the kernel set includes multiple inverse transform kernel candidates, the video decoder decodes the NSPT index from the bitstream and determines the candidate indicated by the NSPT index as the non-separable linear inverse transform kernel from among the multiple inverse transform kernel candidates.

[0209] The video decoding apparatus performs a linear inverse transform by applying a non-separable linear inverse transform kernel to the transform coefficients to generate a residual signal (S1708).

[0210] The video decoding apparatus generates a first-order transform coefficient vector by packing all or a portion of the transform coefficients one-dimensionally according to a preset scan order based on the size and type of the non-separable inverse linear transform kernel. The video decoding apparatus generates a residual signal vector by performing a matrix multiplication between the first-order transform coefficient vector and the non-separable inverse linear transform kernel. The video decoding apparatus generates a residual signal vector by allocating the residual signal vector to a transform block according to a preset scan order.

[0211] On the other hand, if the NSPT flag is false ("No" in S1704), the video decoder performs the following steps.

[0212] The video decoding apparatus decodes the explicit transform flag from the bitstream (S1720), where the explicit transform flag indicates whether to explicitly use the horizontal and vertical inverse transform kernel pairs.

[0213] The video decoder checks the explicit conversion flag (S1722).

[0214] If the explicit conversion flag is true (“Yes” in S1722), the video decoder performs the following steps.

[0215] The video decoding apparatus decodes an index indicating a horizontal and vertical inverse transform kernel pair from the bitstream (S1724), where the index indicates one of multiple horizontal and vertical inverse transform kernel pairs through multiple transform selection.

[0216] The video decoding apparatus applies the pair of horizontal and vertical inverse transform kernels indicated by the index to the transform coefficients to generate a residual signal (S1726).

[0217] On the other hand, if the explicit transform flag is false (“No” in S1722), the video decoding apparatus applies horizontal and vertical predetermined linear inverse transform kernels to the transform coefficients to generate a residual signal (S1730).

[0218] Thereafter, the video decoding apparatus generates a reconstructed block of the current block by adding the residual signal and the predicted block of the current block.

[0219] In the flowcharts / timing diagrams of this specification, the processes are described as being executed sequentially, but this is merely an illustrative example of the technical idea of ​​one embodiment of the present disclosure. In other words, a person skilled in the art to which an embodiment of the present disclosure pertains can apply various modifications and variations by changing the order of execution of the processes described in the flowcharts / timing diagrams or by executing one or more of the processes in parallel, without departing from the essential characteristics of one embodiment of the present disclosure. Therefore, the flowcharts / timing diagrams are not limited to a chronological order.

[0220] It should be understood that the exemplary embodiments in the above description can be embodied in many different ways. The functions or methods described in one or more examples may be embodied in hardware, software, firmware, or any combination thereof. It should be understood that functional components described herein are labeled as "units" to particularly emphasize their independence of implementation.

[0221] Meanwhile, various functions or methods described in this embodiment may be embodied as instructions stored in a non-transitory recording medium that can be read and executed by one or more processors. Non-transitory recording media include, for example, any type of recording device that stores data in a form readable by a computer system. For example, non-transitory recording media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0222] The above description merely exemplifies the technical concept of the present embodiment, and various modifications and variations may be made by a person skilled in the art without departing from the essential characteristics of the present embodiment. Therefore, the present embodiment is intended to illustrate, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by such an embodiment. The scope of protection of the present embodiment should be interpreted by the claims, and all technical concepts within the scope equivalent thereto should be interpreted as being included in the scope of the present embodiment.

[0223] CROSS-REFERENCE TO RELATED APPLICATION This patent application claims priority to patent application number 10-2022-0124290 filed in Korea on September 29, 2022, and patent application number 10-2023-0105462 filed in Korea on August 11, 2023, the entire contents of which are incorporated by reference into this patent application. [Explanation of symbols]

[0224] 140 Conversion Unit 165 Inverse conversion unit 530 Inverse conversion unit 606 Sub-block division section 610 Inverse transformation kernel determination unit 612 Conversion unit determination unit 614 Inverse Conversion Unit

Claims

1. A method for reconstructing a current block performed by a video decoding apparatus, comprising: obtaining dequantized transform coefficients for a transform block of the current block; decoding a non-separable primary transform (NSPT) flag from the bitstream, the NSPT flag indicating whether a non-separable primary transform is applied; checking the NSPT flag; Including, If the NSPT flag is true, determining a non-separable linear inverse transform kernel based on the size of the transform block, the intra prediction mode of the current block, and characteristics of the dequantized transform coefficients; and performing an inverse linear transform by applying the non-separable inverse linear transform kernel to the transform coefficients to generate a residual signal; A method comprising:

2. Furthermore, if the NSPT flag is false, Obtaining a pair of vertical and horizontal linear inverse transform kernels for the transform block, either explicitly or implicitly; and applying the vertical and horizontal linear inverse transform kernel pair to the transform coefficients to generate the residual signal; 2. The method of claim 1, comprising:

3. The step of determining the non-separable inverse linear transform kernel comprises: selecting a mode set including an intra prediction mode of the current block from among a set of preset modes; and selecting a kernel set based on the size of the transform block and the selected mode set; 2. The method of claim 1, comprising:

4. The step of determining the non-separable inverse linear transform kernel comprises: The method of claim 3, wherein the number of inverse transform kernel candidates for the kernel set is adaptively determined based on the sum of absolute values ​​of the inverse quantized transform coefficients or a position value where a first non-zero transform coefficient exists.

5. further comprising the step of decoding an NSPT index when the kernel set includes a large number of inverse transform kernel candidates; The step of determining the non-separable inverse linear transform kernel comprises: The method according to claim 3, wherein the candidate indicated by the NSPT index is determined as the non-separable linear inverse transform kernel from among the multiple inverse transform kernel candidates.

6. The non-separable linear inverse transform kernel is 2. The method of claim 1, wherein the matrix has a size of "P x size of the transform block", the size of the transform block being defined as the product of the width of the transform block and the height of the transform block, and the size of P corresponds to the size of a one-dimensional vector into which the transform coefficients are packed.

7. The size of P is The method of claim 6, wherein the size is less than or equal to the size of the transform block and is determined based on the size of the transform block.

8. performing a step of decoding the NSPT flag when the width and height of the transformation block are equal to or greater than a predetermined NSPT applicable minimum size and equal to or less than a predetermined NSPT maximum size; 2. The method of claim 1, wherein the predetermined NSPT applicable minimum size represents a minimum size of a transform block to which the non-separable linear transform is applied, and the predetermined NSPT maximum size represents a maximum size of a transform block for decoding the non-separable linear transform flag.

9. performing a step of determining the non-separable linear inverse transform kernel when the width and height of the transform block are equal to or greater than the predetermined NSPT applicable minimum size and equal to or less than the predetermined NSPT applicable maximum size; The method of claim 8, wherein the predetermined NSPT applicable maximum size is less than or equal to the predetermined NSPT maximum size and represents a maximum size of a transform block to which the non-separable linear transform is applied.

10. The step of determining the non-separable inverse linear transform kernel comprises:

10. The method of claim 9, wherein, if the width and height of the transform block are greater than the preset NSPT applicable maximum size, the transform block is implicitly and recursively divided to generate the sub-blocks until the width and height of the sub-blocks are smaller than the preset NSPT applicable maximum size.

11. The sub-blocks are: The method of claim 10, wherein the inverse transform is performed sequentially according to a z-scan order, and the z-scan order is determined based on an intra-prediction mode of the current block.

12. The step of performing a linear inverse transformation includes: Packing all or a portion of the transform coefficients into one dimension according to a first predetermined scan order based on the size and type of the non-separable linear inverse transform kernel to generate a linear transform coefficient vector; performing a matrix multiplication between the linear transform coefficient vector and the non-separable linear inverse transform kernel to generate the residual signal vector; generating the residual signal by allocating the vectors of the residual signal to the transform blocks according to a second predetermined scan order; 2. The method of claim 1, comprising:

13. 2. The method of claim 1, wherein if the NSPT flag is true, a secondary transform flag or a secondary transform index is implicitly set to 0, and the secondary transform flag or the secondary transform index indicates whether or not a secondary transform is performed.

14. A method for encoding a current block performed by a video encoding apparatus, comprising: obtaining a residual signal for a transform block of the current block; determining a non-separable primary transform kernel based on the size of the transform block, the intra prediction mode of the current block, and characteristics of the quantized transform coefficients; applying the non-separable linear transform kernel to the residual signal to generate first linear transform coefficients; determining, explicitly or implicitly, a pair of linear vertical and horizontal transformation kernels for the transformation block; applying the linear transform kernel pair to the residual signal to generate second linear transform coefficients; A method comprising:

15. further determining a non-separable primary transform (NSPT) flag based on the first primary transform coefficient and the second primary transform coefficient, the NSPT flag indicating whether a non-separable primary transform is applied; encoding the NSPT flag; 15. The method of claim 14, comprising:

16. 16. The method of claim 15, further comprising the step of encoding the first primary transform coefficient or the second primary transform coefficient based on the NSPT flag.

17. A computer-readable recording medium storing a bitstream generated by a video encoding method, the video encoding method comprising: obtaining a residual signal for a transform block of the current block; determining a non-separable primary transform kernel based on the size of the transform block, the intra prediction mode of the current block, and characteristics of the quantized transform coefficients; applying the non-separable linear transform kernel to the residual signal to generate first linear transform coefficients; determining, explicitly or implicitly, a pair of linear vertical and horizontal transformation kernels for the transformation block; applying the linear transform kernel pair to the residual signal to generate second linear transform coefficients; A recording medium comprising:

Images